I kind of agree with Caroline, but with some additional details. Each technology (say .net vs Java) is strong in certain areas and weak in certain areas. Having said that I feel one of the main driving force would be the technology adoption of your probable clients. Also newer technologies take longer to adopt, which might present you with a first mover advantage. So although the study might be useful for a Middle manager / developer perspective, I don't think it might have a great impact on the final decision making.

I think this approach can be very useful and important for projects that really need to squeeze every bit of performance, but not so much on those that don´t.

In many cases, difference between java and .net (to use the same examples) will be pretty much irrelevant since both technologies will be more than adequate for the job and the decision will be based on what the client is used to use.

I believe this is viable and companies would recognize the value, but on chosen scenarios only

Mathew... I agree with you, and thinking about that I add more information to this discussion.

Forget it about Java vs. .Net (too similar).

Situation 1) Suppose that a small, startup company, have to develop a map of the whole world to show several office locations from it first important client in the web. The map must be vectorial because the client want to make zoom, drag and drop, it need to handle click event and the client said that it must run mostly in every browser and in mobile devices (e.g ipad).

The company is between developing it using SVG technology or HTML5.

Situation 2) Suppose some company have to update thousand of pages from all over the world changing one charting solution based on flash with other based on JavaScript to make it compatible with ipad and other mobile devices (real situation, big client).

The company is looking for the right client framework that offer the best compatibility now and best scalability for tomorrow (they don't want to make this in 6 months again!!!).

For a moment, think that I have a "magic ball" and made a previous experiment comparing the technologies in both situations looking for compatibility and scalability issues.What if I give to that company the results of that experiment to help to lower the risk of the decision?

These new situations may be add more elements to this discussion. Thanks for the response!

To me at least in the first scenario, HTML5 would be the preference even though your study proves that SVG is better in performance . The reason is HTML5 is the buzz word now and business and clients would be more receptive to buzz words.

I agree with mathew. On the first scenario marketing and maybe team experience will matter more than which is technically better,

On the second scenario is where your study will probably be more important.

I feel, it defintely depends on company, whether it is small or big one, one has to consider the comprision for technology decision. It also plays imp role that, whether experiment carried out in-house or out sourced. As a

matter of fact if experiment is carried out in-house then it will be economical. If out sourced then naturally costly.

Further, it will remarkble affect the investment as at the initial stage only one can predict for the project cost.

1. Yes.

2. Depends on various factors. It easily get too expensive though.

3. Definitely.

4. Mostly B & C.

5. Between 30-50%, depending on the time the experiment is initiated, and unpredictable factors/alternatives/constraints.

Late revelation of alternatives and constraints is most of the times the main bottleneck of an experiment. If you perform it too soon then you might be missing critical input factors, and if you perform it too late then it might become quite costly. A firm should be very explicit and precise on the subject and desired outputs of such experimentation. A key decision is when to initiate it.

(1) to (5) A timely move to " Evidence Based Software Engineering" is the answer. (T. Dyba, B.A. Kitchenham, M. Jorgensen-IEEE Software DEC 2005)

Let me add my 2 cents to your questions. Actually there is a whole field called business intelligence where companies make intelligent decisions based on big data collected through their whole enterprise. You can tweet and join #BI.

following such approach will enrich the decision maker and uncovered hidden information and factors that are very significant for decision maker, yet from my own experience as an independent IT software development unit the values of these results are not always welcomed or it weigh very low compared to other approach. However, your approach could be the start to new look for adopting technologies and I strongly support it