There will always be needs which require private servers/clouds.

   The main question for most applications is not technical, but economic.

  Of course there will be servers, since that is what a cloud runs on. Now if you are asking whether all software will be cloud aware, then that is an economic issue.  Can software companies make money that way?

  For years, we had mainframe based computing, where all apps ran on a standard platform.  The apps were very expensive, because the software companies had only a few customers to charge.  

  Then came workstations and PC, where there was a broader base, and $100 per single user app became very common.

  If an economic model develops where the cloud suppliers can purchase and rent out apps, or where the cloud provider is like a grocery store or a consignment house, the going all cloud would be viable.  

  The last would be similar to a utility grid, where you purchase your electricity and get sent a bill. 

  I suspect large corporations and universities would set up their own cloud for business integrity/continuity and security reasons.

   Would you think the IRS will run on a non-private cloud, or air-traffic control?

   There will always be needs which require private servers/clouds.

   The main question for most applications is not technical, but economic.

  Of course there will be servers, since that is what a cloud runs on. Now if you are asking whether all software will be cloud aware, then that is an economic issue.  Can software companies make money that way?

  For years, we had mainframe based computing, where all apps ran on a standard platform.  The apps were very expensive, because the software companies had only a few customers to charge.  

  Then came workstations and PC, where there was a broader base, and $100 per single user app became very common.

  If an economic model develops where the cloud suppliers can purchase and rent out apps, or where the cloud provider is like a grocery store or a consignment house, the going all cloud would be viable.  

  The last would be similar to a utility grid, where you purchase your electricity and get sent a bill. 

  I suspect large corporations and universities would set up their own cloud for business integrity/continuity and security reasons.

   Would you think the IRS will run on a non-private cloud, or air-traffic control?

On the contrary, I think most of large corporations and universities would emigrate gradually to cloud, because of the cost reduction in terms of IT infrastructure and staff. As for IRS or Air-traffic control , I would suggest that they would pay more for extra security measures.  

  If you look at the US electrical grid as a model for this, then one could say that most people use centralized services rather than have their own generators.

  However, big data centers use UPS power to bridge to generator power for their most critical services,  likely computing would be the same.  On site (backup) computing power for critical infrastructure for when network interruptions occur, with automated switchover.  Hybrid private public clouds already exist for this kind of situation.

  Individual even have backup generation capabilities. (Backup gasoline generators or automated natural gas whole house backup generators.)

  Prices for cloud services will have to come down.  We already have heavy users who tried EC2 and have come back to campus for cheaper service.  Light users, or users of free services of course keep using those.  (Free advertizing supported email, etc.)

I would be curious to read a good analysis of why you (and others) think that Cloud is efficient.  Obviously, for very small organizations who only need a VM or two, Cloud is a very beneficial form of outsourcing.  But this is primary an effect of size: with very small needs, you can get rid of issues of hosting IT hardware, maintaining it, etc.  But on the other hand, outsourcing means you're inviting another company (or stack of them) to make profits from you.  Amazon, for instance, is incredibly profitable, since its prices are equivalent to buying the hardware new every year.

I suspect MBA types just swallow whole the assumption that Cloud/outsourcing provides economies of scale that dominate the equation.  The fact is, though, that IT costs, beyond quite small cases, are nearly linear.  If you have 10 racks or 100, your power is strictly linear, your floorspace is linear, your cooling overhead (for a given PUE) is linear. 

For all its hype, the Cloud is in part a re-implementation of 'glass house' computing that was centered around the mainframe (per James Coyle's comment).  As it is not just a single machine, it also suffers from all the security issues that have been solved for distributed computing (see Peter G Neumann's latest article in CACM - http://cacm.acm.org/magazines/2014/10/178776-risks-and-myths-of-cloud-computing-and-cloud-storage/abstract).

IT organizations looking to save budget might find a public cloud (or one provided by a 3rd party) advantageous until the day it fails.  The company where the IT organization lives doesn't care where the fault is, it will be deemed the IT organization's failure - and possible downfall.  A private cloud, run entirely by the IT organization, may solve the problem - but then there are only minimal savings because ownership of the servers and storage is still contained within the organization.

Cristian Chiliperia mentions the other major problem, data security.  In many instances, applications are shared between companies (or there is a standard application everyone uses), but the data generated is the key to the companies success (I'm thinking chemical/pharma companies).  Having this key proprietary data in "the cloud" would be tantamount to owning a diamond necklace and putting it in a vault where others also have the combination.

I think Jerry touches on an important aspect of IT outsourcing: risk.  IMO, the security issues are boring and not novel, but the issue of accountability should make people think carefully about their actual needs.

Suppose you outsource your legacy IT infrastructure (like the OP says).  And further suppose something goes wrong.  What do you do?  Your provider may fix your problem, but let's be honest: you are a flea to Amazon/etc.  Outsourcing means you no longer control your business processes.  This could be OK, if your provider is *perfect* (never fails) or surprisingly responsive.  But it's a fundamental loss of control that is inherent to outsourcing.  (Oh, outsourcing fans can argue that providers have an incentive to make customers happy, but this is an biz-school fantasy.  Outsourcing your floor-cleaning is different than outsourcing your IT, since the latter is highly non-fungible with lots of costs when you change providers.)

Still, PHBs may feel more comfortable reposing their trust in Amazon, rather than the local nest of IT orcs.  It's worth pointing out that any automation that makes Amazon responsive could be applied to internal IT infrastructure, too.  If PHBs are uncomfortable with how inhouse IT behaves, they should fix that problem, rather than paying someone else's inhouse IT...

One last thing: networking.  People who are "high on Clouds" tend to have a very unrealistic view of the state of networking: bandwidth, latency and cost.  Any cloud project needs to consider that clouds charge for (external) bandwidth, and that most existing IT infrastructure is built for intra-company performance.  There has never been a "Moore's Law of Networking"...

Mark Hahn , Jerrold (Jerry) Heyman and Cristian Chilipirea flagged a very vital and reasonable points about trust and security in public cloud computing. There are few other issues regarding performance, networking and privacy that worth further investigation. However, in discussing these "minor" details, we tend to forget that technology as driving force of the industry would have a very different say! Are we "users" going to stop that? The concept of renting on demand is likely to win over buying. The firm would look at this issue from broader perspective " Is it worthy to buy IT infrastructure  or just rent it"? 

Users like me and you are already using cloud, some small/medium IT systems started using cloud already even though some issues are still tangled in the process and the procedure of cloud computing. Why? Simply, because cloud computing is mainstream now and the positives outweigh the negatives. The strategy of most organization aiming success most like would consider emigrating towards cloud computing. This is technology push at its best. 

Almabruk, you asked about legacy IT.  The answer is that Cloud is simply more expensive than the traditional approach, at least for non-trivial installations.  There may be occasions, new and/or small companies, which override a factor of 2-3x in price, but you asked about legacy.  It's absurd to imagine large enterprises ripping out legacy IT in favor of exclusively using higher-cost Cloud services - after all, they would still need IT admins.  (In fact, they'll need vastly more networking...)  And for anything other than SaaS Cloud, you still need a lot of in-house expertise.  (IaaS, for instance, is nothing more than datacenter outsourcing: you still need sysadmins, just not box monkeys.)

When a vast quantity of one particular resource is required (e.g. storage space), cloud providers typically don't have very competitive pricing over investment into private servers. But the economic motivation isn't the only one.

Another reason to maintain private server farms is security. Cloud providers have their own security mechanisms, but when a system is critical enough (e.g. public services) a single point of failure if only in theory isn't permissible. A lot of somewhat critical services have already moved over to such providers and what we're seeing is that they're becoming primary targets for sabotage, in particular from nationstate actors.

The security isn't exclusively about coping with data breaches, though. Due to public trust, political stability, potential for domestic espionage etc., some services may want to maintain internal data storage. This could mean physically internal, i.e. in a guarded facility. It can also mean legally internal, such as in a trusted jurisdiction where certain legal protections apply and local authorities are reputable for adhering to them, or one chosen based on public trust.

Systems in e-governance are a good example of this: in certain east European jurisdictions it would make legal sense to store data under EU jurisdiction  in a country where these safeguards are maintained like the Netherlands or Germany - as opposed to say the UK (http://en.wikipedia.org/wiki/Regulation_of_Investigatory_Powers_Act_2000). However the suggestion of moving the data outside the jurisdiction of the country in question will rarely gain political support.

Dear Mark,

You are right when you mention that cloud adoption after a point becomes less rewarding - there is a non linear relationship between between posited cloud  gains and incremental cloud adoption.

But that is because the risks associated with cloud have not been fully mapped, segmented and monetized.

The technology is still in its nascence.