For hyper environments like SV, probably because there is a shortage of hyper individuals in the general population.<p>For more general environments, it's probably not a shortage of engineers, just a shortage of engineers who will enthusiastically work at the rates that employers would prefer to pay.
This gem from Joel On Software (2000) blog is very relevant:<p><i>"Now, let's review some microeconomics. In a free market, it is almost axiomatic that the market always clears. That's a technical term that means that when somebody tries to sell something, if they are willing to accept the market price, they will be able to sell it, and when somebody wants to buy something, if they are willing to pay the market price, they will be able to buy it. It's just a matter of both sides accepting the market price.?"</i>
Almost any situation described as a labor shortage is actually just a complaint about the price of said labor. Offer a $10 million/year salary, and see how easy it is to get the engineers you're keen on. "That would be a ridiculously high salary", you say? Great, you've established my point; now we're just haggling...
It's a matter of money and career perspective. Being a good software developer requires a lot of learning. The job can be quite stressful and your knowledge devalues continuously because of tech change.
In the long run other professions probably are more attractive, don't require permanent re-learning and at least provide an option to climb up a career+$ ladder.
Provide better pay & conditions and you'll see more engineers.
Is there? <a href="http://www.theatlantic.com/education/archive/2014/03/the-myth-of-the-science-and-engineering-shortage/284359/" rel="nofollow">http://www.theatlantic.com/education/archive/2014/03/the-myt...</a>
The question is too general. There are shortage for certain requirements of engineer. For example experienced engineer to build self driving car is in shortage, or engineer understand high speed trading.<p>I don't think there is a shortage for engineer without much real world experiences.
More realistically Why is there shortage of engineers who are young, smart (no, not the looks), excellent on data structures and algorithms, has good knowledge on bigdata, loves analytics, has awesome great frontend development skills?
There are probably local shortages in certain areas like Silicon Valley, and a surplus in others. the shortage wouldn't be as bad if remote work were more common.
if you are asking why more people don't take up engineering I would tell you that you need to think what percentage of the population has the cognitive ability to write code day in/out and then filter the list for personal characteristics,hobbies,jobs that were given to them, equal opportunities etc. Then think of how much our society has switched focus to software development in the last decade.