There is no such thing as a conservative banking job at these companies. All of these firms have strongly cyclical business, resulting in aggressive hiring and regular layoffs, and think nothing of canning an entire department if they don't need it any more. Also, at higher levels, the odds of being sacked out of the blue for political reasons increase greatly (e.g. new VP arrives, fires the next tier of management and brings his friends from his last company).
Also, if you work in IT you risk being made the scapegoat for some failure outside your control (note the rating agencies trying to blame the subprime crisis on bugs in their ratings code). And if you do get canned, it's merciless - I've seen it happen to people on holiday, people who have just signed mortgages, and so on. These people are escorted from the building by security and then forgotten.
You also have to contend with the risk of burnout, one of the reasons why there are relatively few employees above the age of 30.<p>You have to do exceptionally well to receive the grand salaries and bonuses, as these are heavily skewed towards the minority of top-ranked employees. Plus, for every 'master of the universe' there are 10 people in compliance, legal, audit, helpdesk, facilities, hr, IT, back office, etc slaving away.<p>Going into wall st is an aggressive career choice.