The way I remember it was a bit more complicated. IBM initial card size was originally based on the size of a bank note. To say the IBM just had 80 columns isn't enough, they also toyed with a 96 column card as well. And I wouldn't be surprised if first Jacquard and then Babbage are in this chain of causation also.
It's really interesting how standards just come out of stuff like this. Why are cars the size they are now? Because the Romans said that a cart can't be wider than two mules and designed their roads thusly, every bit of transportation technology in the past two thousand years has been affected by this rule. Even just yesterday I had to pick a length to cut news story previews to for a sidebar. I picked 140 chars solely because it looks "familiar" because of Twitter, which has that limit because of text messaging which has THAT limit because they guy who designed the protocol sat down at a typewriter and typed himself a bunch of sentences that were about that length.<p>Odd right?