I always thought of the TED conference attendees as generous donors who are in return allowed to watch the taping of the content TED produces. The most important thing about TED are the videos, not the actual event. If viewed that way it’s hard to find elitism :)
I hope they are elitist. Elitism is good and elitism is true. If they keep people from attending or speaking who "aren't quite good enough" (as the linked techcrunch article alleges), then so much the better for them for holding up their standards.