I'm curious why people put so much effort into scaling Heroku when you could outperform the maximum number of Dynos with only a handful of AWS instances for a fraction the price. It doesn't make sense. We're literally working around a problem that shouldn't exist.
Doesn't it make more sense to scale based on New Relic response times? Why are you guys using # of users? Depending on the page they are requesting and how the requests are clustered that could produce vastly inferior results.
It's kind of surprising that Heroku hasn't cracked this nut on their own. Perhaps it isn't in their best interest to scale down? Scalability certainly is often touted as one of the benefits of "the cloud".