Goodbye, Google App Engine

Does no one read the documentation before deciding to use a platform anymore?<p><i>App Engine supports Python 2.5. The Python interpreter runs in a secured "sandbox" environment to isolate your application for service and security. The interpreter can run any Python code, including Python modules you include with your application, as well as the Python standard library. The interpreter cannot load Python modules with C code; it is a "pure" Python environment.</i><p>At the top of the FIRST page of documentation: <a href="http://code.google.com/appengine/docs/python/overview.html" rel="nofollow">http://code.google.com/appengine/docs/python/overview.html</a><p><i>Google Apps domains do not currently support HTTPS. HTTPS support is limited to apps accessed via </i>.appspot.com domains. Accessing an HTTPS URL on a Google Apps domain will return a "host not found" error, and accessing a URL whose handler only accepts HTTPS (see below) using HTTP will return an HTTP 403 "Forbidden" error. You can link to an HTTPS URL with the <i>.appspot.com domain for secure features, and use the Apps domain and HTTP for the rest of the site.</i><p>HIGHLIGHTED on <a href="http://code.google.com/appengine/docs/python/config/appconfig.html#Secure_URLs" rel="nofollow">http://code.google.com/appengine/docs/python/config/appconfi...</a><p><i>A request handler has a limited amount of time to generate and return a response to a request, typically around 30 seconds. Once the deadline has been reached, the request handler is interrupted.</i><p>and<p><i>While a request can take as long as 30 seconds to respond, App Engine is optimized for applications with short-lived requests, typically those that take a few hundred milliseconds. An efficient app responds quickly for the majority of requests. An app that doesn't will not scale well with App Engine's infrastructure.</i><p><a href="http://code.google.com/appengine/docs/python/runtime.html#The_Request_Timer" rel="nofollow">http://code.google.com/appengine/docs/python/runtime.html#Th...</a><p>I could go on and on.. reading this I see "I wasted 15000€ by not reading the documentation"<p>I usually think the title <i>technical architect</i> is a bit stupid (my business card says I'm one, so I can say that) but this guy needs a good technical architect to make platform decisions <i>prior</i> to wasting that much money

"developing on GAE introduced such a design complexity that working around it pushes us 5 months behind schedule"<p>This is the core of all their problems. It is a mindset incompatibility between these app designers and GAE.<p>The GAE APIs and rules are actually pretty simple and well defined. It works really well, but only if you work WITH those rules. You have to adopt the GAE application design philosophy.<p>If you don't, and if you work AGAINST the rules and best practices set by GAE then you are in trouble. Big trouble. This is what happened here.<p>I understand this is easy to say afterwards. And you can't really blame them for finding out the hard way.<p>Note that the same applies to for example all the great services that Amazon Web Services provides; they only work if you build your apps with the Amazon specific design approach in mind. Things like eventual consistency, expect things to fail, don't do large amounts of work in single jobs. Etc. Etc.<p>These appoaches suck more or less if you come from a 'total control over a bunch of machines' background. But they are so needed to scale.

I just launched my app 4 days ago (threewiki.com) and its backend is using GAE (Python + Tornado). On the second day, it was featured on TheNextWeb front page and the server got TONS of traffic. It scared me a bit, and I quickly change the daily max quota to $10. Anyways, GAE handles everything very gracefully. Over 70% of our users use either Facebook or Twitter to login and it hasn't been a problem at all.<p>I always think if your site can't be host by GAE, then it's probably not very scalable at the first place. I agree it might be better to host the data-processing end on EC2 or elsewhere if it's intense. Else in terms of the "View" part of your project, I wouldn't give it another thought for using GAE again.

It sounds like a lot of his problems stem from the use of Django. I've tried it before, and believe me, Django absolutely sucks on Appengine.<p>First off, you have a full featured framework which was designed for SQL relational databases. Many of Django's features either have to be given up, or are monkey-patched beyond belief to get partial functionality. Not to mention quite a few Django apps use database features which are simply not supported by BigTable.<p>Secondly, Django is not exactly the smallest framework, so loading time can be quite expensive and will be tacked on to every cold start.<p>All that being said, I've had good success with the tornado framework. It's fast, well written, and thoughtfully designed. Check out my profile if you want to see some examples of apps written with tornado + GAE.

There are a lot of apologists here, and they're all missing the point. Google App Engine has a large number of random issues and limitations which, while they each individually seem like they ought not to matter, add up to a substantial risk for a project running on it. I'm currently working on a project with a web component, and considered GAE. I chose not to use it, because some of the stuff I encountered in the documentation is absolutely terrifying. You're supposed to handle exceptions from the datastore? Really? The best you can possibly do is retry, but if that was going to work the library would be doing it automatically... so I guess it's saying that your app might just randomly fail sometimes. No https with a domain? I guess ever taking credit card numbers is off the table. Thirty second limit per request? Sounds reasonable, except for all the complaints about the app engine taking thirty seconds just to load the Python interpreter and compile their code. That sort of shit will sink a project; until App Engine has had a lot more time to sort out its issues and mature, I want no part of it.

A useful article for people to read before using AppEngine. I only use AppEngine for my own projects, so far no jobs for customers.<p>I was aware of most of the limitations of AppEngine that the author of the article mentions after just a few hours of experimenting with AppEngine. Now, AppEngine now no longer gives me many problems.<p>I think the lesson is to do a lot of experiments before committing to technologies.<p>I don't use the Python SDK. Most of what I have done has been using Java (but with small Clojure and JRuby experiments). One thing that helped was to start using Objectify instead of JDO (as an example).

All the points he makes are valid - and stem from the fact that GAE is designed to be 'infinitely scalable'. Because of this, it forces you into a number of design patterns that facilitate distributed, scalable software.<p>This can be a good thing, if you know scalability is going to be a killer feature in the near future. It can also a real pain in the ass if it's more important to simply get something off the ground quickly and see if it has market traction, and you don't want abandon the convenient but difficult to scale practices like long running processes and JOINs. In my experience, most startups fall squarely into this latter camp. Scalability is a nice problem to have for most of us.<p>AppEngine for Business now has a hosted SQL mode, which presumably uses a less scalable but ACID compliant alternative to the standard GAE data store (disclaimer - I haven't used it). Since he's already throwing down some serious coin on his app on GAE it might be worth investigating that before abandoning the platform completely.

The article mostly describes obvious, well documented limitations of GAE. It's really the story of a team that needed a saw, picked a hammer, and spent months trying to get it to cut wood.<p>This is a pretty common pattern in software, so it might be more interesting to write an article about why they chose the wrong technology and how they stuck with it even with it was clear it wasn't built to do what they needed.

While I feel the OP's pain in terms of difficulty working with the limitations of AppEngine, what he's not mentioning is the collection of benefits you get from AppEngine over other services. We've found that once you work out a solution to the datastore and request timeout issues, you have a remarkably robust and scalable system for free (or at least out-of-the-box).<p>With AppEngine, I've never had to migrate a database schema, build a load-balancer, hire a fulltime sysadmin, or even pay for servers that arn't receiving traffic. I don't have to set up a large-scale deployment system, nor spin up a new database server when traffic gets too heavy. AppEngine so far has been remarkably cheap (we're starting to bring in more customers however, so we'll see how long this lasts).<p>Many of the challenges he mentions come down to thinking about writing a webapp with a longer-term vision in mind. Datastore limitations crop up when you outgrow your first datastore in a standard system; in AppEngine they're properly enumerated and dealt with from day 1. Likewise long-running connections become very tricky to deal with with lots of traffic... this point is a little harder to argue with the recent popularity of asynchronous-io servers, but I think Google is working hard on these limitations. SSL is just annoying; we've had to deal with this by adding an SSL proxy until Google adds SSL support -- but it sounds like Google is pretty close to solving this one (it's been promised by end of year).<p>Also, AppEngine is written in a very high-level way; should you reach a point where AppEngine no longer makes sense, it is amazingly easy to transition over to another system (as the OP apparently found out; I would give more credit to the design patterns inherent in the AppEngine APIs than 'TDD driven development'). Tornado, webpy, etc have virtually the same interface as AppEngine's webapp framework.<p>There are definitely tradeoffs when choosing AppEngine as a production backend right now, and its certainly not the right solution for every problem... but for many people, us included, its been a pretty large net benefit for our startup. Google is actively improving the system, and I expect many of these problems will go away in the next 6 months or so.

A few of these points are just ridiculous:<p>1) If you want "SQL and Joins", use SQL. This is like complaining that you can't play Halo on Linux.<p>1A) There isn't full text search. If you need full text search, use a system with full text search as a feature.<p>2) Some of the points are out of date (or will be out of date soon). The 30 second limit for cron jobs will be 10 minutes after the next release. As noted, the 1000 results per query limit is gone already.<p>3) Anything can fail. If you assume your own system won't fail, you're going to be in worse shape later.<p>4) What objects would you cache that are &#62;1MB anyhow? In almost any case, you'd be better off caching it as multiple, smaller objects.

We have a product built on app engine, sure we've had our share of challenges but the benefits far outweigh them. To name a few benefits: A developer staff of 1 can focus on the application code and not the infrastructure details. Fast image serving auto-scales all your images to any size, and serves them off the fast servers used by Picasa. Versions can be used for testing multiple branches of code on production servers seamlessly.<p>#1 Has never been an issue for us #3 Is incorrect with the new task queue upgrades #6 We have a full-text system working just fine #7 Is a benefit when working with a distributed datastore #8 DB performance after the recent updates has been stunning #10 So they badly designed their queries and blame app engine? #11 Is flat out incorrect #12 What database is immune to failure? Would love to know<p>App Engine doesn't do everything, and no one is claiming it does. We have a secondary VPS we offload certain image processing tasks for example. But what it does do is extremely powerful from a develop perspective, and the application-centric model, like heroku or engine yard, is where things are headed. I would much rather leave the server and scaling issues to the experts so I can spend time improving my application.

I feel your pain, brother. I went through all of this a few years back. AppEngine was still pretty new back then, so it all seemed like things that they would fix before long. But it sure didn't feel ready for production.<p>I'm actually quite surprised that all those same limitations are still in place after all this time. I guess if I took a minute I could come up with an issue or two I had back then that has been fixed since, but his list of show stoppers are all things that people were complaining about, and that Google gave the impression of being on top of.

I agree that GAE has its share of problems. However, many of the complaints listed are about limits and limitations, and can be found in the documentation; at least points #1, 3, 4, 5, 6, 7, 11 and 11.<p>I developed an app with GAE about 2 years ago, and ran into many of the same problems (although some of the limits were probably lower then). Fortunately I could work around them, and the app wasn't used by tons of users anyway. I can see how it would be a serious problem otherwise, though.

We love AppEngine for many of the reasons which he gives. Because of its limitations, the platform makes you think and write code with proper design patterns in mind. If you just start to write code without thinking about it beforehand, yes you won't like AppEngine and it's not for you. It'll cost you LOTS of money and won't perform very well. On the other hand, by writing code designed for AppEngine we've been able to reduce our costs by several thousand $$ a month. Also a side benefit is that we migrated one of our IT roles to a development role, which means that we're able to iterate faster.<p>Granted we don't have a need for SSL and not being able to use C libraries in python has caused us many hours "pain", but compared to the alternative for a small company like we have, it's well worth it.<p>My biggest issue with AppEngine is that there's no full-text index functionality, and there's no way to create your own. We've tried everything, and nothing works if you have millions of documents like we have. Our search is still external to AppEngine but we're hoping that Google will do something about it sooner or later.

The biggest problem for me with GAE is cost. Currently I'm paying around $150 per week for around 50k daily active users (~6 million requests per day).<p>I'm almost sure, that i can run the same amount of traffic from $100/month dedicated server.

This hasn't been my experience.<p>Yes, existing techniques for full text search works or things like geolocation queries won't work but there are other[1] techniques[2] that work just as well; it's just not the sql way. Basically, support for multiple set membership queries against a list of tags stored with entities is extremely powerful and if you index properly, you can do a lot of cool things [3]. Plus, you can do datastore queries in parallel [4], which means you don't have to denormalize as much as you think; just parallelize and memcache results; e.g for a complicated front page, you can fetch different types of content in parallel.<p>The local server behaves remarkably the same as the deployed server, it's quite rare I find a situation where something behaves differently in production. the entire datastore can be tested locally, including complicated schemas / indices / queries in fast running unit tests. This means when I do need to do something fancy with the datastore, I can fully test it with unit tests and be confident it will work when deployed.<p>Long running tasks can always be broken up using the task queue. the limit will soon be 10 minutes for individual tasks and cron jobs [5]<p>I agree that cold start is a huge issue, but looks like it is being addressed in the 1.4 release [5] where you can pay for 3 reserve instances at all ties. Lack of support for https on your domain definitely sucks too, but I don't see how he wouldn't have been aware of that before going with GAE.<p>Finally, there are a number of things that are a huge time / money savers: - really easy deployment process including support for multiple versions. This let's you have staging instances and quickly roll back to a previous version if there are any problems - a nice admin console with a number of tools, including comprehensive access to logs that are coherent across all instances - some really nice libraries for examining performance of datastore queries and other api calls [6] and getting daily email reports of any exceptions [7]. these are built using hooks available to you in case you want to build something similar (for instance I used hooks to have regression tests on the number of datastore queries each page requires). - the services and apis made available are really nice. for instance, the image hosting infrastructure that provides fast access to different sizes for a stored picture based on a url is pretty slick; they basically opened up the same infrastructure that is used by picasaweb to app engine users - virtually no hosting costs until you get a lot of traffic. thousands of daily visitors is still in the free range<p>That said, my biggest outstanding gripes:<p>- cold start problem (until 1.4 is out)<p>- datastore latency spikes sometimes. this has gotten a lot better in the past few weeks, but I'll still have this gripe until I see it more consistent for a couple months<p>- no support for incoming emails with attachments &#62; 1mb (makes incoming photos from smart phones impossible since they are usually &#62; 3Mp these days)<p>- no support for long polling (upcoming channel API seems to be more for chat rooms than for general purpose server push) [8]<p>[1] <a href="http://www.billkatz.com/2009/6/Simple-Full-Text-Search-for-App-Engine" rel="nofollow">http://www.billkatz.com/2009/6/Simple-Full-Text-Search-for-A...</a><p>[2] <a href="http://fluffybunnysoftware.com/node/8" rel="nofollow">http://fluffybunnysoftware.com/node/8</a><p>[3] <a href="http://code.google.com/events/io/sessions/BuildingScalableComplexApps.html" rel="nofollow">http://code.google.com/events/io/sessions/BuildingScalableCo...</a><p>[4] <a href="http://code.google.com/p/asynctools" rel="nofollow">http://code.google.com/p/asynctools</a><p>[5] <a href="http://groups.google.com/group/google-appengine/browse_thread/thread/0f90ef8dda3b8400/b6dc86fafa4f6926#b6dc86fafa4f6926" rel="nofollow">http://groups.google.com/group/google-appengine/browse_threa...</a><p>[6] <a href="http://googleappengine.blogspot.com/2010/03/easy-performance-profiling-with.html" rel="nofollow">http://googleappengine.blogspot.com/2010/03/easy-performance...</a><p>[7]<a href="http://code.google.com/appengine/articles/python/recording_exceptions_with_ereporter.html" rel="nofollow">http://code.google.com/appengine/articles/python/recording_e...</a><p>[8] <a href="http://bitshaq.com/2010/09/01/sneak-peak-gae-channel-api" rel="nofollow">http://bitshaq.com/2010/09/01/sneak-peak-gae-channel-api</a>

the next version will allow you to pay to keep three copies of your app loaded and ready to go at all times. that should eliminate the insanely slow load times for apps that don't get constant traffic (at a price yet to be determined).<p>google is also adding sql database capabilities to the platform soon.<p>and google apps for business will eventually let you talk https on your own domain (at a price yet to be determined).<p>i suspect they're about 6 or 8 months from becoming a solid solution to many problems.

#11 isn't true. Since a recent update, you can now retrieve more than 1000 results in a single call.

if you're using Django, check out <a href="http://www.djangy.com" rel="nofollow">http://www.djangy.com</a> -- it's heroku for django (and eventually other wsgi frameworks)<p>:-)

I recently developed my first GAE site, and whilst I didn't run into the problems the author of this article did, I spent a lot of time hunting down best practises.<p>It's now been ages since I last looked at Amazon's offerings; does anything have any links to best practises / development strategies for either AWS or GAE?

Incredible how many points this article gathers considering the valid arguments made in the comments[1]. There must be a lot of bad experience (or fear) with this platform.<p>Personally, I recently had my fair share of problems with the DownloadError, but I knew it would come up (because I read the documentation - and I concur with the limit for scaling reasons). So I built myself a failover decorator relaying the failing requests to a VPS.<p>Otherwise, I love GAE!<p>1. For example the top commenter nl with 83 points: <a href="http://news.ycombinator.com/item?id=1928148" rel="nofollow">http://news.ycombinator.com/item?id=1928148</a>

#3 isn't true for taskqueue tasks or cron jobs anymore, the deadline is now 10 minutes ( actually will be once 1.4.0 SDK is released in about a week or so )<p>Also at first glance there is no indication of how the author got to a value of 15k€. My best guess, and a guess at that, is that they put the value of a line of code at 1€ and had to migrate 15k lines, but I hope there is more scientific than than.

IMO, i feel this is a design flaw. Without understanding what's a NOSQL used for you can't except joins to work.<p>Shouldn't there be a checklist to show "What applications can be moved to cloud (appengine here)? including the myths, expectations, assertions?"

i think in general what we should take from this article is one user's experience with gae and use it as just another data point. many comments on here seem to suggest that they're upset about the bashing, but this is just feedback, a grain of salt, i find it helpful, although noted that you could probably find most of these in the documentation.<p>i originally also considered gae for my project, but decided against it, my impetus was that i wanted to use a homegrown best of breed stack: tornado, mysql, nginx, memcached, python2.7 and have more control over the environment.

Did someone document their progress, or overview the development of a medium-sized app (~10K LoC) on GAE?

I'm a little bit confused about the number of arguments against the OP's post, here and especially on his own site.<p>As someone who's considering whether going w/GAE or custom, the points he makes are totally valid and applicable.

Really, REALLY I can't understand how this can be #1 on HN now. Ah, Carlos, please go RTFM! Powerful tools... they are not for everyone.

This is the worst critic I've ever read about App engine.

One thing not mentioned is the development server slowness and need to constantly restart it. The live website actually ends up being much faster than the development site on your machine. Apparently, this is due to the fact that the development server is single threaded. TyphoonAE is one work around but then you're depending on a third-party to catch up with the latest API changes.

It is not even in Beta yet, as it states in the documentation from Google;<p>"This is a preview release of Google App Engine."<p>My biggest gripe about GAE (and Google in general) is that when a change is made on Google's infrastructure that causes large problems, no acknowledgement (or answer) is made until enough people complain.

GAE is web scale!

i'd curious to hear what you switch to and, in about a year, how well it is working out.<p>my buddies and i are working on a time manager for college students and we are having a hard time deciding between RoR and GAE. any insight would be helpful.<p>we are looking for: login manager database capable of up to 50 fields for each user sorting and search capability must cost less than $10/year/user.<p>ps. if you are someone interested in solving this problem, please contact me directly at hn@sahajsingh.com

For all I know, GAE is really easy to develop on. I made this using Django in a couple of hours' time: <a href="http://bhoogolvidya.appspot.com/" rel="nofollow">http://bhoogolvidya.appspot.com/</a>