If you're looking at doing this, especially if you're moving from Heroku and/or using Docker, seriously consider AWS ElasticBeanstalk. <a href="http://aws.amazon.com/elasticbeanstalk/" rel="nofollow">http://aws.amazon.com/elasticbeanstalk/</a><p>In case you don't know, ElasticBeanstalk is the AWS Heroku but _a lot_ more flexible. It's basically a layer on top of EC2 and ELB. It takes care of creating auto-scaled EC2 instances to your specification and deploying your application to it (with rolling updates!)<p>Out of the box, it supports PHP, Java, Python, Ruby, Node.js, .NET, Go, and Docker (so, everything else), and it's fairly trivial to set up a heroku-like git-based deploy with something like circle ci.
Suggesting that moving to Docker obviates the need for configuration management is frankly naive.<p>The whole point of CM tools is to make the layout/configuration of systems predictable. That doesn't make Docker <i>or</i> CM redundant. It means that when you build Docker containers, it makes good sense to install a CM tool and use it to do the systems configuration.<p>Saying that the Dockerfile works like BASH and thus makes life easier is a huge step backwards. Ultimately, administrators have to enter, troubleshoot and debug containers. Moving back to shell script-style configuration inside of containers just kicks the problem down the road.<p>Docker, and containers in general, are great. And you should treat their contents with the same respect that you do <i>any</i> system.
Would you mind sharing more details about the Docker setup on AWS?<p>"Our code deployment flow was straightforward, and mostly consisted of building a Docker image off our latest codebase and distributing it to our server instances, with the help of a Docker image server. We only needed to write a few short Chef scripts to download the latest Docker image build and start up a new Docker container."<p>I was wondering what did you mean downloading the latest Dockerfile and build the image, or what sort of building are you performing with an image if it was not an accident? The diagram suggest a slightly different setup.<p>I my mind, you build your docker images on the CI/CB system and push them to a repository, than you can deploy them with docker pull. I was wondering if you used the docker-registry service or went for Docker Hub? I finished migrating some of our stuff to AWS/Docker last week but we skipped Opsworks part. I am familiar with Chef though, I just think it is overly complicated and it is hard to debug if something goes sideways.<p>Anyways, thanks for sharing.
A year into this I'd love to hear a cost benefit analysis including infrastructure & ops cost, uptime, disaster recovery, security management (key rotation, heartbleed-like threat mitigation, etc), and overall developer happiness.
This is a great article and breaks down the components needed to run Docker in production on your own servers. We have been doing that in Cloud 66 - <a href="http://www.cloud66.com/" rel="nofollow">http://www.cloud66.com/</a> (on 8 cloud providers including AWS) for a while and it's been a great experience: <a href="http://blog.cloud66.com/docker-in-production-from-a-to-z/" rel="nofollow">http://blog.cloud66.com/docker-in-production-from-a-to-z/</a>
Did you know your site was blocked in Indonesia?<p>> This site is prohibited. The site is included in the list that should not be accessed through this network because contains porn, gambling, phising/malware, SARA or Proxy.<p>> In accordance with Indonesian Government Regulation refers to Law Number 36 of Year 1999 on Telecommunications Article 21, "Telecommunication provider is prohibited from conducting business telecommunications operation contrary to the public interest, morality, security or public order."
> With Heroku, you lose fine-grained control and visibility over your servers. Installing custom software in your server stack is far from straightforward, and it’s not possible to SSH into a server to debug a CPU or memory issue.<p>I found this annoying at first along with not being able to save files to the server. However, Heroku is forcing you to automate server setup so servers can be restarted, destroyed and created without losing data or functionality. If you allow SSHing in to install software and storing files on the server, you're probably going to get some nasty surprises when you need to recreate the server or add more servers for example.<p>You mention it was a couple of months of work to transition to your EC2+Docker setup. How much time do you find is required to maintain this compared to your Heroku setup? Heroku does seem expensive but I find it can have lower setup and maintenance costs than a custom solution.<p>Being able to run and test your Docker setup locally is a nice benefit though. I haven't found a satisfying solution to this for Heroku yet.
This post is mostly focused on migrating application instances. I wonder what's the best step-by-step way to move database from Heroku/EY/etc provider to Amazon .<p>And is it possible to first migrate app instances that would for some time use Heroku DB and then migrate the DB itself? Instead of doing everything in one big step?
If you are using Docker on AWS then EC2 Container Service (<a href="http://aws.amazon.com/ecs/" rel="nofollow">http://aws.amazon.com/ecs/</a>) would be an option. It's still in beta but looks promising.
It takes care all of the cluster management, launching, stopping etc.
I'm in a situation like this as well, using Heroku but not happy with the performance on the Dynos and looking to move to EC2 (we already use our own databases etc hosted on EC2).<p>The biggest issue is finding a solution which can give similar ease of use and functionality as Heroku. Doing a opsworks/chef setup like this isn't appealing - over time we want to have many microservices which can be scaled independently, and you don't want to be doing a lot of setup for each one. It's also complex to setup the kinds of scaling and rollbacks with git deploys like you can get on Heroku.<p>Right now Deis or Flynn looks best, giving most of the advantages of Heroku, but they are also pretty immature. Deis only just got the ability to send logs to a syslog instance, and can't be upgraded without either building a new cluster and migrating across (lots of work), or doing a in-place upgrade, which could mean up to 10 mins of downtime.<p>Flynn has no ability to send logs to an syslog server from what I can see, and seems even more immature than Deis, but looks like it might have a better technical foundation.<p>I've also been evaluating Amazon's Elastic Container Service, but that is unfortunately also very immature - No load balancing integration yet and also no information on logging, rollbacks, etc.<p>Elastic beanstalk is also interesting in a way, but we already have one app deployed on it using Docker and it's not great - logging is a kludge using a logger running in each container, and you either use your own docker repository (in which case you can't do rollbacks), or give them a zipfile with a Dockerfile and your app, which allows rollbacks but means the docker image gets built by every server in the cluster. It also doesn't seem to have any way of easily running and scaling multiple processes per app like you can with the Procfile on Heroku.<p>Anybody else in a similar situation? It seems that there are a bunch of interesting projects that will be very competitive with Heroku soon, but nothing that's really matured yet.<p>I wish Heroku would just introduce a new range of PX (running on their own EC2 instance) Dynos without the crazy pricing - right now the only PX dyno they have is $500 per month when you can get a substantially faster instance on EC2 for ~170 per month.
Since you folks serve real users in the real world (as opposed to being a much higher QPS product like a consumer oriented game or analytics or data service), I can't imagine your needs are extremely high in terms of QPS. Why not optimize other front-end experience, like serving one CSS/js file instead of many, serving assets from CDNs, etc? Cheers on moving to AWS and using Docker, though!
Interesting way to go about it. But how scalable would this solution be? I would love to hear your opinion on your implementation vs using the traditional configuration management tools. Which would perform better? My point being that in the future when you have scaled out to a lot more servers you shouldn't have to end up trying to move everything to chef later on.
I loved this article. It was very clear: the reasoning the high level step by steps were very helpful. +1 suggestions to follow up in 3/6/12 months. Also, I'd love to see a more detailed explanation, chef scripts, etc.
Docker on a single host is good enough for my develop/use, but I don't know whether swarm like tools is ready for real multiple-servers on production.
> it’s not possible to SSH into a server to debug a CPU or memory issue.<p>Not true. You can do all sorts of SSH-like things with 'heroku run bash'. Of course, your changes will be wiped when you end the session, but you can inspect things.