Hello,<p>I am researching how to automate the development environment setup for a team of 15+ developers that is currently manual and requires a lot of repetitive work. In addition to that I am researching how to automate testing, integration and deployment of these web apps to different environments (staging, and production).<p>How would you implement this huge workflow and how would you tackle it? Any success stories of automating the deployment of similar apps (Python, JS, SASS, Config files, git cloning of projects) in other companies/projects?<p>The current workflow is that each developer clones all the apps (30+) and sets them up locally via editing config files, then sets up Apache for some internal URLs, and then starts working on them. Finishes work, commits to remote development branch and from their everything is just manual work: review of pull-requests and merging branches...etc. Production releases are manual as well.<p>I am also interested in knowing how to setup/refactor/change Apps in such away that would enable further automation to be as easy as possible.<p>Cheers!
1) Look into configuration management. Ansible and Salt Stack are both python-based so everyone should be able to contribute if needed.<p>2)It's popular to run your automation through CI tools like Jenkins because it has auditing and logging built in and it's easy to set up.<p>3) Don't run bower/npm install/update on production because if npm or github go down then your production site will break because a partial code change has taken place. You should build (and test) on a separate server and then copy the code changes over to production.<p>Really you're looking for a Capistrano for python apps. If you're going to build a deployment tool on your own it's worth looking at their approach. For python, fabric is a fairly standard way to automate code deployments. The good news is that you're on the right path. Having automated deployment procedures will give devs a lot more time back AND remove a whole class of errors<p>I've moved my current company's deployment from manual line by line ftp deployment to an automated build and deploy using Jenkins to trigger ansible playbooks. Given the choice, I'd probably just use capistrano or fabric instead of building from scratch. I'd estimate that we've saved several months of work per a year in both deployment time and in debugging production issues.
I could integrate my entire workflow with Jenkins and custom parameters and custom development of console applications ( creating automated versions of packages ( npm / nuget / .. ), mostly with the use of the "build number")<p>It handles 3 situations:<p>- local ( not included in Jenkins, but is automated within the development tools -- eg. Visual Studio. It uses local databases and local paths)<p>- development<p>- beta<p>- live / production<p>So my entire workflow is automated now, the merging of branches and etc. is done by the developer. Testing is automated ( unit tests) ... Deploying is just a matter of minutes instead of 1,5 hours. And best of all, all related packages are updated automaticly with the correct tag ( development, beta, ..)<p>Another way would be using TeamCity ( instead of Jenkins)
For deployment: use your OS's packaging system, for both dependencies and
actual applications. `git pull' rarely does good job, but `apt-get install'
often is just enough.<p>For development: use your OS's packaging system for installing dependencies
and a makefile for compiling things.<p>Really, I don't understand why programmers tend to avoid using sysadmins'
tools in their work, especially now with this DevOps thing, when programmers
are supposed to see some administration themselves.
Longer blog post here: <a href="http://dna.hioscar.com/2015/10/14/scaling-deployments-at-oscar/" rel="nofollow">http://dna.hioscar.com/2015/10/14/scaling-deployments-at-osc...</a><p>We work out of a mono-repo and use pants (<a href="https://pantsbuild.github.io/" rel="nofollow">https://pantsbuild.github.io/</a>) for python/jvm/js packaging to deploy on mesos (<a href="https://mesos.apache.org/" rel="nofollow">https://mesos.apache.org/</a>). Pants gives us (effectively) the same development workflow as in production and let's us treat any application (we have ~500 at this point) as just a command line executable (for things with weird lib dependencies, we do statically compiled wheels for each architecture - CentOS and OS X in our case - on our own pypi mirror). We've got a few kinks to still workout for local development of web projects (mostly related to JS/CSS) but for things that don't touch the frontend it's pretty beautiful.
I am currently finishing up writing a step-by-step tutorial on this very topic. Please check it out and, if you have a question about any piece of it, write a comment and I'd be happy to help.<p><a href="https://amfarrell.com/saltstack-from-scratch/" rel="nofollow">https://amfarrell.com/saltstack-from-scratch/</a><p>Also let me know if any of the integration tests for any of the tests fail on your setup.