Concourse CI Hits 1.0.0

I wish documentation would offer a &quot;Concourse made easy&quot; or something.<p>I imagine the documentation is currently a work in progress. But starting for example from Github, it is extremely easy to go down a deep dark hole trying to figure out all the new terms: I guess I have to use BOSH, what is BOSH (itself a deep dark hole: uhh, stemcell? How does this relate to all the other similar tools? Why would I commit to using this new tool solely for the sake of Concourse?); now I have to learn a new strange distinction between the otherwise similar English words &quot;job&quot; and &quot;task,&quot; something about Garden (yet another hole to travel) and a new meaning for the acronym &quot;TSA&quot; and for some reason the command line utility is called &quot;fly&quot; which has no obvious relationship to the project name (what if the command to interact with git were &quot;spudge&quot;?) can I please just install a .deb and run a service and add a build real quick to try this out?<p>This could be why it is so common to see Jenkins deployed everywhere. While it is wrongly designed in so many frustrating ways, you can still set it up for real work in an hour or so, and you won&#x27;t need a dictionary to translate from a unique new jargon into English just to try it out. Every new CI that makes this even one bit harder than Jenkins isn&#x27;t trying hard enough.

Full Disclosure: I also work at Pivotal on a team that has done a large amount of automation using Concourse (check out our pipelines [1])<p>Things that we&#x27;ve done so far with Concourse:<p>* A dependency check notifier that based on an RSS feed from various language maintainers: Sends an email, updates our pivotal tracker account, sends a message to our slack channel<p>* A job, that based on the completion of a build will upload the resulting artifact to Github and tag the release with a new version, auto generated release notes and checksums<p>* Travis.ci style github pull request analysis<p>* A security feed monitor that notifies us when a part of our application is vulnerable and when it is, kicks off a build and sends an email notification<p>* Automatically updates submodules and gems for git repos and runs specs and tests against the updated artifact.<p>These tasks were not difficult to implement. Every pipeline for Concourse is explicitly defined in its configuration file [2], which we version control. There is never any reconfiguring or rebuilding &quot;snowflake&quot; servers when we re-deploy our Concourse VMs. They rebuild identically based on the config files.<p>[1] - <a href="http:&#x2F;&#x2F;buildpacks-ci.cfapps.io" rel="nofollow">http:&#x2F;&#x2F;buildpacks-ci.cfapps.io</a> (Chrome friendly, Firefox not)<p>[2] - <a href="https:&#x2F;&#x2F;github.com&#x2F;cloudfoundry&#x2F;buildpacks-ci" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;cloudfoundry&#x2F;buildpacks-ci</a>

I work at Pivotal, which sponsors Concourse development, so I got one of the fancy t-shirts we made for this occasion.<p>I wore it to the office today. Other people in the office were coming up to me and raving about Concourse. I&#x27;ve been proselytising it, but I feel like pretty soon I just won&#x27;t have to. One of my colleagues was telling me that in two days he got an iOS CI pipeline that he never managed to achieve with weeks of tinkering with TeamCity.<p>I suspect that pretty soon the default choice for CI&#x2F;CD in Pivotal Labs projects will be Concourse, and that will seed it pretty widely.<p>If you want to get in on the ground floor of something great, here&#x27;s a good opportunity.<p>If you&#x27;re wondering about what the heck it is, go to the homepage for the 40,000 view: <a href="http:&#x2F;&#x2F;concourse.ci&#x2F;" rel="nofollow">http:&#x2F;&#x2F;concourse.ci&#x2F;</a><p>If you&#x27;re wondering why the heck <i>another</i> CI system, look at &quot;Concourse Vs.&quot;: <a href="http:&#x2F;&#x2F;concourse.ci&#x2F;concourse-vs.html" rel="nofollow">http:&#x2F;&#x2F;concourse.ci&#x2F;concourse-vs.html</a><p>After that my advice is to look at public Concourse pipelines and pop the hood. And drop into the Slack channel, the team are in there now posting funny gifs. I&#x27;m in there too, under jchester_robojar, if you want to ask me anything.

Congratulations for this release.<p>However as already seen in the case of GoCD, the success of Concourse will depend on how easy it is to convince Jenkins believers.<p>The &quot;vs Jenkins page&quot; needs to be augmented with more information regarding workers, plugins and simple jobs scripts.<p>At the moment that page is not really convincing<p>1)&quot;Jenkins servers become snowflakes&quot; All configuration is saved on the disk as XML files. Backing them up or putting them in version control is very easy. Restoring Jenkins servers is very easy. I have done this actually.<p>2)&quot;Jenkins has no first class support for pipelines&quot; True. But Jenkins 2.0 will be designed around pipelines, so once it is released this argument will no longer hold. And even today with 1.x not all organizations use pipelines (or are happy with the existing plugins)<p>3)&quot;Trying to find the build output log for a failed build in Jenkins can be up to 3 clicks from the home page&quot; That is not really an argument. I get an email when my build fails.<p>As others have mentioned having readers understand new concepts in order to run&#x2F;use if effectively is problematic.<p>The UI is impressive! Congrats! I think you should market this more.<p>Basically if the effort required to setup Concourse with 2 workers is bigger then setting up the respective Jenkins setup, Concourse adoption will suffer.<p>It needs to be stressed that for a lot of small&#x2F;middle sized companies, the build server is just an afterthought. Not everybody has full time people working on setting up pipelines.

I&#x27;m looking forward to experimenting with this.<p>At least in the documentation, there seems to be a lot of focus on extra tools. BOSH seems very off-putting. Having to use the &quot;fly&quot; CLI command to update pipeline configuration seems unnecessary - can I just update the files on the filesystem myself, either via my configuration management tool or a git repository?

For anyone else that got excited about having a Jenkins alternative, this project seems to be only about containers and not other workloads that VMs could tackle, such as running browser or desktop-based software tests. At least that&#x27;s my understanding after giving the documentation a cursory glance. Could someone familiar with the project please confirm (or hopefully correct) this?

I very much appreciate the &quot;Concourse vs.&quot; page[1] which compares Concourse to several other CI solutions, but I wish they would include Buildbot[2]. Everyone always forgets about Buildbot!<p>Buildbot can integrate with Docker to run tasks on containers, create complex conditional flows with triggered builds&#x2F;steps (among others), and has been in use for many years at organizations large &amp; small.<p>While the YAML based Concourse configurations seem very useful, we&#x27;ve found that for many non-trivial pipeline style builds are better served by being written in a fully functional language like Python. It has some disadvantages compared to a declarative approach taken by Travis &amp; Concourse, but I feel that they outweigh the limitations imposed by the latter.<p>The next version of buildbot is also being refactored into a core + plugins system, which should allow for some pretty impressive flexibility that should serve the needs of almost everyone.<p>--<p>1. <a href="https:&#x2F;&#x2F;concourse.ci&#x2F;concourse-vs.html" rel="nofollow">https:&#x2F;&#x2F;concourse.ci&#x2F;concourse-vs.html</a> 2. <a href="http:&#x2F;&#x2F;docs.buildbot.net&#x2F;latest&#x2F;" rel="nofollow">http:&#x2F;&#x2F;docs.buildbot.net&#x2F;latest&#x2F;</a>

Have recently rebuilt CI systems at work, and took a look at Concourse alongside Jenkins, Buildbot and Gitlab. Seems like not so many people have mentioned gitlab on this thread, but in my experience it combines all the benefits of Concourse (pipeline-driven workflows, passing forward build artifacts, config in source control) with a much simpler architecture. Concourse looks so complex and has so many moving pieces to learn - BOSH, fly, etc...<p>Gitlab is just a rails app and then has a go-based runner to do the CI work. Omnibus packages make installation easy. They even have some autoscaling stuff for the runners using docker engine there which I still think needs some work for my use case but for may others is ready to go and looks awesome.<p>CI covers so many use cases and so there is lots of room for different alternatives, but make sure to check out gitlab if you&#x27;re looking for the simplest and most flexible CI system I&#x27;ve yet to see :)

This looks awesome! I hope to transition to it, handling Jenkins configurations is a huge pain.<p>Some of people also use their CI tools for running arbitrary one-off scripts (like deploys). Is this possible with Concourse? Currently we have a few Jenkins jobs that just execute a fab task that then does the actual work.

Does Concourse offer any integration with defect or agile development software that allows an easy way to track which builds contain a given feature or defect fix?

Are there any plans to support OS X workers? This would be extremely helpful for Mac and iOS apps.