Hey everyone, I'd like to announce the launch of Conducto and its CI/CD tool. As a co-founder, I originally developed the technology to do data science at my previous employer, Jump Trading (top quant trading firm like Citadel and Jane Street). As we built it into Conducto, we realized how good it was for our own CI/CD and switched course to target that first. Main distinguishing features:<p>* CI/CD in Python, not YAML: simple and powerful way to define CI/CD pipelines that is understood by IDEs<p>* Trees not DAGs: DAGs are messy. If you've ever traced a node's dependencies in a medium-sized CircleCI project (speaking from experience here) you know how frustrating it is to search through a 2000-line YAML file. Trees help your pipelines stay simple as you grow, and they're much more useful to look at.<p>* Free mode that doesn't suck: Run pipelines on your laptop for free, forever, for you and as many teammates as you want. When you need more scale you can run hundreds of tasks simultaneously in the Conducto cloud.<p>We're launching today, and I'd like to invite you all to check it out. I want feedback as much as I want users, so please look at it and let me know what you think.
I'm another user of the predecessor to Conducto. We're using it to orchestrate very large regression test (usually ~40-50k independent processes run as part of the regression testing).<p>I also use conducto's predecessor to manage "data science" workflows (simlar to <a href="https://www.conducto.com/docs/advanced/data-science" rel="nofollow">https://www.conducto.com/docs/advanced/data-science</a>) on a large computer. It is miles ahead of the alternatives we've tried (and built).<p>The biggest win we've gotten with conducto comes from the composability. We can ship part of our application as a function which returns conducto nodes. Someone else can call this function an stick it into their own tree (code, not configuration!)<p>Our integration test suite builds a conducto tree ships containing test code from 4ish different libraries (developed by 2 different teams), each of which includes it's own tree-building functions. Likewise, we ship many of our applications for the large computer as functions which return conducto nodes. Users just call our functions and embed our portion of the workload into their own trees.<p>Give it a shot!
“Configuration as code, not YAML” - this is a hilarious and relatable tagline.<p>I wonder how this compares to some existing CI/CD tools like Jenkins, circle ci, GitHub actions, etc.