TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Dagger Python SDK: Develop Your CI/CD Pipelines as Code

161 点作者 yurisagalov超过 2 年前

27 条评论

diarrhea超过 2 年前
I&#x27;ve long wished to be just writing (ideally) Python to define and run CI&#x2F;CD pipelines. YAML is simply hell, `if` keys in such data description languages are cruel jokes.<p>Round-trip times for GitHub Actions are too high: sometimes you&#x27;re waiting for 10 minutes just to run into a dumb typo, empty string-evaluated variable or other mishap. There&#x27;s zero IDE support for almost anything beyond getting the YAML syntax itself right.<p>We have containerization for write-once-run-anywhere and languages like Python for highly productive (without footguns as bash has them), imperative descriptions of what to do. The main downside I see is it getting messy and cowboy-ish. That&#x27;s where frameworks can step in. If the Dagger SDK were widely adopted, it&#x27;d be as exchangeable and widely understood&#x2F;supported as, say, GitHub Actions themselves.<p>We currently have quite inefficient GHA pipelines (repeated actions etc.) simply because the provided YAML possibilities aren&#x27;t descriptive enough. (Are Turing-complete languages a bad choice for pipelines?)<p>What&#x27;s unclear to me from the article and video is how this can replace e.g. GitHub Actions. Their integration with e.g. PR status checks and the like is a must, of course. Would Dagger just run on top of a `ubuntu-latest` GHA runner?
评论 #33552244 未加载
评论 #33553437 未加载
评论 #33554300 未加载
评论 #33557673 未加载
评论 #33555057 未加载
评论 #33556174 未加载
评论 #33556479 未加载
评论 #33552960 未加载
评论 #33553796 未加载
评论 #33558353 未加载
helderco超过 2 年前
Hey, lead developer for the Python SDK here.<p>I have a few examples in <a href="https:&#x2F;&#x2F;github.com&#x2F;helderco&#x2F;dagger-examples" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;helderco&#x2F;dagger-examples</a> and I plan to add more.<p>There&#x27;s also reference documentation in <a href="https:&#x2F;&#x2F;dagger-io.readthedocs.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;dagger-io.readthedocs.io&#x2F;</a> so you can get a birds eye view on what&#x27;s possible.
评论 #33554256 未加载
评论 #33550696 未加载
shykes超过 2 年前
Hi everyone, Dagger co-founder here. Happy to answer any questions.<p>We also released an update to the Go SDK a few days ago: <a href="https:&#x2F;&#x2F;dagger.io&#x2F;blog&#x2F;go-sdk-0.4" rel="nofollow">https:&#x2F;&#x2F;dagger.io&#x2F;blog&#x2F;go-sdk-0.4</a>
评论 #33552407 未加载
评论 #33555228 未加载
评论 #33554428 未加载
评论 #33552165 未加载
评论 #33571699 未加载
评论 #33554887 未加载
bilalq超过 2 年前
Haven&#x27;t heard of Dagger before, but using IaC as code to model your pipelines (including pipelines of infra that itself is modeled as IaC) is absolutely the right way to do things. And doing it in a real programming language rather than some limiting yaml schema is so much nicer. Places I&#x27;ve worked at in the past would refer to these as meta pipelines or self-mutating pipelines.<p>For this reason, I love using the AWS CDK. Being able to model things in TypeScript was so much nicer than the janky flow of always feeling lost in some Ruby&#x2F;JSON&#x2F;YAML IaC template monstrosity.<p>Curious how Dagger differentiates itself from AWS CDK or cdktf.
评论 #33550981 未加载
brianzelip超过 2 年前
FYI one of Dagger’s developers runs the most excellent, and breath of fresh air really, Ship It podcast, which focuses on CI&#x2F;CD and Infrastructure as Code (and always be improving!).<p><a href="https:&#x2F;&#x2F;shipit.show&#x2F;" rel="nofollow">https:&#x2F;&#x2F;shipit.show&#x2F;</a>
dvasdekis超过 2 年前
Hi guys, love the idea and the code examples look simple and understandable. Nicely done!<p>One of the reasons we use proprietary pipelines is the automatic &#x27;service principal&#x27; login benefits that exist on e.g. Azure Devops, where the pipeline doesn&#x27;t need to authenticate via secrets or tokens, and instead the running machine has the privileges to interact directly with Azure. (See <a href="https:&#x2F;&#x2F;learn.microsoft.com&#x2F;en-us&#x2F;azure&#x2F;devops&#x2F;pipelines&#x2F;tasks&#x2F;deploy&#x2F;azure-cli?view=azure-devops" rel="nofollow">https:&#x2F;&#x2F;learn.microsoft.com&#x2F;en-us&#x2F;azure&#x2F;devops&#x2F;pipelines&#x2F;tas...</a> particularly &quot;addSpnToEnvironment&quot; parameter). I&#x27;m sure other clouds have something similar.<p>Running the same pipeline locally, there are ways to synthetically inject this, but there&#x27;s no ready support in your framework yet for this (as ideally you&#x27;d have an &#x27;authentication&#x27; parameter that you can set the details for). Is something like this planned?
评论 #33555383 未加载
rekwah超过 2 年前
Curious if cuelang just ended up being too much of a hurdle for onboarding. I like it and have used it quite a bit but there&#x27;s something about the syntax that makes it impenetrable for many.
评论 #33551665 未加载
评论 #33558165 未加载
chologrande超过 2 年前
This post is for the python sdk, but the golang SDK is fantastic too.<p>Complete gamechanger to be able to move away from jenkins groovy CPS hell
评论 #33559856 未加载
pm超过 2 年前
So far I&#x27;ve been using Earthly for structuring builds, and it&#x27;s been brilliant for the most part. Dagger was interesting but a no-go without Python, so curious to see how it stacks up.
travisgriggs超过 2 年前
Years ago, I had inherited a complicated C code base with some clever Makefile jujitsu. None of the recent maintainers understood it at its core. They just put a little lipstick here, and a little there to keep things running.<p>I sat down and extracted the compiler&#x2F;link flags, and then wrote a Python script to do the build. The code was smaller, and built faster.<p>Every “build” engine evolves from being a simple recipe processor to the software equivalent of a 5 axis CnC mill. Some things should not succumb to one size fits all.
评论 #33555887 未加载
评论 #33554913 未加载
mkoubaa超过 2 年前
While I prefer python over yaml, I tend to think imperative build systems are inherently more brittle.
评论 #33551802 未加载
评论 #33555531 未加载
epgui超过 2 年前
Even if this was a great, well-built tool, which I am sure it is...<p>Doesn&#x27;t this introduce a whole dimension of extra stateful complexity compared to configuration YAML?
评论 #33550800 未加载
madjam002超过 2 年前
I had a look at Dagger, but I&#x27;m having a lot of success recently just building CI&#x2F;CD pipelines with Nix. My `.gitlab-ci.yml` file is then just a thin wrapper of `nix build` commands, and pipelines can be run locally in the exact same environment as CI&#x2F;CD.
评论 #33558187 未加载
aclatuts超过 2 年前
This makes a lot of sense but would it prevent developers from using the default diagrams already built into most CICD websites. I assume migrating to this would make the CICD pipeline look like one step from gitlab, bitbucket, and githubs perspective.
评论 #33551784 未加载
xedx超过 2 年前
This is an absolute god send for data engineers! We often get blocked full stop with CI&#x2F;CD headaches and the alternative of “no CI&#x2F;CD” is just unbearable at times. This needs to catch on quick !
mehanik超过 2 年前
Pipelines as code is the way to go, no matter whether it is CI&#x2F;CD, data or ML. It’s unfortunate that a lot of pipeline tools use YAML. Glad to see projects like Dagger that use Python instead.<p>However it is not clear for me what is the benefit of using it instead of calling commands like docker, pytest and kubectl from Python with Plumbum or similar library. Add Fire and it is trivial to create a complete DevOps CLI for your app that can run locally or called from GitHub actions.
tatoalo超过 2 年前
This is something I’d love to try on my personal projects.<p>I recently used act[0] to locally test my GitHub actions pipelines and worked okay, the fact that I could interact with the Dagger API via a Python SDK could be even more convenient, will definitely try!<p>[0]: <a href="https:&#x2F;&#x2F;github.com&#x2F;nektos&#x2F;act" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;nektos&#x2F;act</a>
mkesper超过 2 年前
Might be an interesting middle ground between the nightmare of everthing-as-plugins-but-flexible Jenkins and something like tekton.
__warlord__超过 2 年前
Shameless plug:<p>I&#x27;m working on a similar project, portable pipelines built with Nim :)<p>The idea behind this project is to:<p>1. Dev and Prod pipelines runs the same code, the only difference is where are executed. Allows easy troubleshooting and faster development.<p>2. Tests output by default on each script.<p>3. Decouples the code from the image&#x2F;container. This allows it to be embedded into any CI stack.<p>4. Complex rules for retries specific scripts rather than the whole pipeline or even the step (stage)<p>A typical pipeline definition looks like this:<p><pre><code> ```yaml kind: pipeline version: 1 policy: strict steps: - hello hello: - script: echo &quot;hello world&quot; expected_output: &quot;hello world&quot; expected_return_code: 0 retries: 5 ``` ```bash .&#x2F;takito --tasks tasks.yml --step hello ```</code></pre>
kpen11超过 2 年前
Here&#x27;s the demo video linked in the blog post: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=c0bLWmi2B-4" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=c0bLWmi2B-4</a><p>It goes step by step through the getting started guide from the Dagger Python SDK docs
quelltext超过 2 年前
Confusing. I initially thought someone ported the Dagger DI framework to Python: <a href="https:&#x2F;&#x2F;dagger.dev&#x2F;" rel="nofollow">https:&#x2F;&#x2F;dagger.dev&#x2F;</a>
评论 #33558283 未加载
pianoben超过 2 年前
(NB: No relation to the widely-used Android&#x2F;Java DI library)
meling超过 2 年前
This looks awesome! I’ve built similar CI logic using git and docker apis for an autograder system. Looks like I can throw out some of this code… and replace it with dagger.
throwawaaarrgh超过 2 年前
Debugging and maintaining code is to be avoided at all costs. You may prefer writing code as a developer, but it doesn&#x27;t provide value to the people paying your paycheck, and it doesn&#x27;t result in better outcomes than simple composeable actions.<p>Proprietary CI&#x2F;CD systems are a waste of time. If it&#x27;s more complicated than a shell script, you need to strip it down.
评论 #33558304 未加载
yakkityyak超过 2 年前
Wish I could use this for work, but with no buildkit access its a no go :(
评论 #33555468 未加载
goodpoint超过 2 年前
Why does it require docker? That&#x27;s a show stopper.
评论 #33560278 未加载
dith3r超过 2 年前
how this compares to buildbot or gajapipelines