<i>> There is no way for changes in the outside world—such as a new version of a dependency being published—to automatically affect a Go build.</i><p><i>> Unlike most other package managers files, Go modules don’t have a separate list of constraints and a lock file pinning specific versions. The version of every dependency contributing to any Go build is fully determined by the go.mod file of the main module.</i><p>I don't know if this was intentional on the author's part, but this reads to me like it's implying that with package managers that <i>do</i> use lockfiles a new version of a dependency <i>can</i> automatically affect a build.<p>The purpose of a lockfile is to make that false. If you have a valid lockfile, then fetching dependencies is 100% deterministic across machines and the existence of new versions of a package will not affect the build.<p>It is true that most package managers will automatically update a lockfile <i>if it's incomplete</i> instead of failing with an error. That's a different behavior from Go where it fails if the go.mod is incomplete. I suspect in practice this UX choice doesn't make much of a difference. If you're running CI with an incomplete lockfile, you've already gotten yourself into a weird state. It implies you have committed a dependency change without actually testing it, or that you tested it locally and then went out of your way to discard the lockfile changes.<p>Either way, I don't see what this has to do with lockfiles as a concept. Unless I'm missing something, go.mod files <i>are</i> lockfiles.
Most package managers have lockfiles. Yes, npm's decision to have both "npm install" and "npm ci", just so you can confuse and mislead developers, is a bit silly.<p>But Ruby's Bundler, for example, has been refusing to run your code if your lockfile is inconsistent for as long as I remember.<p>Locking dependencies is, generally, a solved problem across most ecosystems (despite node botching its UX). Go doesn't get to claim that it's superior here.<p>But of course, supply chain attacks are still possible with lock files. Because <i>somebody</i> is going to update your dependencies at some point (often for security reasons). And at that point you might be pulling in a malicious dependency which you haven't carefully vetted (because nobody has time to vet all their dependencies thoroughly nowadays).<p>That's still an unsolved problem, as far as I know. I don't think that Go has solved it.
The elephant in the room here is NPM, and I think the obvious problem there is the culture. I have a tiny app I've been playing with using create-react-app. There are over 800 directories in node_modules. That absolutely dwarfs the number of any other language I've used. Even in a medium sized rails app, you likely have some awareness of what every dependency is. It's just impossible with npm.<p>One thought I've had to "reboot" the npm culture is to somehow curate packages that are proven to have minimal and safe dependencies, and projects can shift to using those. I imagine there has to be some sort of manual review to make that happen.
Something this articles glosses over is that some of these approaches, especially the way 'All builds are “locked”' is achieved with minimum version selection, and “A little copying is better than a little dependency” are tradeoffs against an alternative security model, where transitive dependencies are automatically updated to pick up security fixes.<p>Part of the churn and noise in the Node.js dependency ecosystem actually stems from security-related issues being noted in a low-level module, and the ripple effects caused by that when a bunch of maintainers have to go around bumping lockfiles and versions.
I'm annoyed by the false dichotomy that colors most discussions around package management that there are only two solutions to publishing software packages: 1. a carefully curated professionally maintained standard library, 2. the complete wild west where anything goes. It's not really "false" because this is the reality of how package managers are designed today, but it's false in the sense that <i>it doesn't have to be this way</i>.<p>You can see this tension in virtually every discussion, users resisting using packages that aren't published in the standard library for fear of attacks and poor quality, and maintainers that resist publishing in the standard library for fear of changing requirements and the appearance of better designs. Sure there are admissible entitlement / responsibility arguments against these respective positions, but that's mostly a distraction because <i>both have a valid point</i>.<p>The problem is that there's no space for intermediate solutions. We need packaging tools to aggregate and publish groups of packages that relate to a particular domain, and organizational tools to ensure quality and continuity of these package groups over time. This mitigates users' fears and reduces their cognitive load by curating the solution space, and it mitigates maintainers fears of ossification and backcompat hell by enabling them to create new package groups.<p>I'm saying there's an entire dimension of valid tradeoffs in this space, but the current design trend of package managers force us into one extreme or the other.
The article, and the comments praising this approach, don’t do a great job of explaining how any of this is substantively different from running the likes of yarn install --frozen-lockfile, or cargo build --frozen.<p>Here’s the thing: You can argue about being secure by default and encouraging better CI practices. I’d fully agree it isn’t great that one has to know a somewhat obscure flag to get a secure CI build in those environments.<p>But claiming in what I perceive to be in parts a somewhat grandiose tone to have reinvented the wheel, when you’re just describing a standard approach, can make you sound uninformed.
> Unlike most other package managers files, Go modules don’t have a separate list of constraints and a lock file pinning specific versions.<p>The weird thing about the Go devs is there is always that little bit of elitism under the surface that I detect in their writing (whether it be colors in the playground, the GC, etc). I spent years writing Go and have now moved to Rust. What I find odd is the Rust team has done (IMO) one of the greater achievements in PL history and yet they seem to not have this elitism thing going on (or maybe I just haven't noticed). Go on the other hand, IMO, made some "interesting" language choices (like keeping null) and they seem to want to be celebrated for it and claim their achievements as new and novel.<p>EDIT: To clarify, I'm talking about the core Go devs - those that work on stdlib and the compiler
Go is a distillation of many decades of software engineering experience. The people behind Go (e.g., Russ Cox) have learned from history.<p>The peanut gallery loves to complain about superficial aspects of Go. Typically these are people with little or no actual experience using the language and tools. They fixate on imagined problems that don't matter in practice.<p>But anyone who has used Go full-time for a few years is likely to deeply respect and appreciate it.
Dependencies being immutable and identified by hash was such an obvious thing 20 years ago. The problem was fitting that into the flow of these crappy UN*X based build systems where to do any mundane task you need to fiddle with files and encodings and semi documented folder heirarchies and use a CLI tool to change other stuff that is too cumbersome to encode directly into files / text. The most obvious concrete instance of this being that it's cumbersome to import a dependency by hash (in Java for example, you really want hashes everywhere instead of reverse TLDs) but obviously better tools (such as a structural code editor) solves this perfectly. It's also annoying every time a company takes a nano step asymptotically towards these proper solutions and spams their crap and everyone buys it, but that's to be expected when your field is broken industrially and academically and even in hobbiest communities.
There are many gems in its design and implementation in addition to those mentioned in OP.<p><a href="https://verdverm.com/go-mods/" rel="nofollow">https://verdverm.com/go-mods/</a>
Off topic but: Since I assumed this was about <i>physical</i> supply chain attacks (where someone nefarious will either intercept your package to install custom firmware etc. or even change the physical device in some way) - does anyone know where I could find a good guide on mitigating such attacks?
Worth noting is that, while go build doesn't run arbitrary code, go generate <i>does</i>:<p><pre><code> package evil
import "fmt"
// the echo is intentional, in case someone actually tries this for some reason
//go:generate echo rm -rf /
func PretendGood() {
fmt.Println("I am good")
}
</code></pre>
When they say fetching and building code doesn't execute it, that's specific to go get and go build. There's no guarantee that every go subcommand is safe. This is pretty obvious if you know how go generate works and it isn't a flaw of the language, but if I were new to go, this is the kind of article I'd read but still not understand exactly what was safe and what wasn't.
One downside of such pinning of versions is that if one of your transitive dependencies has a security vulnerability affecting your package, your package will remain affected until you update.<p>As I understand it dependencies-of-dependencies are fetched from the config provided by the library that includes the dependency, in the case of a transitive dependency, everyone between you and the vulnerable package needs to update, and they need to do so in order (i.e. if you depend on A, A depends on B, B depends on C, and C fixes a vulnerability, then <i>first</i> B, <i>then</i> A have to update in order for you to pull in the change).
I feel like everyone is beating around the bush here: NPM is a garbage fire, from the interface to the tooling implementation to the theory to the governance. We can talk at each other back and forth about theoretical benefits, and "friction" vs. "usability" or whatever, but NPM has been <i>and continues to be</i> an unmitigated security disaster. The module proxy could have a package takeover a month for the next three years and still not even come close to the ridiculous shit that has happened on NPM.
>> <i>The only commands that will change the go.mod (and therefore the build) are go get and go mod tidy. These commands are not expected to be run automatically or in CI, so changes to dependency trees must be made deliberately and have the opportunity to go through code review.</i><p>GO doesn't do jack shit to mitigate supply chain attacks. Version pinning with checksum and that is it. But what could go do? Solve supply chain attacks as a language feature? That doesn't even make sense.<p>Application developers using Go must prevent supply chain attacks against their applications. So go get some SAST for your pipeline.<p>Sure there is truth in saying: always verify your dependencies (and their dependencies) yourself with a code review on every update. But you should not do that alone, so let's talk about collaborative vulnerability management. <i>(there is more to sast than vulnerability assessment, but we have to start somewhere)</i><p>Let's say repositories that publishe go modules should also publish a curated list of known vulnerabilities <i>(including known supply chain attacks)</i> for the modules they publish. This curation is work: reports must be verified before being included in the list and they must be verified quickly. This work scales with the number of packages published. And worse, modules could be published in more than one repository, module publishing repositories can be different from the modules source code repository, and vulnerability lists can exist independent from these repositories - so reports should be synced between different list providers. Different implementations and lack of common standards make this a hard problem. And implicit trust for bulk imports could open the door for takedown attacks.<p>There is an argument that vulnerability listing should be split from source and module publishing: each focusing on their core responsibility. For supply chain attacks this split in responsibilities also makes it harder for an attacker to both attack suppliers and suppress reports. But for all other issues it increase distance as reports must travel upstream. And it creates perverse incentives, like trying to keep reports exclusive to paying customers.<p>To pile on the insanity: reports can be wrong. And there are unfixed CVEs that are many years old <i>(well ok maybe not for go... yet)</i>. Downstream there are "mitigated" and "wont-fix" classifications for reports about dependencies and many SAST tooling can't parse that for transitive dependencies.<p>Really, supply chain attacks are the easy case in vulnerability management, because they are so obviously a "must-fix" when detected. <i>(and to please the never update crowd: for a downstream project "fix" can mean not updating a dependency into an attacked version. you are welcome)</i><p>Long story short: go get some SAST in your pipelines to defend against supply chain attacks. Don't pretend pinning the version and half-assing a code review when you update them is actually solving supply chain attacks. Don't tell me everyone who uses go can find a sophisticated data bomb or intentional rce in some transitive dependency of some lib they update to a new feature release. And don't give me some "well if its transitive then the lib dev should have." Should have doesn't solve shit.<p>Examples for Vulnerability Assessment in GO dependency graphs are GitLabs Gemnasium ( <a href="https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/tree/master/go" rel="nofollow">https://gitlab.com/gitlab-org/security-products/gemnasium-db...</a> ) or GitHubs Dependabot ( <a href="https://github.com/advisories?query=type%3Areviewed+ecosystem%3Ago" rel="nofollow">https://github.com/advisories?query=type%3Areviewed+ecosyste...</a> ) among many, many others. Not recommendations, just examples!<p>Tools like these help you sort out supply chain attacks that other people have already found, before you update into them and push them downstream. Collaboration is useful. Sure you are still left with reading the source changes of all dependency update, because who knows, you may be the first one to spot one, but hey, good for you.
node, python, ruby...etc including go modules can lock their dependencies, Go really wins when it can be wrapped in a reasonably-sized one binary, nothing beats that in deployment, for all the others you have to pull in lots of packages into the target system
I don't like the authoritative tone of the article. Especially
given the fact that author "conveniently forgets" about<p><pre><code> go mod edit
</code></pre>
and<p><pre><code> go work
</code></pre>
both of which are deliberately designed counter-mitigations, i.e. they exist
to poke small holes in the pin-everything wall.
I agree with the spirit of the message though, the surface is much smaller
with Go and it shows much planning went into that.