Writing a Package Manager

The author added a lockfile without understanding why they exist.<p>A lockfile is meant to &quot;freeze&quot; dependency version resolution when package authors can specify dependencies on other packages using version ranges... it also &quot;freezes&quot; choices of transitive packages&#x27; versions when different packages depend on the same one, but with different versions.<p>They chose to not handle package dependencies at all, and I believe there&#x27;s no version ranges either, so I really don&#x27;t see why they added a lockfile.

Honestly pretty strange to write a package manager for sqlite and to use directories + json files to store the data instead of sqlite.

Awesome post! I worked with a bunch of package managers over the years and one can see that this design got inspired by the hood parts of a few I know.<p>The only design part I don‘t really like is the ‚latest‘ version specifier in the spec file. Which moves the declaration what the latest version is to the hosted location (in the example GitHub via GitHub API) paired with the fact that the checksums are also fetched rather than being part of the spec. This makes no sense for me. The spec needs to be versioned or better the spec is the actual location to declare a release. I kind of understand where the desire comes from to have a floating spec. Makes the publish process easier since one only creates a GitHub release in this case. But I would still argue that explicitly creating a new version of the spec for each version with baked checksums is better. The benefit for me would be that one could create a hash for each spec version for instance and use that internally for instance.

Sounds like Maven had all this solved many years ago. Yes, it cannot run arbitrary code, like NPM does, it just copies files, but the dependencies and specfiles were there from the beginning.

It&#x27;s surprisingly to me that no-one has built a asdf style package manager. (I&#x27;m not talking about system package managers, their language software packages are always out of date and get installed globally instead of locally to a project).<p>Having a unified interface to a package manager per language that will use the languages registry could be really nice (I guess you&#x27;d have some core dependency management functions and agnostic ways to store and download the packages, and plugins per language that would use these common building blocks to actually do stuff?).<p>Another advantage of this could be cross language dependencies which often aren&#x27;t handled well by package managers.

I think some day rather than the current paradigm, even including declarative package managers and environments and distros, the future will be per-user and even per-app chrooting or jails, or something similar. Apple already uses something like this today. Many people who are smart about information security have one login for shopping and banking and bill pay, one for their business, and one for cruising the web or gaming or whatever. That way a breach of one doesn&#x27;t necessarily end up pwning the whole system.<p>I only have two accounts, one &quot;serious&quot; and one &quot;off hours&quot; but I still feel better protected than most people.

Bit of a tangent here but what’s a pip&#x2F;npm&#x2F;cargo like package manager for C++? For example ‘pip install boost’? I’ve never worked it out for hobby projects and never worked with it commercially

Wonderful post! I&#x27;ve written a couple of packager manager like things over the years. Although they&#x27;ve all been internal-only which gives some flexibility.<p>&gt; I like the idea of allowing both project and global scope<p>Yikes! Hard no from me. Globals are pure evil. Avoid like the plague. Environment variables too. Kill them all with fire.<p>&gt; what if the user wants to reinstall the packages on another machine or CI server? That’s where the lockfile comes in<p>I&#x27;ve typically bypassed the need for a lockfile by simply checking in the dependencies. Dependencies belong in version control! That&#x27;s a rant for another day.<p>Treating the packages folder as source of truth is basically the equivalent I think?<p>&gt; I understand that dropping dependencies altogether may not be something you are ready to accept.<p>Nice. A corollary to this is that if all your packages are internal then you can simply disallow dependency graphs that want different versions of the same package. Simply bump and fix-up all version number dependencies for all packages.<p>The basically means &quot;all packages play nicely on latest version&quot;. Which when all the packages live in a monorepo is perfectly plausible. Totally doesn&#x27;t work for public package managers, but that&#x27;s a different story.

Source control is my package manager. Package managers as we usually think of them are syntax sugar and abstraction for the sake of abstraction<p>Working on a Linux distro that is one unified&#x2F;generalized&#x2F;normalized code base (with the help of AI&#x2F;ML) and a model to sample and establish correct state from memory of the initial code base.<p>One way to think of it is like a game engine with action plans to allocate resources to recreate Firefox, for example. Not compile and run Firefox, but using *alloc() and free(), etc to establish the correct machine state to browse websites after learning what that state is in the abstract from Firefox’s code.<p>My thinking is many of our “truisms” in IT are outdated given modern machine performance and network reliability relative to the 80s and 90s when many of those “truisms” were defined.

So, how does it handle diamonds in the dependency graph? How does it handle version compatibility? IMO these are the most interesting aspects of a package manager, other stuff are implementation detail.<p>Although there is a mention for semver at the end:<p>&gt; Decide what versioning scheme to use (Probably semver, or something like it&#x2F;enhancing it with a total order).<p>I wonder if total order is appropriate (assuming the relation &lt;= means compatibility), but it sure simplifies things. Basically throw away the major version, embed that in the name of the package if you have to.

Man, I really wish frontend package management could be this simple, but with a system that ultimately ships bytes over the wire, we have to not only massage the dough with chopsticks (insofar as try to reduce our dependency tree with weak, feckless tools), but also, with the sheer number of transitive dependencies for each package addition, it’s almost impossible to really know what’s going on with your dependencies unless you write plug-in after plug-in.

Not a package manager, but I made this somewhat relevant &quot;app&quot; if anyone&#x27;s interested:<p><a href="https:&#x2F;&#x2F;sr.ht&#x2F;~tpapastylianou&#x2F;misc-updater&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;sr.ht&#x2F;~tpapastylianou&#x2F;misc-updater&#x2F;</a><p>It has solved my own pain points elegantly, so happy to share.

Why do we need a separate package manager for every programming language and every extensible library&#x2F;app?

Isn&#x27;t this missing the SQLite version?<p>Are SQLite extensions guaranteed to work across different SQLite installs?

Wonderful post, how about using standard OCI to unify products?<p>We have also been working hard recently on a package manager for configuring language KCL, which currently supports Git, OCI, and more.

Writing a package manager is not one of the most common programming tasks. After all, there are many out-of-the-box ones available.