This seems to be missing the obvious point: Make isn't anchored to any single language. If you will only ever use npm for your entire life, then you go ahead an live happily in package.json script land.<p>But I've lost track of the number of languages and environments that I've worked in. Make ties it together by being the Good Enough anchor point that documents and launches all the other tools and compilers.<p>- Compile this Rust app with Cargo then flash it to the esp32? Make.<p>- Build this page with JS using Tool Of The Week and push it to a test container? Make.<p>- Compile ancient C app and build a .deb? Make.<p>And so on.<p>Why not use shell scripts you may ask? Because shell scripts are way too free form and your tastes will change over time. Make forces just enough structure that you won't get carried away with yourself. Not for the basic task running stuff anyway.
One thing I appreciate about the javascript ecosystem and "scripts" is the focus on keeping everything contained in the project. The only assumption about the system is that it has node, from there it's npm i, npm start. Scripts are a way to make sure people always run the project-local version of various development tools and not whatever they happen to have installed.<p>When I see a makefile, I expect they will be making more assumptions about my system and expect a little more work.<p>I do appreciate the "has worked fine for decades" aspect of makefiles, and I guess docker solves some of this, but I still prefer a fully self contained javascript project.
I would like to call attention to `just` (<a href="https://github.com/casey/just" rel="nofollow">https://github.com/casey/just</a>) which is a modern take on "make if we didn't have to keep it backwards compatible", and more focussed on just task running rather than tracking dependencies.
I'm pleased to see this being espoused. One thing I always tell junior people is ""don't be tempted to think of make(1) as out-of-date [1], and to view packages available via npm etc as better replacements, as many tasks fit the model of make(1), if not the title "a program for directing recompilation" "" I am often viewed with suspicion until I show them a (much more compact) replacement for their custom python/js program expressed as a makefile without any of the process management requiring debugging. :-)<p>An alternative approach for the "batches of files to process" situation is to generate the commands and feed them to GNU parallel for execution.<p>[1] no pun intended
May I also direct your attention to self-documenting Makefiles:<p><a href="https://www.thapaliya.com/en/writings/well-documented-makefiles/" rel="nofollow">https://www.thapaliya.com/en/writings/well-documented-makefi...</a>
Make has been around for decades, and with good reason. I've enjoyed watching the grunts and gulps and whatevers appear as the New Shiny Thing, become complicated and collapse under their own weight as people struggle with overly complex configs, as I quietly do the thing that has always worked and has not changed in forever.
Over here in crazytown I'm using Makefiles to automate system deployments.<p>I thought it was insane at first but actually I've come around to it; Make hits that sweet spot of being ubiquitous, simple, and flexible. There's a little bit of a learning curve but once you're over that initial hurdle it's pretty straightforward, especially for one-way operations, e.g. install without uninstall, build without clean, etc.
I've only started using a Makefile ~two years ago when I started a big Go project at work, and I quite like it. There's a few caveats here and there - Makefile specific syntax, shell-script specific syntax, and whether I should put .PHONY in front of every command (I guess 'no unless you have a file with the exact same name'?), but it's quite compact and straightforward.<p>The most complicated command I have is something that removes a folder or set of files, invokes the swagger generator with a list of options, copies it to a target folder, and passes it through a formatter/processor. But it's very straightforward, and it's "just" shell script, not shell script wrapped in a JSON document, or some Java tool invoked indirectly through an XML configuration file like back in the day with Maven.
Reminds me of something I wrote years ago: <a href="https://blog.jgc.org/2010/11/things-make-got-right-and-how-to-make.html" rel="nofollow">https://blog.jgc.org/2010/11/things-make-got-right-and-how-t...</a>
Why not both?<p>Use make for dependency graph, but also use "npm run" or "yarn run" to make sure you're running with the correct Node and correct paths to all your tools.<p>I know you can specify those in your Makefile too, but the plus of running via npm/yarn is that they know where your binaries live, where your mode_nodules are, and that sort of thing.
For a lot of the same reasons in this article I like to use tup[0] when I can. It doesn't integrate into anything which is both good and bad. I wish my IDE could check with tup to see what dependencies get pulled in by what files. However, it's nice that it doesn't care what language or ecosystem I'm using.<p>Also, it's very strict about declaring dependencies properly, and will actually fail the build if you've set things up in a way that depends on something not tracked (by watching filesystem access as the compiler runs). That gives me warm fuzzies that my builds are reproducible.<p>Also I can get a neat dependency graph as a PNG if I want.<p>[0] <a href="https://gittup.org/tup/" rel="nofollow">https://gittup.org/tup/</a>
The problem I have with Make is that it has a syntax that most web developers I have worked with would find archaic and unfamiliar.<p>On some projects I've worked on, it's been used as basically a command alias system that only a few people can maintain. None of its caching or dependency chain capabilities were utilized.<p>In these cases, shell scripts would have been a better option, and in some cases were later introduced with success.
I use makefile for all my builds.<p>cmake can help portability across OSes but it has a layer of its own complexity, I mostly work on linux alone, makefile seems more than enough.<p>google etc produces new tools to build its huge code base but I don't have that large scale source code, makefile so far worked well for my scale.<p>unless you have specific needs I feel makefile will do just fine. it's simple, readable, manageable, and get the job done fast.
I also use Makefiles, been using them for 40 years. I find that Makefile targets for misc. things like grabbing remote training data, running tests, building documentation, etc., etc. augments information in READ files, that is in addition to functionally saving time, serves as documentation when I haven’t looked at a project in 6 months.
Why I prefer package.json over makefiles:<p>- turborepo (<a href="https://turborepo.org/" rel="nofollow">https://turborepo.org/</a>) can describe dependency pipelines and provide automatic caching. Makefiles aren't designed for multi-input, multi-output scenarios - its really awkward: <a href="https://www.gnu.org/software/automake/manual/html_node/Multiple-Outputs.html" rel="nofollow">https://www.gnu.org/software/automake/manual/html_node/Multi...</a> and <a href="https://stackoverflow.com/questions/39237306/makefile-compiles-all-tsc-regardless-of-changes" rel="nofollow">https://stackoverflow.com/questions/39237306/makefile-compil...</a><p>- turborepo can also run never-ending targets in parallel (e.g. API server and static file server in development mode). Not sure how well make supports that<p>- turborepo can depend on env var values. Makefiles can to, but like everything with makefiles, its an awkward workaround: <a href="https://stackoverflow.com/questions/14840474/make-target-which-depends-on-an-environment-variable" rel="nofollow">https://stackoverflow.com/questions/14840474/make-target-whi...</a><p>- makefiles do not work on Windows (They are portable, just not to platforms most node devs care about)<p>- Its unclear whether `make` will ever add features to remove the awkwardness for supporting the various scenarios above. It doesn't seem likely to happen.
For the clojure crowd, there's bb tasks [0].<p>* parallelism<p>* hooks (before/after each task, etc.)<p>* command-line arg parsing<p>* --help equivalent<p>* bash/zsh/fish autocompletion<p>* you can also just spawn a shell with bb/shell<p>* yes, dependencies<p>* regular programming language (Clojure), with commonly used shell functions like glob()<p>* import code from locations (don't need to copy/paste between Makefiles)<p>* built-in JSON and CSV support, so you can use your app's JSON files right in your build file<p>[0]: <a href="https://book.babashka.org/#tasks" rel="nofollow">https://book.babashka.org/#tasks</a>
Stick around in any programming ecosystem long enough, and you live long enough to see strategy, taste, and "discovery" come full circle. We had make, then package.json "scripts," then grunt, then gulp, then the trend shifted back towards package.json "scripts." (Note: Using "scripts" to differentiate the package.json property versus the generic word use)<p>I like that the author isn't being authoritative, but there could be some additional due diligence. I ran the make-is-faster loose benchmark via PNPM and runtime was 0m0.022s for make and 0m0.012s for PNPM. If I care about those 10ms, PNPM is my horse. Yarn is a glacier compared to PNPM.<p>Another thing I would've liked to see comment on is the automatic pre and post paradigm that package.json "scripts" affords. The big three package managers all support pre and post, and it makes arranging "scripts" a breeze, it breaks down dependent steps into separate console output, and is generally easy to organize imho.<p>All in all a nice write up for folks who might not really like package.json "scripts" to begin with, or for those who'd rather not gain more granular insight into how "scripts" works, but I don't see this being the nail in the coffin case against them.
(GNU) Make's advantages include ease of use, a modicum of portability, and support for a variety of different software development command line stacks.<p>Unfortunately, the shell commands that are typically setup in a Makefile, are rather unportable. They waste a touch of time and space. They can break in surprising ways.<p>For this reason, I try to write my build tasks in terms of the same programming language as the application language. For example, Grunt for JavaScript projects, Shake for Haskell projects, Gradle for Groovy projects, Rez for C/C++ projects, Dale for D projects, TinyRick for Rust projects, and Mage for Go projects. Leiningen for Clojure projects, SBT for Scala projects. Vast for shell projects.<p>Make, like Python, suffers from the appearance of simplicity, at the cost of long term maintainability. Ideally both the build system and the main application are compiled. That doesn't have to be a cumbersome process, but rather a guide to identify potential bugs sooner during development.<p>Make is my first choice for random projects, but only when better build systems are unavailable.
I use make to run daily/hourly/etc batch processes in the ERP system at work. There's a Python shim to actually run processes in the ERP, plus a shell script to get things ready to run and send success/failure emails, and cron to kick batches off, but other than that it's pure make. In theory I should also be able to pass '-j 4' to make and have it run multiple processes at once, in the correct order with their dependencies properly satisfied, but I haven't actually tried that yet.<p>It's clearly an abuse of a build tool, but it's also a testament to make's incredible flexibility.<p>I've also heard of someone who replaced their Linux startup scripts (in the pre-systemd days) with make to speed up boot time. That's actually what inspired my batch schedule work.
I like Make, and use it in personal JavaScript ecosystem projects that do actually need to build things with interdependency (eg <a href="https://github.com/justjake/quickjs-emscripten/blob/master/Makefile" rel="nofollow">https://github.com/justjake/quickjs-emscripten/blob/master/M...</a>). I’ve also seen Makefiles grown to be horrendous monstrosities masquerading as command-line tools; for about a year at Airbnb we used a 1000+ line Makefile as the main tool for fiddling with Kubernetes cuz one of our senior engineers didn’t like Ruby, and I’ve seen another one get close to that level of cravenness.<p>What I learned from supporting Make and shell among a few different audiences is that most developers have no interest in how to write or maintain shell-like tooling. They forget or mess up quoting rules constantly, and eschew learning things and good design in these tools to a much greater degree than in their “normal” work in Java/Python/Ruby/Typescript/Golang.<p>For every POSIXly Correct HN Commenter (of which I count myself a member), there’s 100 regular software engineers who won’t read a `man` page on what $@ or $< mean in Make. I know that if I start writing a Makefile for $JOB, it’s gonna be me and that one guy who uses tcsh who are gonna maintain it and answer questions.<p>(Although for what it’s worth we don’t use package.json scripts either thank goodness. All our complicated build steps are typescript commands, and our glue is CI system YAML files.)
I'm also a fan of make but whenever another frontend dev starts working with one of my projects the first thing they do is rip out the perfectly good Makefile and replace it with npm scripts (often introducing errors). I guess it's a generational thing.
I don't have a problem with make files, but I do prefer to work directly with `package.json` when reasonable. Most notably, most of the IDE's work very well with `package.json`, but don't always handle make as cleanly.
I was amazed to discover Taskfile. Didn't realise it was a copy of Make.<p><a href="https://github.com/adriancooney/Taskfile" rel="nofollow">https://github.com/adriancooney/Taskfile</a>
<a href="https://github.com/ysoftwareab/yplatform" rel="nofollow">https://github.com/ysoftwareab/yplatform</a> that this a notch higher (disclaimer: author here)<p>As others have mentioned on this thread, the problem is not just "use Makefiles instead of package.json scripts", but quickly ending up with duplicate Makefile content and then supporting more than just one language. It's inevitable.<p>PS<p>for those that don’t know, npm’s “scripts” functionality as we know it today was not “designed”, but it was simply a byproduct.<p>First introduced as an internal functionality for lifecycle support <a href="https://github.com/npm/npm/commit/c8e17cef720875c1f7cea1b49b7bc9f1325ccff5" rel="nofollow">https://github.com/npm/npm/commit/c8e17cef720875c1f7cea1b49b...</a> on 1 Mar 2010<p>and later extend to run arbitrary targets <a href="https://github.com/npm/npm/commit/9b8c0cf87a9d4ef560e17e060f4ddc03b2ff1a1c" rel="nofollow">https://github.com/npm/npm/commit/9b8c0cf87a9d4ef560e17e060f...</a> on 13 Dec 2010<p>This is the entire design backstory <a href="https://github.com/npm/npm/issues/298" rel="nofollow">https://github.com/npm/npm/issues/298</a> .<p>Needless to say that GNU Make’s equivalent (or any other build system) is not to be found in npm’s scripts.
We are using make targets to automate ci/cd, local dev and a lot of other things by adding a set of easy to extend, customise template makefiles as submodules.<p>The main advantage for us is it's same everywhere and it's agnostic to underlying technologies. (we are only supporting unix based environments and have guidelines to setup gnumake in macos).
By using this way, we ensure that the general SDLC is same across different repositories and underlying tooling can change anytime without inducing a lot of refactoring<p>We are also using make targets to document themselves. How to use, what are other targets, variables you need to define etc. This makes it a powerful CLI app for us that can run and support many things.<p>For anyone interested: <a href="https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile" rel="nofollow">https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile</a>
and the similar parsing method for self documenting makefiles: <a href="https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile/-/blob/master/help.mk" rel="nofollow">https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile/-/blo...</a>. We didn't know about magicmake that's linked here that does the similar thing
Just wanted to share <i>nps</i> as a nice way to manage scripts for your <i>package.json</i><p><a href="https://www.npmjs.com/package/nps" rel="nofollow">https://www.npmjs.com/package/nps</a><p>You can create a dedicated JS file (with comments and more!) that will house your scripts, which you can run by using "nps" instead of "npm" as a command.
Out in Grand Rapids, Mi where AO is located there seems to be a bit of a tradition of abusing make in a good way. It's where I picked up the habit as well. I think its just institutional knowledge/tradition that has creeped through the various firms here.<p>I kind of wonder what other "traditions" get passed around on a regional level?
But you don't write scripts in package.json and it also failed to show an example how the javascript dependencies could be loaded with make.<p>I think make works well in combination with package.json, but not as a replacement. You can also do the same stuff in an NPM command (which just loads node code) that you could do in a makefile script.
I have very similar use-cases, and I also I found makefiles a bit limiting. I wrote lk[1] to make this sort of thing easier, and you can just write plain bash functions instead of makefiles.<p>[1] <a href="https://github.com/jamescoleuk/lk" rel="nofollow">https://github.com/jamescoleuk/lk</a>
The second I see a node project built in make, I remove it<p>That's a person who doesn't understand portability, tooling, or the need to stick to community norms. Everything about their library is going to be pure pain.
Back when I was doing network engineering I used make to localize configs from boilerplate code templates. Not a common use for it or maybe even the best tool for the job but because I already knew Makefiles it was an easy solution for me to implement. Saved me endless hours of tedious error prone text editing. Just an example of why it's worthwhile to learn how to use these fundamental tools that have survived the test of time. Even if you don't have a specific use for it today it's a very useful tool to have in your toolbox.
At one point I tried to do the same thing but then I realized there isn't any make feature I need to use. The key thing about make is to avoid building already built artifacts, if you don't need that you end up with things like .PHONY. There is'nt anything in make that helps with composing or showing a list of scripts etc either. At the end I ended up with plain javascript scripts for js project and plain bash scripts for everything else.
package.json scripts are definitely a bit out of place. I think their prevalence stems from popular npm projects using the pattern for easy cross-platform developer experience. I've seen some shell scripts in opensource node projects, but not much makefile, and I actually had an interviewer once joke that makefiles were old and out of style.<p>But as I have worked with less Windows developers over time, my projects have relied more on shell scripts and makefiles. I think the containerization movement has started to shift software development towards Linux-based tooling as well. As a cross-platform effort this is a bit ironic, but having overlapping developer experience and CI/CD tooling is pretty convenient when you're on a Unix kernel or WSL2.<p>However, much of the npm ecosystem and community are focused on frontend work that often does not involve infrastructure-related tooling. Without the absolute need for any OS-specific tools, and with the many composable crossplatform scripts in vogue already, package.json scripts make sense (until they get out of hand).
Here is Makefile "starter" I use:
<a href="https://github.com/awinecki/magicfile" rel="nofollow">https://github.com/awinecki/magicfile</a><p>People call this "self-documenting makefile".<p>It migrated with me from company to company, from project to projects. Through node, php, aws, docker, server management, cert updates, file processing and many many more.
Earthly is an interesting open source project in this space that offers many of the benefits of a make file plus containerization plus some of the performance benefits of a Bazel/Pants.<p><a href="https://earthly.dev/" rel="nofollow">https://earthly.dev/</a>
It’s helpful for mitigating an NPM arbitrary code execution vulnerability along with ‘ignore-scripts=true’<p><a href="https://blog.uidrafter.com/getting-rid-of-npm-scripts" rel="nofollow">https://blog.uidrafter.com/getting-rid-of-npm-scripts</a>
The argument about variables in makefiles was convincing to me, but looking it up I learned for the first time you can actually use variables in package.json files as well. After all these years mucking about with package.json...
Pretty convincing writeup. I've only used makefiles when compiling C / C++ but lately I'm finding even those are moving to CMAKE with makefiles becoming less and less common.