If you use GNU Make it's worth using my GNU Make Standard Library: <a href="http://gmsl.sourceforge.net/" rel="nofollow">http://gmsl.sourceforge.net/</a><p>And also reading everything I wrote as Mr. Make: <a href="http://blog.jgc.org/2013/02/updated-list-of-my-gnu-make-articles.html" rel="nofollow">http://blog.jgc.org/2013/02/updated-list-of-my-gnu-make-arti...</a><p>Or buy my book: <a href="http://www.lulu.com/shop/john-graham-cumming/gnu-make-unleashed/paperback/product-2937580.html;jsessionid=853955F416C443C6FCCCF7E7700BF62E" rel="nofollow">http://www.lulu.com/shop/john-graham-cumming/gnu-make-unleas...</a>
Absolutely do not use make for any new project. If you love make, it's a big, red, burning flag that you're not demanding enough of your tools and that you're not keeping up with changes in your ecosystem.<p>There are many, many way better alternatives to make. Which one is better depends on the platform you're on. The majority of them throws in automatic dependency management for free.<p>Yes, I know that the essence of the post is "use a build system". I agree completely. In fact, script <i>everything</i>. Then script your scripts. Then refactor your scripts because they are getting messy. But don't give impressionable souls the idea that "make" is anywhere near an acceptable (generalised, default) choice today.
If your project sports auto-generated code, or any sort of build targets that depend on inputs only known after generation -- Make simply cannot handle this. There are ugly workarounds but they don't work well.<p>Make is:<p>* Slow (e.g: when compared with tup[1])<p>* Very easy to get it wrong (under-specify dependencies), with cryptic bugs (or over-building) as a result. Pretty difficult to get it right (e.g: doing proper #include scanning, especially with the above-mentioned generated code).<p>* Terrible scripting language (mentioned by many others)<p>* Lacks useful features of other build systems (e.g: tup, or shake) of useful profile reports and queries about the build process.<p>It is just an antiquated, poor tool that ought to be replaced by betters that already exist, whenever possible.<p>[1]: <a href="http://gittup.org/tup/" rel="nofollow">http://gittup.org/tup/</a>
Everybody seems to be missing the fact that he's not talking about building a big software project. He's talking about scripting a simple workflow around a few files. Make is probably better than anything else for this, because under these circumstances it's so simple it's hard to get it wrong.<p>Something like Rake is obviously better when you need to really program your builds.
Using make gets quickly really ugly. I wrote many Makefiles in my life and if you have to try to stick to GNU Make. At least it provides some basic functionality. But even that gets nasty quite soon when you need a simple thing like conditions with "and" or "or". If you have to write portable Makefiles ... well fsck.<p><a href="http://www.conifersystems.com/whitepapers/gnu-make/" rel="nofollow">http://www.conifersystems.com/whitepapers/gnu-make/</a><p>There were recent discussions about adding GNU Guile to GNU Make. I think this is a brilliant idea. Not only would a major GNU project not only use Guile. But it would make writing Makefiles much more enjoyable because you have a real programming language at hand.
Most systems I use have GNU make 3.81 installed. GNU make 3.81 added the special target .ONESHELL which makes code like the following possible. I'm not sure if it is a good idea, but it does remove the dependency that make has on shell programming.<p><pre><code> .ONESHELL:
SHELL = /usr/bin/python
VAR := lorem ipsum
all:
@
import re
n = 3
print('make variable: $(VAR)')
print('local variable n: {}'.format(n))
rx = re.compile('\d+')
s = 'foo 17'
m = rx.search(s)
if m:
print('has number: {}'.format(s))
else:
print('no number: {}'.format(s))</code></pre>
I find this a really interesting conversation. On the one hand I've built some really really complicated systems with make (like all of SunOS) and some even more complicated systems with a custom build tool (the google base infrastructure packages) and there are pluses and minuses to both approaches.<p>If you're building something small, its hard to beat a simple make file. Its easy to write, it allows you to capture dependencies and refactor quickly, and it doesn't interrupt your coding flow.<p>If you're building something quite large there are some real productivity benefits from building knowledge of what you are building into the build system. And computational build systems (which is to say build systems where the build spec file includes the capability to do local computation) can make retargeting the same build to different environments easier.
Three years ago I wrote a blog post arguing in favour of Make : <a href="http://blog.jgc.org/2010/11/things-make-got-right-and-how-to-make.html" rel="nofollow">http://blog.jgc.org/2010/11/things-make-got-right-and-how-to...</a><p>I'm not sure I still agree with everything in that post, but Make is very terse and expressive which I like.<p>One of the biggest problems with Make is that it's a macro language and some people don't get along well with them.
Like everyone seems to be mentioning this post is really about "you should have automatic builds" and make is just an incidental. redo[1] is another interesting make replacement that keeps most of the good points of make (e.g., simple and shell based), while having a more powerful dependency mechanism. It's even compatible with make itself allowing you to move a part of a recursively built project to redo, while having it call into make for other subcomponents.<p>What I've done before when doing data-driven blog posts[2][3] was to write the whole analysis end-to-end in ruby, including branching out to R for stats and graphing[4]. I ended up using rake (ruby's make-like tool) to tie everything together with dependency tracking but I could just as well have written a script that would call everything in order. Doing that gives you a way to quickly reproduce results (what's discussed here) but also, together with version management, a way to go back to previous versions and use things like "git bisect" to figure out when you introduced a bug.<p>[1] <a href="https://github.com/apenwarr/redo/" rel="nofollow">https://github.com/apenwarr/redo/</a><p>[2] <a href="http://pedrocr.pt/text/how-much-gnu-in-gnu-linux" rel="nofollow">http://pedrocr.pt/text/how-much-gnu-in-gnu-linux</a><p>[3] <a href="http://pedrocr.pt/text/preliminary-results-open-source-evolution" rel="nofollow">http://pedrocr.pt/text/preliminary-results-open-source-evolu...</a><p>[4] <a href="https://github.com/pedrocr/codecomp" rel="nofollow">https://github.com/pedrocr/codecomp</a>
To everyone with alternatives to make:<p>Make is easy to learn.<p>It simplifies a slightly complex task.<p>Other build tools have a steeper learning curve. If they're more complex than the problem being solved, people won't want to adopt them.<p>When things get complex, there's the GNU make manual and libraries.<p>When you have to target multiple platforms, there's autotools, which is really complex and intimidating, but less complex than targeting multiple platforms.<p>Alternatives to make seem to fit in between make and autotools.<p>Make sucks. I've also heard that Unix sucks, and C sucks, too. Despite this alleged suckyness, these things not only persist, but accumulate improvements over the years.
Drake was just released, it's a make replacement specifically for data processing and seems to have some nice features.<p><a href="http://blog.factual.com/introducing-drake-a-kind-of-make-for-data" rel="nofollow">http://blog.factual.com/introducing-drake-a-kind-of-make-for...</a>
I like discussing build systems. However, personally I still come back to make all the time. There are certainly better alternatives (tup, shake, redo, etc), but make is simply available everywhere. It is available, if I use an OS which is a decade old. It will be available, if I use an OS in ten years from now.<p>I am writing this on Ubuntu LTS (12.04) and none of those three advanced build systems above is available from the default repos. This means a serious dependency burden, if somebody wants to build my stuff.<p>A few years ago, Jam was a promising build system. It seems to be dead now. Hopefully, tup and shake will stay longer.<p>Until you have good reasons against make, it is a perfectly fine initial solution.
Everyone points at make and says, "use something better". They point at Rake, they point at Maven. However, Make has one feature that I really need, that I rely on to speed up builds: parallel dependency construction.<p>The last time I checked, neither Maven nor Rake do this properly. Maven runs everything inside a single VM. Who wants to have to worry about multithreading in their unit tests? Rake requires thread support in the underlying Ruby interpreter (why?).<p>I've got 64+ hardware threads (Sparc T4-1) all sitting there waiting to be spun up and do my bidding. Please help me to convert electricity to heat!
I can't tell you how many projects I've shelved for months and when I returned thought, "Damn it. How do I build this again? Oh look a Makefile. Thank you Past Eric."<p>Even for projects that use more sophisticated build tools like rebar, leiningen or npm I write a Makefile so I don't have to remember those tools. Make provides a universal interface to those tools.
I've used Make in a similar context: building documents.<p>Raw simulation results (.log) ->
Processed for plotting (.plot) ->
Ugly Fig files (.x.fig) ->
Pretty Fig files (.fig) ->
EPS files (.eps) ->
The final document (.pdf).<p>By including the right dependencies in there, you can have individual figures update themselves when the raw data changes, and whole swathes of charts update themselves when the 'fixer' scripts get updated.
I once used gmake to implement a multi-stage Mechanical Turk workflow.<p>It was awful. The syntax sucks. But it worked consistently, and the core logic was only 110 lines of Makefile. It described the files and the data flow between them. Even now, I can read it and understand it with not too much effort.<p>Make is a very simple functional language. It's restartable. If you type 'make -j 2' it becomes parallelizable. For almost anything for which you might write a shell script, you have to ask yourself: Why not make instead?<p>I would like a cleaner, kinder make, but I also want it to retain make's essential make-iness. Nothing else seems to do that.<p>I do feel bad for anyone downstream of my makefiles, though.
I agree Make is great at what it does but too many people really abuse their makefiles and don't use dependency management correctly. I suggest taking a look at
<a href="http://www.amazon.com/Managing-Projects-Make-Nutshell-Handbooks/dp/0596006101/ref=sr_1_1?ie=UTF8&qid=1361732761&sr=8-1&keywords=gnu+make" rel="nofollow">http://www.amazon.com/Managing-Projects-Make-Nutshell-Handbo...</a><p>Really well written book. You don't have to agree with everything but its a great look at some of the better ways of using Make.
As noted in many of the comments, make sucks, and the article is not promoting make per se. But make seems to be the topic of conversation, so:<p>To me, make is a cruddy low-level declarative language that gets abused as a pseudo-imperative language, compounding the problem. Phony targets like "make <verb>" break the paradigm, because <verb> is not an artefact that can be tested for up-to-date-ness.<p>But one advantage of make that I'm seeing underrepresented in the comments is its ubiquity. If I just want to try your project, and I need to build your project to try it, and your project is using the cool new SchnauzerBuild[1] tool, and there's no SchnauzerBuild package for my OS, I have to go install that from source... which might have its own build dependencies... ok, your project looks kind of cool but I have better things to do than this.<p>I think it's great when build tools are able to write out a Makefile or sh[2] script that just builds everything, and when projects ship with that pregenerated. (and remember to keep it updated)<p>[1] fictional
[2] another technology that sucks but is ubiquitous
This is a sloppily phrased call for automation. I can wholeheartedly endorse automation as a goal, but the idea that make can be your go-to tool for this is embarrassing. Make will get you into trouble the moment your task becomes non-trivial. Everyone has their favourite tools and no single tool is best for every job (although grep and find are often your friend).<p>As I often say in job interviews, I'm lazy. I protect the lazy guy inside of me, because he's the one who cries out "Didn't we do this manually before? Shouldn't we automate it?" He's a good guy to have around.
"Makefiles are machine-readable documentation ... "<p>Yes it is machine readable but how about us humans? Makefiles are ugly and when working on someone else's project it is very hard to reverse engineer the build system. Build systems are themselves software projects and we need better tools to develop and maintain them.<p>SCons was a promising project at one point, it improved things by capturing the build system in Python classes. I thought things would be more maintainable and readable. However for me it wasn't a well design, it obscures build system development with mixed declarative and iterative programming.<p>Makefiles are the defacto standard today, but they're no where near beautiful, or maintainable, or readable.<p>There are also variations of it such as gmake, imake, and so on, who only add their own quirks without solving the real problems.
all these "make for beginners" tutorials i've seen point out (correctly) how make is, at its most basic, just a dsl for specifying a dag.<p>it seems to me that it would be pretty useful to have a tool that let you build up said dag graphically, perhaps dragging and dropping files from an explorer pane, and then generated a makefile under the hood.<p>add in simple "infinite undo" git support that checkpointed every time you built with new inputs, and you'd have a dead simple way for non-programmers (who nonetheless have to do some programming to work with their data) to get the benefits of programming best practices. does such a tool exist?
This is a nice article, especially on a general level of writing scripts that automate work so you don't have to redo it manually.<p>I want to point out one thing.<p><pre><code> targetfile: sourcefiles
command
</code></pre>
This violates DRY, there is duplication between command and source files (and target file?). In theory, it should be possible to automatically deduct the source files from the command.<p>It's far from easy in the general case, but it would save you from having to manually update source files when the command changes. (Any small inefficiency times lots of occurrences times lots of people really adds up.)
I've started using make for production data generation (not building software) instead of pure Python. Mostly to tie together Python scripts doing the real work.<p>Pro<p>-dependency management for free<p>-well-known paradigm makes it easy for someone other than me to figure out where to look if something went wrong.<p>-scripts I call out to can be focused<p>Cons
-syntax<p>-for processes that don't generate an output (think adding data to a file in place) I wind up creating placeholder files ("file.transformA.done").<p>I actually want my dependency management to be terse and declarative, which is the opposite of what I'm looking for in a programming language, so it feels like a pretty natural divide.
> You can approximate URL dependencies by checking the
> Last-Modified header via curl -I.<p>... or better, use the '-z' option.<p><pre><code> counties.zip:
curl -o counties.zip -z counties.zip 'http://whatever.zip'</code></pre>
No you shouldn't. I know make inside and out, a result of being stubborn and lazy, and I am positive I can build a better replacement for it in a matter of days. Something like, but more general than, Rake.<p>The way Make works is very complex with lot of implicit rules, special exceptions, etc. Debugging Makefile is a chore.<p>If you are bent to use it, two pieces of advice: there is a handy debug flag that will tell you everything make is doing, and do disable all the implicit rules.
More than convincing me to use Make for dataviz, this really begs the question, "Is there a good dataflow manager for data visualization?" Something that can use URLs as dependencies, simpler syntax than Make, perhaps has a node/pipe dataflow GUI... Cascading.org comes to mind, but it is too complicated and Hadoop-oriented for this kind of dataviz.
My problem with make for data pipelines is that a lot if decisions have to be content based instead of timestamp based. Multiple platforms is also an issue, I usually don't bother installing the whole Cygwin just for gnu make. I end up with custom python scripts which may be more verbose but more flexible and almost always cross-platform.
This is funny to read, especially after spending several hours trying to get a project to compile with make, while tracking down an annoying and cryptic error that doesn't even make sense.<p>Make is a horrible tool, and is extremely hard to learn. I think it should only be used when generated by other build systems.
Posted a new link to Fbuild. Its a build system that has quite a different and interesting take on the build process.<p><a href="http://news.ycombinator.com/item?id=5276504" rel="nofollow">http://news.ycombinator.com/item?id=5276504</a>
And don't forget how broken recursive make can in fact be: <a href="http://aegis.sourceforge.net/auug97.pdf" rel="nofollow">http://aegis.sourceforge.net/auug97.pdf</a> . I lot of builds are nasty "Heisenbuilds."
I think what you were looking for here is actually a shell script, rather than a Makefile. A shell script is much more suitable for general purpose programming. Or better yet, use Python ;-) And, let's admit it, for small projects dependency analysis is overrated (although it can be implemented easily enough in a real programming language).
<i>Created in 1977, Make has its quirks. But whether you prefer GNU Make or a more recent alternative, consider the benefits of capturing your workflow in a machine-readable format.</i><p>Like a programming language?