Someone said on HN a while back that this sort of approach is the most sensible, because any design doc is just an incomplete abstraction until you get in the weeds to see where all the tricky parts are. Programming is literally writing plans for a computer to do a thing, so writing a document about everything you need to write the plan is an exercise in frustration. And if you DO spend a lot of time making a perfect document, you've put in far more work than simply hacking around a bit to explore the problem space. Because only by fully exploring a problem can you confidently estimate how long it will take to write the real code. Often times after the first draft I just need to clean things up a bit or only half complete a bad solution before improving it. Yes, there are times you need to sit and have a think first, but less often than you might imagine.<p>Of course, at a certain scale and team size a tedious document IS the fastest way to do things... but god help you if you work on a shared code base like that.<p>I've always thought the rigid TDD approach and similar anti-coding styles really lend themselves to people that would rather not be programming. Or at least have a touch of the OCD and can't stand to not have a unit test for every line of code. Because it really is a lot more work both up front and in maintenance to live that way.<p>Cyber-paper is cheap, so don't be afraid to write some extra lines on it.
I think it's a dangerous philosophy for professional work. You know what happens to code that works? It ships.<p>If you code out a solution to the problem in order to discover the problem space, I think the idea here is that you then can go back and write a better solution that accounts for all of the stuff you discovered via refactoring and whatnot. But you're not going to do that refactoring. You're going to ship it. Because it works and you don't know of any problems with it. Are there scaling problems? Probably, you haven't run into any yet. Does your solution fit whatever requirements other interested parties might have? Who knows! We didn't do any designing or thinking about the problem yet.
I am an avid discovery coder and was actually day dreaming an outline for a similar article on my way home on the bus today. I think this is an extremely important concept at all levels of engineering and something we all need to adopt at one point or another in our careers / practice.<p>I think it follows a few topics, "the art of the POC/Spike" or just exploratory coding. These things give us a tangible hands on approach for understanding the codebase, and I think lend to better empathy and understanding of a software system and less rash criticisms of projects that may be unfamiliar.<p>This is particularly relevant to me right now as I am discovery coding a fairly large project at my company and working with product to lay out design and project planning. Whats difficult to express from my current standing is how the early stages of these types of projects are more milestone / broad based rather than isolated small key pieces. Sure I can spend a week delivering design, architecture, epic, outline docs for all the known and unknown features of the project (and I am). But at the same time I need to discover and test out base case / happy path solutions to the core business problem to more accurately understand the scope of the project.<p>I think its something I particularly love about being a TL / IC at my company. I have the flexibility and trust to "figure it out" and the working arrangement to provide adequate professional documentation at the appropriate time. I am fortunate to have that buy in from leadership and certainly recognize it as a unique situation.<p>All that being said:<p>1. Learn how to effectively isolate and run arbitrary parts of your system for YOUR understanding and learning
2. Make it work, make it right, make it fast.
3. Learn to summarize and document your findings in a suitable fashion for your situation
4. Encourage this throughout your team. Useful in all aspects from bug triage to greenfield work
One trick I find helpful is to <i>start</i> by coding a subset of the problem with the goal of understanding the structure better. A brute-force solution, a simulation, a visualization of data, etc. And then use the discoveries of that process to do the real planning.
I call this exploratory programming, though my approach aligns more with the article I posted here, than with the Wikipedia definition.<p>I primarily use this method as a step preceding the actual production-quality implementation. It’s not like a prototype—I don’t throw everything away when I’m done. Instead, I extract the valuable parts: the learned concepts, the finished algorithms, and the relevant functions or classes. Unit tests are often written as part of setting up the problem, so I lift those out as well.<p>I’ve greatly enjoyed this approach, particularly in JavaScript&|TypeScript. Typically, I solve the difficult parts in a live environment and extract the solutions when I find them. I used to use my own "live environment" (hedon.js), but I eventually reversed the approach and built an environment around the built-in Node.js REPL (@dusted/debugrepl). I include this, at least during debugging and development builds, allowing me to live-code within a running system while having access to most, if not all, of the already-implemented parts of the program.<p>This approach lets me iterate at the function-call or expression level rather than following the traditional cycle of modifying code, restarting the program, reestablishing state, and triggering the desired call, something that annoys me to no end for all the obvious reasons.
I really appreciate this essay.<p>I've never been a [traditional] artist, but I reckon that those working in the arts, and even areas of the programming world where experimentation is more fundamental (indie game development, perhaps?), would intuit the importance of discovery coding.<p>Even when you're writing code for hairy business problems with huge numbers of constraints and edge cases, it's entirely possible to support programmers that prefer discovery coding. The key is fast iteration loops. The ability to run the entire application, and all of its dependencies, locally on your own machine. In my opinion, that's the biggest line in the sand. Once your program has to be deployed to a testing environment in order to be tested, it becomes an order of magnitude harder to use a debugger, or intercept network traffic, or inspect profilers, or do test driven development. It's like sketching someone with a pencil and eraser, but there are 5-10 second delays between when you remove your pencil and when the line appears.<p>Unfortunately, it seems like many big tech companies, even that would seem to use very modern development tooling otherwise, still tend to make local development a second class citizen. And so, discovery coders are second class citizens as well.
I believe this is actually where Gen AI tools like Claude.AI come into their own. For example, in the past few weeks we needed to plan a complex integration project with both frontend and backend integrations, dependencies on data provided in backend and frontend, need to send data backwards and forwards from the 3rd party and our backend, etc, and in total probably a half-dozen viable alternative ways of doing it.<p>Using Claude plus detailed prompting with a lot of contextual business knowledge, it was possible to build a realistic working prototype of each possible approach as separate Git branches and easily demo them within about two days. Doing this also captured multiple hidden constraints aka "gotchas" in the 3rd party APIs.<p>Building each of these prototypes in the working Java codebase would have been a massive, time-consuming and pointless activity when a decision still needed to be made on which approach to go with. But getting Claude AI to whip up a simplified replica of our business systems using realistic interfaces and then integrate the 3rd party was super-easy. Generating alternative variants was as simple as running a script to consolidate the source files and getting Claude to generate the new variant, almost without any coding needed.<p>And because this prototype was built in few html and js files and run using node js, there is literally zero possibility of it becoming part of the production codebase.
This reminds me of a tongue-in-cheek phrase we used to use in college.<p>"Hours of coding can save minutes of planning."<p>"Discovery Coding" sounds fun, but be careful with your time!
Reminds me of the alleged programming approach of Dr. Joe Armstrong of Erlang fame (RIP): write a program, then rewrite it, then rewrite it, and so on until it's good enough.<p>That's also how I tend to program, though usually as an accidental consequence of my ADD brain getting distracted, then being entirely dissatisfied with my code (or worse: I was too clever with it and it's indecipherable) when I come back to it, prompting yet another rewrite.
Never thought of it this way, but it makes sense. My default response to probing/planning type questions from business is "uhhh no clue, I have to dive into the code first and find out" precisely because of this.
My workflow these days is too start by writing a Python notebook that solves the problem. There's no faster way to iterate on writing something. Once it works I usually have o1 Pro write tests, clean it up, and then convert it to whatever language I actually need it written in.
I use a mixture of both discovery and planning. The problem with pure discovery approach is I sometimes start to code when I am not completely attentive because of deadlines, and if I am working with a larger codebase it is difficult to instantly start writing code because you need some time to understand the context. I always have a text file open all the time where I note down stuff such as: context, plans, ideas, edge cases, conflicts, todos etc.; it serves as a swap memory.
In Pharo discovery coding is really encouraged. I personally like it for writing scrapers and/or web tests (I use Selenium in Pharo). I once had an improvized talk about it [1]. Nowadays, I'm back to Python and JavaScript due to their ecosystems, but for discoverability coding and Selenium-based testing/scraping, I still think the interactive experience in Pharo is unmatched.<p>[1] <a href="https://youtu.be/FeFrt-kdvms?si=g12m7aZtDWtMMgwJ&t=2270" rel="nofollow">https://youtu.be/FeFrt-kdvms?si=g12m7aZtDWtMMgwJ&t=2270</a>
The actual term for a "discovery writer" is a "pantster" - i.e., you're writing by the seat of your pants - and I think that's a reasonable term to adopt here too.<p>Confession: I'm a pantster in writing both code and prose. In both cases, coming back and writing a spec (an eng spec in the case of code, a synopsis in the case of prose) is a reasonable thing to do. Structure is good, but the point is that it shouldn't get in the way of actually getting started and making some progress.
> I don't think many tools today are designed with discovery coding in mind. Things like live programming in a running system (see how (most) Clojurists or SmallTalkers work) are incredibly valuable for a discovery programmer. Systems that let you visualize on the fly, instrument parts of the system, etc., make it much easier and faster to discover things. This is the sense of dynamic (unrelated to typing) that I feel we often lose sight of.<p>Python's REPL achieves this nicely, but also in C# in Visual Studio, the Immediate Window, lets you type out code when you debug and have a breakpoint set. I almost always copy an If statement's expression there, and pase it, and get back either "true" or "false" which usually tells me insanely quickly if my assumption is spot on or not.<p>I like that Blazor has Hot Code Reloading, though it is definitely finicky.
Feels unnecessary to call it _____ coding and try to bless it with the air of methodology when it's just hacking or jamming just as you would on guitar or a drawing or other manual creative activity. you're just moving forward with whatever level of intuition your experiences and learnings afford you :shrugs:
Common Lisp, with its excellent REPL, is great for this type of exploratory work. The ability to build up a function from the inside out, and to build systems and classes from functions, again from the inside out, trivially creating mocks and ad-hoc tests as you go, is fantastic DX.
I feel I finally can put a name to my coding style! Incidentally, I also like to pretend classes and functions exist even if they don’t. Of course my discovery code won’t work, but I can go very far and discover a lot of useful information this way.
Interesting piece. As a coder and writer (hardcore outliner here), I’ve thought about this too.<p>I wonder about writing user stories for fiction. If software is something people use to realize an outcome, fiction is also something readers consume to realize an outcome. What might a “reader story” look like for some of our favorite novels? What kind of impression or change does a writer seek to produce in a reader’s mind? Such documentation could be valuable, similar to requirements documentation.<p>Aside to the author: King’s first name is spelled with a “ph.” Sorry, long time fan here :)
Related concepts in Peter Naur's "Programming as Theory Building" [0] or Gerald Sussman's "Problem Solving by debugging-almost Right Plans" [1]<p>[0] <a href="https://pages.cs.wisc.edu/~remzi/Naur.pdf" rel="nofollow">https://pages.cs.wisc.edu/~remzi/Naur.pdf</a>
[1] <a href="https://www.youtube.com/watch?v=2MYzvQ1v8Ww" rel="nofollow">https://www.youtube.com/watch?v=2MYzvQ1v8Ww</a>
Guilty, but as with anything there is a right time to use it and a time to most definitely avoid it. If you are working with others in particular there is very little room to wing it.<p>Experienced devs can do so in a first pass to flesh out an idea before letting the team get involved, but thereafter the design immediately becomes rigid. Inexperienced devs can wing it to learn in an exploratory way, but their work is unlikely to be re-purposable.
Am an "outliner", currently working together with a "discovery coder" on a project. We are half a year in and have no common working build, just my "outline" and a bunch of non-integrated throwaway discovery bits. I do believe that eventually they will produce the solution, but it is very hard to reason about the timeline in such a setup.
I call this exploratory prototyping and I think it's an absolute super-power. Sitting down and noodling with a prototype for 30-90 minutes is an incredibly powerful way to build a deeper intuition for a project. You can throw that all away and still be in a much stronger position to design the approach (and have useful conversations about it).
I wish was a more recognized process. Build a rough plan, then poke, and adapt.<p><a href="http://lambda-the-ultimate.org/node/5335" rel="nofollow">http://lambda-the-ultimate.org/node/5335</a>
I think if you also try to constrain yourself to only building orthogonal components it leads to a needs-based Lego set. I think it is difficult to see the underlying symmetry in a problem space if you design top-down.
For a similar set of ideas, see "A high-velocity style of software development".<p><a href="https://news.ycombinator.com/item?id=42414911">https://news.ycombinator.com/item?id=42414911</a>
How is this different from <a href="https://en.wikipedia.org/wiki/Exploratory_programming" rel="nofollow">https://en.wikipedia.org/wiki/Exploratory_programming</a>?
It took me 10+ years to realizing I’m the opposite of a discovery coder. I’m much more efficient with thinking through the problem with pen and paper before even touching the keyboard.
There's a time and place for everything.<p>I find that this style of programming is super useful in the technical design phase. During development I prefer to have a plan in place.
I worked with a manager who was diametrically opposed to writing a singl single of code until a test had been written for it.<p>That's when I discovered I hate test driven development.
> Discovery coding is a practice of understanding a problem by writing code first, rather than attempting to do some design process or thinking beforehand<p>When you write larger systems you better start making up your mind about the domain, understand it, come up with concepts and names that make sense instead of starting to code right away. Yeah, there are problem domains where an exploratory approach makes sense, but not when you create a product or a conplex system.
In my experience, some projects are very suited to discovery coding.<p>For my Machine Learning projects, I usually run a bunch of experiments either using pytest or directly on jupyter. I tend to plot out stuff and try out different types of feature engineering. Once I have nailed the approach, I then port a subset of this to a different jupyter notebook or python script, which is cleaner and more readable. This is because ML experiments are compute bottlenecked. So I want to ensure I spend enough time to pick the best features and model.<p>At work (which is not ML-related), I tend to do much less discovery coding because most of the unknowns are business related - does this API (owned by another team) scale, what is the correct place for this business logic etc. And, doing a viable Proof of Concept is time consuming, so I'd rather spend the time sweating out the nitty-gritties with product. The Discovery here must happen in the discussion with Product or other stakeholders because that is the expensive part. This is also why Product changing the Spec once the project is underway is infuriating. Sometimes a good chunk of the discovery is nullified.
I like it. This reminds me of the "tracer bullet" concept, as described in The Pragmatic Programmer by Andrew Hunt and David Thomas. As physical tracer bullets are used to illuminate the path to a target by glowing as they travel, the aptly named software development approach aims at creating a functional, end-to-end version of a system early in the project lifecycle. This strategy balances discovery and outlining, by deliberately planning for development of the minimal functional path that will help discover and overcome criticalities. No big castles of imagination, neither wasteful roaming, but targeted hands-on discovery. Key points here are: end-to-end functionality, real code (not throwaway, unlike a mockup or sketch), feedback-driven, interative improvements, risk reduction. Benefits: clarity, alignment, momentum, adaptability.
I always appreciate someone coining a useful expression, but I gotta say, sometimes these coiners are trying a bit too hard, no? I think I'll stick with "exploration-based", "exploratory" or "explorative" programming, which are commonly used and all sound less awkward to me than than "discovery coding". But hey, if that's what people flock to I'll get with the times! I've always used a combination of both approaches and the descriptions made here are great.
> For some reason, we have no such distinction in programming, so I am here to introduce it.<p>What? Of course there is. Exploratory programming and just writing code without big or even little design is (or was?) incredibly common. It doesn’t scale beyond one or two programmers so we don’t talk about it very much, but popular products like Minecraft originated this was, and almost definitely lots of one off useful tools fall into this category as well.<p>Even in big products, proof of concepts or prototypes can and are often done this way.<p>is incredibly
“A programming language is for thinking of programs, not for expressing programs you’ve already thought of. It should be a pencil, not a pen.” - from PG’s “Hackers & Painters”
<i>There is no reason that a discovery programmer cannot create a highly structured, rigorous end-artifact.</i><p>They often stop before they do.<p><i>Designing software is a human process and not all humans are the same.</i><p>...nor is the quality of their results.