TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Out of the Tar Pit (2006) [pdf]

119 pointsby n0wabout 2 years ago

11 comments

dcreabout 2 years ago
This paper was very influential on me when I first started programming professionally around 2012. I don&#x27;t plan on reading it again, but my vague memory of what I got out of it is pretty simple and I think has become pretty standard practice at this point: avoid mutable state and use pure functions where possible.<p>The framing of accidental and essential complexity is of course very useful and not really unique to this paper. The difficulty is there is nothing but experience-informed judgment that can tell you which complexity is accidental and which is essential. There is always a framing of the problem that can justify some bit of complexity as essential. You have to judge.
评论 #34971617 未加载
评论 #34976983 未加载
评论 #34977598 未加载
评论 #34977577 未加载
dagssabout 2 years ago
I feel event sourcing is a real world pragmatic approach to declarative programming that this paper advocates.<p>For state changes you add events to the database to describe something that happened. Any question you may need an answer for &#x2F; business decision you want to make can be answered by querying the events.<p>The problem at the moment is that while event sourcing is excellent at reducing accidental complexity surrounding implementing business rules, there is little standard &#x2F; commonly used tooling around it and you end up with lots of accidental complexity in that end.<p>An example would be a database not designed to be a CRUD store but to store events and manage read models, and manage computation of projections etc -- while being suitable for OLTP workloads. At a minimum, very strong support for using any kind of construct in materialized views (since in a sense the entire business logic is written as a &quot;materialized view&quot; when doing event sourcing)
评论 #34972535 未加载
Verdexabout 2 years ago
In the past, I have been unimpressed by this paper. Perhaps someone can shed some historical context...<p>But from my perspective what happens is that the paper defines complexity in exactly the way that allows them to deride OO, FP, etc programming whilst simultaneously showing how awesome functional relational programming is. It ignores complexity that&#x27;s orthogonal to what FRP addresses and ignores areas in which FRP itself contributes to unnecessary complexity.<p>It feels like a scenario where the authors had something that they thought was neat and went out to create metrics that would in fact show that it was neat. Maybe FRP is really neat, but I feel that the paper itself doesn&#x27;t contribute to anything because its logic is so custom and purpose built.
评论 #34971110 未加载
ZitchDogabout 2 years ago
A classic paper with a dream that has yet to be realized. We continue to bolt state on top of state. Redux bolted on top of GraphQL on top of Redis on top of Postgres. We can do better.
评论 #34973743 未加载
cloogshicerabout 2 years ago
I think the crux of this paper is section 7.2.2:<p>&gt; There is one final practical problem that we want to consider — even though we believe it is fairly rare in most application domains. In section 7.1.1 we argued that immutable, derived data would correspond to accidental state and could be omitted (because the logic of the system could always be used to derive the data on-demand). Whilst this is true, there are occasionally situations where the ideal world approach (of having no accidental state, and using on-demand derivation) does not give rise to the most natural modelling of the problem. One possible situation of this kind is for <i>derived data which is dependent upon both a whole series of user inputs over time, and its own previous values</i>. In such cases it can be advantageous to maintain the accidental state even in the ideal world. An example of this would be the derived data representing the position state of a computer-controlled opponent in an interactive game — it is at all times derivable by a function of both all prior user movements and the initial starting positions, but this is not the way it is most naturally expressed.<p>Emphasis is mine.<p>I think that this type of derived data, which I put in italics above, is quite common - contrary to what the authors of the paper argue. Any UI code or game-like system, like simulations, will have this kind of data. And the paper does not have a good answer for it. I honestly think that nobody has an answer for it, and it&#x27;s why most of our UIs suck.<p>I would love to see something that makes handling this type of derived data easy.
评论 #34976215 未加载
n0wabout 2 years ago
I came across this after seeing relic[0] submitted the other day and thought it was pretty interesting.<p>I&#x27;ve been into CRDTs for a while and have started wondering about generic mechanisms for distributed data. This lead me to read a lot more about the Relational Model of data and eventually to the Event Calculus.<p>What&#x27;s interesting to me is that these things end up feeling a lot like CRDTs[1] or Event Sourcing. I haven&#x27;t quite finished pulling on these threads but the relic link was a timely read considering!<p>I really liked the first half of this paper and the Authors categorization of complexity. However the second half fell a bit short for me. It seems they made the same mistake as many other people (SQL != Relational) and their idea of Feeders and Observers seems a bit more like an escape hatch than an elegant method for interfacing with the outside world.<p>[0] <a href="https:&#x2F;&#x2F;github.com&#x2F;wotbrew&#x2F;relic">https:&#x2F;&#x2F;github.com&#x2F;wotbrew&#x2F;relic</a> [1] <a href="http:&#x2F;&#x2F;archagon.net&#x2F;blog&#x2F;2018&#x2F;03&#x2F;24&#x2F;data-laced-with-history&#x2F;" rel="nofollow">http:&#x2F;&#x2F;archagon.net&#x2F;blog&#x2F;2018&#x2F;03&#x2F;24&#x2F;data-laced-with-history&#x2F;</a>
评论 #34975166 未加载
carapaceabout 2 years ago
I&#x27;m designing a such a system now, based on the pure functional Joy language with two additional data stores: a relational db system (Prolog (or maybe Datalog), not SQL) and what is effectively a git repo although I think of it as a &quot;data oracle&quot;.<p>The role of the relational db system is explained in TFA. (Prolog makes a fine Relational Model DB (the &quot;relations&quot; in RMDBs are the same <i>logical relations</i> that Prolog &quot;relations&quot;, um, are.) The language is cleaner and simpler and more powerful than stock SQL, there&#x27;s an ISO standard and several solid implementations, and you can always back it up with SQLite or PostGRES or whatever if you need to.) The trick to integrating it with a purely functional system is to only use &quot;pure and monotonic Prolog code&quot; which you want to do anyway ( <a href="https:&#x2F;&#x2F;www.metalevel.at&#x2F;prolog&#x2F;debugging" rel="nofollow">https:&#x2F;&#x2F;www.metalevel.at&#x2F;prolog&#x2F;debugging</a> ) or as I like to say, &quot;Don&#x27;t put lies in your database.&quot;<p>The &quot;data oracle&quot; (which again is more-or-less just a git repo) provides bytes given a three-tuple of <i>(hash, offset, length)</i>. These are immutable, so you can cache the results of (pure) computations over them (e.g. a predicate like &quot;is valid UTF-8&quot; is true&#x2F;false for all time, yeah?) This replaces the filesystem.<p>I was working with Prof. Wirth&#x27;s Oberon RISC CPU as a basis, but a couple of days ago a fantastic new 64-bit vm went by here on HN and I&#x27;m going to use that going forward. <a href="https:&#x2F;&#x2F;github.com&#x2F;maximecb&#x2F;uvm">https:&#x2F;&#x2F;github.com&#x2F;maximecb&#x2F;uvm</a> <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34936729" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34936729</a>
bob1029about 2 years ago
This paper was career changing for me.<p>Chapter 9 is effectively the original concept for our business rules engine today. We use SQLite and C# UDFs as the actual foundation.<p>Using a proper relational model and SQL queries for all the things means that domain experts can directly contribute. It also makes it feasible to turn your domain experts into internal customers of your developers. For B2B products, this can be make or break.<p>Building an actual business around this kind of thing is very hazardous in my experience. Unless you are <i>completely</i> certain that you have the schema figured out, this promising foundation converts to quicksand.<p>Using higher normal forms is one way to reduce the consequence of screwing up domain modeling, but you still really have to know the relational guts of the business one way or another.<p>One design-time trick is to get all stakeholders to compose their ideal schema in something like excel and have them fill in some sample data. Showing them an example of one of these for a different domain is a powerful lesson in my experience.
feorenabout 2 years ago
My impression from reading this has always been &quot;Almost there! Keep going.&quot; If you keep pulling on these threads, I claim that you reach some inevitable conclusions. These ideas are a superset of all the other advice they give (and most other advice you hear, too):<p>1. Separate identity from data. These are <i>completely different things</i>. The number 7 is not mutable, even though my age is. Keep going! The phrase &quot;It was a dark and stormy night&quot; is not mutable, even though the opening line of your book is. Keep going! The statement &quot;there are seventeen cars parked on level 3 and four spots available&quot; is <i>not mutable</i>, even though the status of the parking garage is. Identity is immutable, and statements (data) are also immutable. Only the assignment of a statement to an identity is mutable.<p>2. Think about the operations that make sense on your data. Sure, you have &quot;map&quot;, &quot;filter&quot;, &quot;reduce&quot; that are mathematically pure. What about &quot;offset&quot; or &quot;rotate&quot;? Those have identity, they are associative, individually commutative (but not with each other!). Is there a distributive property that applies there? Okay, what about &quot;buy&quot; and &quot;sell&quot; operating on commodities? Are those mathematical operations? Do they have an identity element? Are they associative and commutative? &quot;Buy 2000 lbs of chicken&quot; -- is that equivalent to &quot;Buy 1 ton of chicken&quot;? What are its domain and range? Not &quot;chicken&quot;, nor even &quot;warehouses&quot; -- you&#x27;re not teleporting the chicken as soon as you commit that record. More like &quot;contract&quot;, or even &quot;contract request&quot;. What can you do with contracts? Is there an &quot;identity contract&quot;? &quot;Buy 0 chicken&quot;? Zero is a useful number! Is &quot;Buy 0 chicken&quot; the same as &quot;Buy 0 beef&quot;? Explore these questions. Find the math around your domain. Functional purity is all good, but it&#x27;s wasted if your domain is just &quot;Chicken : Meat : Commodity&quot;, &quot;Beef : Meat : Commodity&quot;, &quot;Pine : Lumber : Commodity&quot;. Wrong. Don&#x27;t think about nouns. Think about sensible pure operations in your domain. Successive abstraction layers should be <i>restricting</i> what&#x27;s possible to do with your data, by defining higher and higher-level operations. If your abstraction layers aren&#x27;t restricting what&#x27;s possible to do, they&#x27;re an inner platform and you don&#x27;t need them.<p>3. Don&#x27;t make big decisions upfront. That&#x27;s how you add accidental complexity. Don&#x27;t make any decisions you don&#x27;t need to, and prefer solutions that allow you to defer or lighten decisions. If you follow this thread, that means you&#x27;re using a relational model. They absolutely got that right (section 8). Otherwise you&#x27;re making decisions about ownership that you don&#x27;t need to be making. Domain Driven Design has it <i>wrong</i> here with aggregate roots. What&#x27;s an aggregate? Which concept goes inside of which aggregate? You do not need to make that decision. The authors of TFA get it right, and then wrong again, when they try to apply a relational model to shitty noun-based domain ideas like &quot;Give Employee objects a reference to their Department, vs. Give Department objects a set (or array) of references to their Employees vs. Both of the above&quot;. No. They&#x27;re not following their own advice. Can an employee exist without the concept of &quot;department&quot;? Yes. Can a department exist without the concept of employees? Probably. Therefore you <i>must</i> encode the relationship separately from both. Your hands are tied. If concept A can exist independently of concept B, you <i>must not</i> draw an arrow from A to B. The answer is not &quot;both&quot;, it&#x27;s &quot;neither&quot;. An employee&#x27;s assignment within a department is its own independent concept, that knows about &quot;employee&quot; and &quot;department&quot; both. And now it&#x27;s obvious you can give this concept its own tenure and add a new record whenever they change departments. You&#x27;ve found append-only event sourcing without needing to start there -- it came from first principles (in this case).<p>4. Operate on the broadest scope you can. Manipulate sets of things, not individual things. This is half of functional programming&#x27;s usefulness right here. This is supported by using a relational model. Operate on sequences of things, not individual things: there&#x27;s reactive programming. How else can you broaden the scope of what you&#x27;re operating on?<p>5. Don&#x27;t just do something, stand there! That is: don&#x27;t <i>do</i>, <i>plan</i>. Instead of immediately looping, just use &quot;map&quot;. Instead of immediately hitting the database, write a query plan. This doesn&#x27;t mean offload your thinking to GraphQL -- that&#x27;s just kicking the can down the road. See point #2. This is the other half of functional programming&#x27;s usefulness. Do only as much as you need to make (or append to) a plan, and then stop. Someone else will execute it later -- maybe! You don&#x27;t care.<p>There&#x27;s probably more, but there are more fundamental ideas than &quot;use functional programming&quot; or &quot;use event sourcing&quot; or &quot;use SOLID&quot; -- first principles that actually lead to almost all the other good advice you get. This paper kinda almost gets there a couple times, then walks back again. Keep going. Keep pulling on those threads. I suspect that the more you do this, the more your code will change for the better. But you have to be willing to keep going -- don&#x27;t half-ass it. If you only go part way and then stop, the benefits won&#x27;t be apparent yet.
vendiddyabout 2 years ago
There are a lot of comments saying that we can&#x27;t avoid mutable state and dismiss this paper entirely.<p>I find a practical interpretation of this paper is:<p>- Favor pure functions over impure functions<p>- Reduce mutable state and be deliberate about where those mutations have to happen<p>- Prefer derived state over keeping state in sync<p>There are always exceptions. Use your judgment on when this simplifies code and when it doesn&#x27;t.
coffeebeqnabout 2 years ago
Does anyone have this in audio form? I dont have a ergonomic way to read this even though it looks really on point