The Programming Language Conundrum

&gt; A lot has been written about &quot;hyperproductive teams&quot;. ... there’s one thing in common between all the stories I’ve heard and the successes experienced first hand: small teams<p>Personally, I&#x27;d go one step further. It&#x27;s small teams that were together from the start. Going through the initial setup of a project makes all the difference. When people come later, they lack the intuition that comes with being around when all the structural decisions were made. Early people make tradeoffs and bake good and bad practices into the pipeline, later people are told not to touch certain things because it&#x27;s more complicated than it looks or will break something else that doesn&#x27;t seem explicitly connected.<p>I&#x27;ve been on both sides of this fence. Having started a company and sold it - my partner and I were hyper productive, but while the small team that inherited the code is extremely capable, it took years longer to feel comfortable making big changes than it took to write.<p>Joining other people&#x27;s projects and large codebases, on the other hand, has always taken a long time for me to be productive in. Typically it takes a year at a new company before I&#x27;m moderately useful.<p>Personally I don&#x27;t think this has a ton to do with programming languages, even though I feel much more productive in python and javascript than C++ myself, even though I&#x27;ve been using C++ longer. I still suspect familiarity with the early decisions is a greater factor in productivity than language.

I have the exact opposite experience going from working primarily with Java to Clojure. I worked with the former for nearly a decade, and I haven&#x27;t contributed to a single library or project I used in that time. Every time I&#x27;d see a bug or a feature I&#x27;d like to add, I&#x27;d check the source see thousands of classes each with its own semantics and behaviors, and give up. The amount of effort needed to familiarize yourself with a non-trivial Java codebase is herculean.<p>Meanwhile, I&#x27;ve contributed to many Clojure libraries, and lots of people contributed to the ones I maintain. I can open a Clojure project and figure out what&#x27;s going on much easier than with a Java one. The code tends to be much more direct. It&#x27;s primarily written using pure functions that you can reason about independently. It&#x27;s focused on the data, so I don&#x27;t have to learn hidden behaviors. And I have a live environment with the REPL where I can run the code and see what it does immediately.<p>My team has been working with Clojure for nearly a decade now. We have many applications in production that have been developed and maintained for many years. We&#x27;ve hired many new devs during that time, and we&#x27;ve never had problems onboarding.<p>A language being expressive and flexible does not preclude it from scaling to large projects. And when the language is not expressive, you end up with huge amounts of incidental code because you&#x27;re forced to map your problem domain to the language as opposed to the other way around.<p>I think that community practices for using the language are far more important. Clojure community started discouraging heavy use of macros early on, and a small set of common patterns tends to be used to solve problems, and codebases across projects tend to have a similar style to them.

Even if there is a scaling limit, you can still use SmallTalk or a Lisp for small projects (and the scope of small projects in those languages is larger than that of, say, small Java projects) and benefit. I imagine plenty of unglamorous software projects are small enough in scope.<p>I am a mathematician, not a programmer, but even I have a Lisp story like this. I&#x27;m working with a computer science professor at my university who has in the past worked as a programmer both for private companies and for our government. We&#x27;re studying some combinatorial game played on graphs and wanted experimental data of which player has a winning strategy on thousands of small graphs.<p>To feel more certain of the computer results we decided to each write our own implementation independently without looking at each other&#x27;s code. He wrote his in C and I wrote mine in Common Lisp. You can imagine the punchline since it is similar to all of the Lisp stories out there: my program is about a tenth of the number of lines of code, took much less time to write and runs about twice as fast!

Odd languages attract above average programmers, if nothing else than because they are engaged. Small teams are more productive than large teams, because there are almost no communication problems. I wonder how much is due to this?

&gt; Create a language to solve your problem;<p>&gt; Solve your problem.<p>When I worked on a complex framework in a small team, we used Objective-C. In the early days of the project, we created some powerful utility methods like map and reduce which we used extensively in the early stages. These methods let us do things like traverse complex structures very compactly. Way better than the more plodding while-loop (perhaps with a stack) way of doing it. But, I found myself moving away from these later in the project in favour of more verbose code because it took me too long to understand the map&#x2F;reduce code if I needed to revisit it. The longer code was, um, longer, but it was much easier to understand what I (or my colleagues) had done when I came back to this code. In my view, a bunch of very smart programmers can indeed be &quot;hyperproductive&quot; using a language like Smalltalk, but this hyper-productivity may not survive the longer term life of the project.

I&#x27;m skeptical for the simple reason that this post never calls out any specific feature of Lisp or Smalltalk which could cause this problem, and other languages which apparently don&#x27;t suffer from these problems have most of the same features. It&#x27;s almost as if unpopularity is the root cause.<p>Guy Steele famously said that with Java they &quot;managed to drag a lot of [C++ programmers] about halfway to Lisp&quot;. I don&#x27;t hear many complaints about Java being too &#x27;organic&#x27; or difficult to document or that it only works on 2-pizza teams.<p>If you think that sufficiently powerful languages only work on small teams of dedicated people, why draw the line at Lisp&#x2F;Smalltalk? Why aren&#x27;t the Ada people going crazy about how loose and dynamic Java is, much less Python and Ruby -- all of which apparently &quot;breeze through the 100, 200, 500 developer marks without much trouble&quot;?<p>I remember reading about how Codd&#x27;s relational databases struggled with acceptance because it was believed at the time that &#x27;tables&#x27; were too complex for the average programmer, yet today that idea is laughable. People tend to rise to the level of expectation.

The fact that dozens of companies have been built around millions of lines of Common Lisp code at all levels of abstraction solving some of the most complicated problems is a testament to the scalability of Lisp.<p>What has changed is that most software shops aren’t solving technical problems anymore. Most places are engaged in the business of plumbing—connect this database (whose problems are solved) to this dashboard (whose problems are solved)—but many of us don’t want to admit it. It’s not glamorous and Silicon Valley prefers to keep its death grip on titles like “Senior Solutions Architect.” Plumbing doesn’t necessarily need expressive power of a programming language, but I contend that problem-solving greatly benefits from it.

I would say there are several languages that straddle the line.<p>AFAIK there exist 50+ people teams that use i.e. ocaml (Jane street, Facebook), haskell (gallois, Facebook, Google), erlang (whatsapp, pivotal)<p>I would say that beyond i.e. 10 developers you suddenly need to have onboarding solved. You need to be able to teach your new hires. You need to have community beyond your workgroup to ask questions. You need decent library support.<p>I would say it is beyond 50 where your leads start to worry you didn&#x27;t `go with IBM` so to speak. You suddenly can&#x27;t just hire an expert. And you can&#x27;t coast on the fact your programmers find the language familliar.<p>Like, I joined 4 people team working in clojure. It was fine. You mostly could see &quot;oh, this is where the macro magic starts, and here I need to grok protocols first and here I need to grok transducers first&quot;. Then my manager got nervous that he couldn&#x27;t hire new people fast enough so we had to do a python rewrite :-&#x2F;<p>Different team, different project, 60+ people, python. You&#x27;d think, it is just python. But you never know when there is a metaclass hiding behind an inheritance chain. There are parts, where we might have single person understand the code. Sometimes they left year ago.

I don&#x27;t think the issue is that the language weaves with the mind (or the reverse). Instead I think there&#x27;s several issues at play, at least these, and probably others: a) mindshare, b) dynamic type systems suck.<p>(b) alone may seem insufficient, since TFA mentions Python as a language that scales, but I&#x27;ve never seen large Python systems -- lots of small Python programs and libraries, yes, but monolythic multi-mloc Python applications? no. When you get to that size, dynamic typing issues become time sinks.<p>As to mindshare, if the number of developers who know Lisp is small, you&#x27;re not going to get a lot of them. Mind you, you should be able to train them -- once you speak N&gt;3 programming languages, getting to N+1 is not that difficult. But it&#x27;s a lot easier to hire C++&#x2F;Java&#x2F;whatever programmers. This is a mindshare effect.<p>Also, if 1.5 FTEs allows you to build a successful company, then to scale development to 100 FTEs you have to rewrite, well, that&#x27;s still a success. Many would like to have that problem. And nowadays we&#x27;re getting good at incremental rewrites as an industry -- witness the Rust effort.<p>I think the future lies with Rust. Rust is a very high-level programming language, but it still lets you get down to the same low level as C. If you sold Rust as a Haskell w&#x2F; linear types, no&#x2F;minimal monads, no I&#x2F;O monad, and a C-ish syntax -basically, what it is-, I think TFA might run away saying &quot;oh no, another Smalltalk that won&#x27;t scale&quot;.<p>At the very least I&#x27;d like to think and hear a lot more about the issue -- whether and why some programming languages do or do not scale teams -- before accepting TFA&#x27;s conclusion.

&quot;These languages are very powerful, but they don’t scale. So even when you get ten times the productivity, the mostly tribal complexity will not scale beyond your “pizza-sized team” and therefore you now have a hard upper limit on your total software development capacity.&quot;<p>Ultimately, the problem boils down to the languages giving too much freedom. More conventional languages like Java can scale to a massive degree because they standardize so many things, which means that you can now share components which amplify your work. Lisp et al are cool toys and small team tools, but you&#x27;re not going to see any large scale db libraries and parsers and concurrency libraries and web servers and cachers and HA systems that can interoperate with each other without a ton of shoehorning.<p>When every foreign chunk of code requires a ton of ramp up time just to figure out what it&#x27;s doing and how you could even integrate it, or requires changing your IDE and workflow in ways incompatible with what you have already, you lose more than you gain.<p>Worse is better. Worse gives you interoperable, reusable, amplifiable components, in a common framework everyone understands.

I am yet to see a team of 100 successfully working on the same problem. I&#x27;ve seen large codebases that solve lots of different problems with 100&#x27;s of people working on it at the same time, but this isn&#x27;t the same thing.<p>My point is, if you are on the second situation, you can very likely split your people into different teams, facing different trade-offs and with a formal (but not static) interface between their code. So if you can get a 10 people team with the productivity of a 100 people team, but unable to scale further, well, you will face the same problem on the 100 people team, and can apply the same kind of solution to both. The only difference is that you are saving 90 people.

I think there&#x27;s a really important aspect to this that the software industry needs to confront in order to mature.<p>To <i>some extent</i> - not completely, but at least partially, power and unmaintainability are two sides of the same coin. That is, it is exactly the same properties that allow your amazing macro to boil 15 dimensions of a problem down into a single beautiful one line piece of code that also make it an unacceptable liability that should never be allowed into a mature codebase [1]. Because if the macro didn&#x27;t exhibit that amazing sublte complex behavior then it wouldn&#x27;t be powerful. But exactly by doing that, it is now something that nobody new can understand.<p>There is also a fraction of the software &quot;power&quot; equation for which it is a win-win: more power equals less overall non-essential complexity in the system and the burden of propagating the associated knowledge is lower than what is saved.<p>However distinguishing the above two things is completely non-trivial and if you can&#x27;t actually do that well you need to fall back on the side of avoiding overly powerful behaviors if you want to have an actual long term, large scale, maintainable outcome.<p>[1] Obviously this is an overreaching statement. But to a first approximation this is true and the exceptions by definition have to be rare.

The proposition that &quot;custom development tooling doesn&#x27;t scale&quot; is very much news to Microsoft, Google, Facebook, LinkedIn, etc.<p>All of them have custom programmable configuration stacks, meta languages for their environments, and custom application harness instructions.<p>So why don&#x27;t those count as counterexamples? It seems like a just-so story.

&gt; These languages are very powerful, but they don’t scale.<p>The problem, it seems to me is that software developers often think: &quot;what programming language do we really like &amp;&amp; can continue to use for $long_time over $scaling_of_business&quot;.<p>When this is the case, business management expectation problems emerge regarding costs, maintenance and timing when the business scales by 10 or 100x. Early phase teams are lambasted for making poor decisions - and &quot;adult supervision&quot; ( i hate that term ) is hired to &quot;fix things&quot;.<p>Thing is, the early &quot;poor decisions&quot; may not have been poor decisions at all relative to the business needs. However, since costs and retooling issues weren&#x27;t discussed and planned for - the early team look like a bunch of fools&#x2F;amateurs.<p>As software developers, we need to understand and communicate that the tools can we use to accomplish the needs for the business at one stage are different than those of a later high-scale stage. its OK, completely natural, and a requirement if we want to keep our jobs and grow with the business.<p>analogy: If a specialty carpenter is designing and building high end furniture with a set of well made brand-name niche tools and techniques they wouldn&#x27;t expect that those same tools would be used if sales take off and 10x or 100x the number of units need to be produced. It would be understood that retooling would be required, people with expertise in those tools be hire and that there would be new costs involved. Also, the carpenter who starts the business with small scale tools isn&#x27;t lambasted for being an idiot for picking those tools in the early phases. It would be quite odd to suggest that that person have picked tools and hired people for 100x production when they were selling single digit units a year.

We are traveling down this road. We&#x27;ve implemented Clasp (<a href="https:&#x2F;&#x2F;github.com&#x2F;clasp-developers&#x2F;clasp.git" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;clasp-developers&#x2F;clasp.git</a>) - an implementation of Common Lisp that interoperates with C++ and uses llvm as the backend. On top of this, we built Cando (<a href="https:&#x2F;&#x2F;github.com&#x2F;drmeister&#x2F;cando.git" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;drmeister&#x2F;cando.git</a>) - a computational chemistry programming environment for designing large molecules. The end of our story hasn&#x27;t been written yet.<p>Clasp could be used for a lot of other things.

The poster makes a point about conservatism in large companies, but doesn&#x27;t explore this further. Its actually a more complicated matter than just technical ignorance in managers. The real problem is that programming languages are seen as a tactical matter for project managers (&quot;just use the best one for the job&quot;), whereas they should be considered strategic.<p>* At the start of a project the PM is given a budget and a timescale which were probably driven by the question &quot;whats the earliest you can&#x27;t prove you won&#x27;t be done by?&quot; There will be no slack for developers to spend 6 months coming up to speed on a new programming language and paradigm.<p>* Nor can the PM afford the career risk of trying something new. If you do it the standard way and it doesn&#x27;t work, the decision to do it the standard way was obviously not responsible and the search for blame can be pushed elsewhere. But if you pick something novel and it doesn&#x27;t work, the blame for any failure will always focus on the novel element. So there is a strong institutional pressure to keep doing things the same way to avoid the blame for failure.<p>* Once a piece of software is finished it has to be maintained. Most software expenditure actually goes on this phase. If you use a new programming language then you are going to have to keep a bunch of programmers around who speak that language just for that project, even if there are no changes needed this year. That is expensive, so this is another disincentive for trying a new language.<p>* Every technical manager has seen enthusiastic young programmers bearing some new silver bullet (see Fred Brooks) which, the youngster says, will solve all the problems at a stroke. After you have seen half a dozen of these they all start to look the same. There is also concern about &quot;CV engineering&quot;: getting the company to take on the risk and cost of training employees in trendy technologies which won&#x27;t provide real benefit but which will make those employees more likely to find a higher paid job somewhere else.<p>If programming languages were considered as a strategic matter then a large company could make a decision to invest in tooling, training and increased project risk in order to adopt a new language which would make them more productive in the long term. But until then change will be forced on the industry by small companies who beat the averages.

I enjoyed the build up throughout the article to his conclusion. While I can&#x27;t prove him wrong, a lot of very smart people came up with languages like Java and C# to solve these problems (ramp up, maintainability, retention, and more) as efficiently as possible. While I salute the author in his quest to be proven wrong, objectively, everything is a trade-off. I don&#x27;t believe in better.

The real problem is that there&#x27;s a missing step:<p><pre><code> 1. Create a language to solve your problem 2. Solve your problem 3. Document the language </code></pre> Presuming that the original creators made a reasonable language for the problem at hand, who wouldn&#x27;t want to use a powerful, well designed language with good fitness of purpose?<p>Now imagine that you were brought onto a team using a widespread (not homegrown) language you didn&#x27;t know that was used for the company&#x27;s main application, but you have no access to documentation.

Are teams using these languages small because the language doesn&#x27;t scale well or because you don&#x27;t actually need more developers?

&gt; My theory, completely made up and untested, is that these powerful Lisp&#x2F;Smalltalk systems weave code and brains together in a way that amplifies both. It’s very organic, very hard to document, and thus it becomes very hard to absorb new brains into the system. &lt;<p>Here&#x27;s a simpler and less mystical theory. These programming systems are fundamentally image-based, and much of their leverage comes from the coder&#x27;s ability to leverage all aspects of the meta-system. However, the more the core image is evolved the easier it is for a team to get out of sync. This is the ultimate namespace clash.<p>A slew of other highly-productive languages share this feature, e.g., APL and Forth.

&gt; 1. Create a language to solve your problem;<p>&gt; 2. Solve your problem.<p>But don&#x27;t we do this in every language, at least if we&#x27;re doing a good job?<p>I remember the final &quot;exam&quot; of my high school CS class. It was mostly an in-class programming exercise in Pascal.<p>I was the only one to finish in less than the allotted time. The first one to finish after me did so after the <i>second</i> extension period.<p>I have to admit while I was obviously inordinately pleased with myself, I was also baffled, because the other people in the class were certainly just as smart as I was.<p>So what was the difference?<p>The problem we were given was to draw a bunch of boxes with character graphics with contents and some overlap. Not really hard. Everybody else coded up each box individually with straight code. One after the other. Each time having the handle all the edge cases (literally).<p>I was the only one who first created a procedure (or set of procedures) for box drawing, and then used those procedures to draw the boxes. And yes, that is very much a simple &quot;language&quot; for solving the problem. Of course the language set some pretty tight limits on how your &quot;language&quot; could look, which is a bane for expression and a boon for understanding.<p>So I did &quot;define the language, solve the problem in that language&quot;. In Pascal. Classic bottom-up programming.<p>So I dispute that there is a <i>fundamental</i> difference, though there definitely is a significant difference in degree. The difference with high-power languages is that they allow us to apply this much more broadly, whereas with other languages you often run into technical limitations that prevent you from really defining the right language.<p>With there not being a fundamental difference, I also don&#x27;t buy that we have a fundamental, unbridgeable divide&#x2F;chasm&#x2F;whatever, with these high-power languages with their problems on one side and the &quot;blub&quot; languages with their other problems on the other. Rather, let&#x27;s explore that regions of the language space and see if we can&#x27;t narrow the gap.

The idea is pretty simple: concise languages ( like python, smalltalk etc.) work well when you can get away with being concise. Verbose languages ( like the C family ) work well when you can&#x27;t. It only takes a couple of tries writing a project that favors verbosity in a concise language before this becomes apparent.

&gt; So why aren’t we all using either of these two languages?<p>because $bigcorp&#x27;s dreams with languages such as Java was not to be incredibly productive, but to be able to fire a whole team, hire replacements, and have them be more-or-less productive on day 1. Can&#x27;t do that if every project lives in its own DSL.

I put a reaction on the site of the writer of the blog, but it looks like he removed it. So I&#x27;ll repeat here. The text misses structure, clearness and the writer makes the same assumpion as Paul Graham, namely that a programmer thinks in the programming language he uses. To start with the last: &quot;to write a good program, we need to think above the code level. Programmers spend a lot of time thinking about how to code, and many coding methods have been proposed: test-driven development, agile programming, and so on. But if the only sorting algorithm a programmer knows is bubble sort, no such method will produce code that sorts in O(n log n) time. Nor will it turn an overly complex conception of how a program should work into simple, easy to maintain code. We need to understand our programming task at a higher level before we start writing code. [..] There are two things I specify about programs: what they do and how they do it. Often, the hard part of writing a piece of code is figuring out what it should do. Once we understand that, coding is easy.&quot; (Leslie Lamport, <a href="https:&#x2F;&#x2F;archive.li&#x2F;2zZKy" rel="nofollow">https:&#x2F;&#x2F;archive.li&#x2F;2zZKy</a>). Sentences in de Groot&#x27;s blog like &quot;these powerful Lisp&#x2F;Smalltalk systems weave code and brains together in a way that amplifies both.&quot; are incomprehensible. The question at the end of the blog looks rhetorical to me &quot;I think I’ll end while hoping that someone out there on planet Earth reads this and proves me wrong :)&quot;. It is not clear what should be proved wrong, I suspect the writer is satisfied with his musings.

&gt; A lot has been written about &quot;hyperproductive teams&quot;. ... there’s one thing in common between all the stories I’ve heard and the successes experienced first hand: small teams<p>Funny because I don&#x27;t think its small teams or a particular programming language that leads to great productivity. You don&#x27;t have to look very far or far back to see teams, of various sizes, pulling off some amazing stuff with assembler and C. The key to productivity is a lot simpler, and independent of software development. Its a sense of responsibility, interest, and dedication.<p>In a small team that is using small talk the responsibility might come from being on a small team with a lot of lifting to do, the interest might come from having fun using a language that you like (regardless of the language), and the dedication might come from the promise of getting in early on a promising venture - hence the small team. All of these things add up to hyper-production.<p>If you can establish those three things, somehow, with a large team I don&#x27;t see why hyper-production can&#x27;t be achieved. Unfortunately modern project management for large teams seems to rely heavily on techniques that rob developers of all three qualities mentioned above in the name of so called &quot;agile&quot; and in pursuit of some fictitious software assembly line.

I agree with the OP. This has been called the Lisp Curse elsewhere. Over time, your codebase becomes so high-level, full of advanced, domain-centric abstractions that basically alienates everyone else from effectively working on the project.<p>If you have a small, stable team that such issue might be relatively easy to overcome. But with bigger teams, with higher turnover, than it might totally cancel out any technical advantage gained from using such non-mainstream programming languages.

I don&#x27;t buy that the &quot;privilege&quot; of<p>&gt; 1. Create a language to solve your problem; &gt; 2. Solve your problem.<p>belongs mainly to Lisp and Smalltalk. Each time you define a method or a class (C#, Java, C++), you&#x27;re extending and defining your &quot;language&quot;.<p>I&#x27;m the leader of a 2.5-developer startup, we use C++, Java and C# for different purposes, and I have not once thought &quot;shit, we&#x27;d be so much more productive if we used XXX instead of C# or Java&quot; (C++ being the exception). Because the real problems are around the business domain. Understanding customer needs, wrapping our heads around Azure AD, etc, etc, etc.<p>The programming language has ZILCH to do with our challenges and I do not feel hampered by PL during refactoring either -- thanks to static types.<p>I have some regrets, and these go against frameworks we decided to use (EFCore, i.e., ORM, being the biggest regret of mine).<p>Programming language? Almost not a factor. I&#x27;m happy with all of the 3 we use, though I&#x27;m least productive in C++ due to the amount of ceremony needed -- header&#x2F;implementation files, linkage, fixing preprocessor mess by variations of PIMPL (looking at you windows.h), slow compilation, etc, etc, etc.<p>If anything is indispensable for our productivity, it&#x27;s a good ide and becoming comfortable with its capabilities.

I think what really makes those languages&#x2F;teams productive is iteration time being low. Part of that is a natural consequence of small teams and lack of legacy code but certain languages definitely encourage it more than others. If you have fast iteration times you can really quickly figure out a system by experimenting, or work through bad ideas on the way to good ones.<p>For whatever reason software developers always seem to undervalue fast iteration times, and the less productive languages always seem to make iterating quickly harder, whether it’s C++s ghastly compilation times and lack of an ABI making it hard to integrate libraries, or Java’s interdependent class based nature causing excessive mocking and&#x2F;or lots of generally unrelated state being required to have a working object to test with. In contrast smalltalk lets you just highlight a chunk of code and run it, and the lack of separation between runtime and ide generally means you have enough of the system live for things to work. I think where smalltalk doesn’t scale is it can’t easily take advantage of things invented outside of it because it’s so integrated and practically its own OS

I’m not sure I buy this: I recently had to write a bit of ruby (an omniauth strategy for my company’s OAuth2 server) and the code I encountered there (as well as in the various rails projects I’ve worked on) may as well have been written in macro-heavy lisp: in fact, lisp or smalltalk would be preferable because they would have good source navigation tools for the macros, such as a stepping macro-expander.

&gt; I can recite many of such stories; of people keeping up with their ten person team by coding whatever the team did during the day in 2-3 hours at night using Squeak Smalltalk; about an old Lisp hacker working in a research setup with mostly Java colleagues and always estimating to need 10% of the time (and then actually making the estimate). So why aren’t we all using either of these two languages?<p>Maybe it isn&#x27;t about the language, primarily, but about the team? Maybe those super productive teams would have been successful with any language that gave them enough freedom?<p>That would also explain why you can&#x27;t switch your 20 Java developers to Smalltalk, and have them perform miracles of software engineering.

Maybe this is part of the benefit of the whole micro-service trend? I&#x27;ve largely dismissed the idea until now as impractical in many cases, but this seems like a case where a team of between three and six developers could work in something like lisp (that may not scale well), expose an api, and communicate with other parts.<p>It seems like we really need cross-language oop for this reason. MS COM might be an example. I was reading something I clicked on yesterday when doing some reading about reference counting, and someone commented that things like MS Office (COM based) needed such things: something could be initialized in one language, passed to another, and deinitialized in a third.

That&#x27;s a very good insight here on large&#x2F;small teams.<p>Although I typically hate discussions of &quot;programming languages are like human languages&quot; I think it&#x27;s the key here. All the team members need to share <i>and be comfortable in</i> not just a spoken language, but the teams local jargon, and the programming language in use (including your team-specific conventions, which form a kind of jargon too). So teams in these esoteric languages (and I say this as a Lisp hacker myself) tend to be by definition small, have a lot of cultural overlap, and can be hyper-effective. And small teams <i>can be</i> hyper productive&#x2F;effective because of the lack of coordination overhead and because it&#x27;s easier to have a high 1st derivative from a small starting point (in other words: no large team can beat a small team in efficacy&#x2F;person, but small teams can&#x27;t do everything).<p>That only answers half the problem; the other is: well if small team X can beat small team Y in significant part because of the choice of tools, why don&#x27;t bigger teams use the same tools?<p>There&#x27;s no easy answer. One is power: Lisp and C++ have too many degrees of freedom so it&#x27;s easier to get in to trouble with languages with fewer &quot;guard rails&quot;. This was an explicit design criterion for Go BTW: new grads are coming here and screwing up, so let&#x27;s make it hard for them to make the most common&#x2F;dangerous mistakes.<p>Second in the case of Lisp: it wasn&#x27;t considered a deployment language as many of the things that made is powerful (e.g. GC, lambda, dispatch, dynamic polymorphism) were considered too expensive at a time when C and smaller machines were taking off. Now many of those features are considered mainstream in other languages, while the Lisp community itself therefore attracted people who were more power users.<p>C++ had a driver (Windows) and an impetus (like Go&#x27;s: C gets you into a lot of trouble) but even then people looked for easier &#x2F; safer languages (e.g. Java, an explicit reaction to ++)

One downside of Lisp and Smalltalk for large projects may be that there is no official mechanism for declaring types and interfaces. Even if you can adopt your own conventions for implying the types they are just your convention and other programmers might have difficulty grokking and trusting them.<p>Hyper-productive languages may be hyper-productive just because they use dynamic typing and static types may be better for creating a large piece of software by many programmers evolving over years hopefully decades.

I think the small team aspect is far more important than anything else. With two developers in a startup, you can split up key components and have quick informal talks concerning design choices. In a real company, huge amounts of time are wasted on all-staff meetings, team meetings, 1:1 meetings, management fads, mandatory compliance trainings that come up quarterly and take hours, and the sheer human problem of coordinating that many people. Big companies have huge waste stemming from many different areas. For example, at a large company they have people in charge of requirements that have no understanding of the domain or how to write software. It&#x27;s like the telephone game you get as a kid where the message completely changes by the time it gets around the circle. Of course you could have your devs do that job, but most companies simply aren&#x27;t that logical. I bet Paul Graham and another competent developer could use a language that is not considered to be very productive (Ex: C++) and still swim circles past a large IBM team that is being held down by it&#x27;s own weight.

The root issue is extensibility. The same sort of arguments come up with the simplest extensible language: Forth. There are whole categories of embedded applications where Forth can cut the development time down to a fraction of what it would take to do it in C. Once that programmer leaves it takes a long time for someone else to come up to speed to do the sort of simple modifications that such applications tend to require.<p>... and where do you get a programmer that can do anything effective with an extensible language anyway? Such people tend to be very good programmers in general and are rare and expensive.

Could it be that these languages haven&#x27;t evolved all of the tooling and ecosystems other languages have? You don&#x27;t spend time setting up stuff. With web dev, you constantly spend time switching between various tools to just even build the damn thing&#x2F;app.<p>None of it is <i>needed</i>, per se - for years everybody made web sites without it - but we&#x27;ve all convinced each other that it&#x27;s just what you do.<p>Maybe Smalltalk doesn&#x27;t have these conventions (yet)?

&gt; It’s very organic, very hard to document, and thus it becomes very hard to absorb new brains into the system.<p>I think they way out is [something like] Alan Kay&#x27;s &quot;active documents&quot;. If you really need large teams (rather than communicating constellations of small teams, say) then your software has to be structured to be communicable. (Beyond Knuth&#x27;s &quot;literate programming&quot; even, I think.)

Sounds like language oriented programming: <a href="https:&#x2F;&#x2F;resources.jetbrains.com&#x2F;storage&#x2F;products&#x2F;mps&#x2F;docs&#x2F;Language_Oriented_Programming.pdf" rel="nofollow">https:&#x2F;&#x2F;resources.jetbrains.com&#x2F;storage&#x2F;products&#x2F;mps&#x2F;docs&#x2F;La...</a>

&gt; Create a language to solve your problem;<p>&gt; Solve your problem.<p>But a typical problem consists of multiple sub-problems. Are you going to create a new language for each of them?

I think it really comes down to the availability of training material.<p>Look at the amount of material for Python compared to say SmallTalk

you’re given a $10M budget to solve a problem so you write it in Lisp and keep all the money.<p>Break this absurdity in any number of ways and see. Tech, business, people etc.

There is more than one way to do it. Or, only one: Perl.

&quot;(and do not forget: they are that powerful)&quot;<p>Are they? Because small teams are naturally much more productive, I thought that was going to be the &#x27;aha&#x27; of the article.