We deploy distributed, multi-language, centrally Lisp/SBCL servers as well. A few specifics that I'd point out:<p>Many of SBCL's optimizations are fine grained selectable, using internal SB-* declarations. I know I was at least able to turn off all optimizations for debug/disasm clarity, while specifically enabling tail recursion so that our main loop wouldn't blow up the stack in that build configuration. These aren't in the main documentation; I asked in the #sbcl IRC channel on FreeNode.<p>You can directly set the size of the nursery with sb-ext:bytes-consed-between-gcs, as opposed to overprovisioning the heap to influence the nursery size. While we've run in the 8-24GB heap ranges depending on deployment, a minimum nursery size of 1GB seems to give us the best performance as well. We're looking at much larger heap sizes now, so who knows what will work best.<p>While we haven't hit heap exhaustion conditions during compilation, we did hit multi-minute compilation lags for large macros (18,000 LoC from a first-level expansion). That was a reported performance bug in SBCL and has been fixed a while back. Since the Debian upstream for SBCL lags the official releases quite a bit, it's always a manual job to fetch the latest versions, but quite worth it.<p>Great read, and really familiar. :-)
Common Lisp's macros and grammar go together like bread and butter. A grammar module in the app I built [1] uses macros to generate huge amounts of repetitive code.<p>[1] <a href="https://github.com/tshatrov/ichiran/blob/master/dict-grammar.lisp" rel="nofollow">https://github.com/tshatrov/ichiran/blob/master/dict-grammar...</a><p>I wonder if they're still hiring Lispers. I once passed on the opportunity to work in their Kiev office, but I might give it a shot again.
Great article, and good reminder on using trace. Every time I rediscover trace, I can't remember how I ever forgot to use it in the first place for most of my problems.<p>I used CL in a production environment a while back for a threaded queue worker and nowadays as the app server for my turtl project, and I still have yet to run into problems. It seems like you guys managed to push the boundaries and find workable solutions, which is really great.<p>Thanks for the writeup!
Slightly off-topic, but does anybody know of a kind of "LISP Challange" set? I recently started the Matasano challenges[0] and I found them really well-suited to my style of learning (learning by doing and expanding by reading relevant material, enabled by my own internal motivation). Is there anything like that that has a relatively small set of condensed yet rich challenges that demonstrate key elements from LISPy functional programming? I read some of SICP but reading long form really puts a damper on my motivation/excitement. Also there were a lot of exercises (with a lot of overlap in concepts) so I didn't know what to do and what not to do, since I wasn't about to do every single one. Any pointers would be much appreciated!<p>[0] <a href="http://cryptopals.com" rel="nofollow">http://cryptopals.com</a>
One of the things I would have liked to see on the article is how do they handle the deployment itself. Do they build an executable with build app? To they used sb-daemon? An home-grown solution using sb-posix:fork?
Apparently they use "JVM languages", JavaScript, Python, Go, Lisp and Erlang in production.<p>I may be in the minority, but that would drive me mad. I assume they're not routinely jumping between those stacks multiple times a day, but even so is there really that much benefit that it's worth keeping track of how to do things in that many different environments?
Aw, now they have disclosed their secret weapon! [1]<p>[<a href="http://www.paulgraham.com/avg.html" rel="nofollow">http://www.paulgraham.com/avg.html</a>]
Good lord, I would go insane if I ran into a bug like this:<p>"We've built an esoteric application (even by Lisp standards), and in the process have hit some limits of our platform. One unexpected thing was heap exhaustion during compilation. We rely heavily on macros, and some of the largest ones expand into thousands of lines of low-level code. It turned out that SBCL compiler implements a lot of optimizations that allow us to enjoy quite fast generated code, but some of which require exponential time and memory resources. "
I love it :0) I tried grammarly, and typed in a remembered poem. It informed me that it had detected significant plagiarism.<p>Edit: It's still not advice I would pay for, though.