TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

A simple core.async job system in Clojure

127 点作者 janetacarr12 个月前

5 条评论

lkrubner12 个月前
I appreciate this emphasis on simplicity and I recommend it to others. I&#x27;ve seen trends in the other direction which I think are dangerous. I notice that as some companies adopted a microservices approach there was a tendency to allow in more technologies than necessary.<p>&quot;Premature polyglot programming&quot; is a disease that afflicts certain startups. While any sufficiently big company will be polyglot, a small startup needs to stay focused on a limited tech stack, for as long as possible. After all, each new technology requires someone on staff who has the skill for that technology, and when your startup is small, your talent pool will also be small, and so it becomes common that you only have 1 person on the team who knows how to run some particular technology (Kafka, RabbitMQ, Redshift, DynamoDB, MongoDB, Tornado, etc).<p>So you end up with a lot of single point of failures, where the &quot;single point of failure&quot; is the single engineer who knows how to run the technology that you&#x27;ve made critical to the company. Be wary. Avoid this if possible.<p>I notice, especially, as the tech industry developed better tools for managing complex devops situations (Docker, Kubernetes, Terraform) there was a tendency from some engineers to think &quot;Nowadays it is easy to run 10 different technologies, therefore we should run 10 different technologies.&quot; Be wary of this.<p>Janet Carr&#x27;s emphasis on simplicity is something we should all imitate.
评论 #40474747 未加载
评论 #40474590 未加载
评论 #40475405 未加载
评论 #40474964 未加载
评论 #40475723 未加载
tombert12 个月前
core.async was actually the &quot;killer app&quot; that sold Clojure to me. I&#x27;ve always liked Go&#x27;s CSP-style concurrency, but I wanted a proper functional language to go with it. It turns out that Channels&#x2F;Queues are just an insanely good abstraction for getting threads to talk to each other. The closest general-purpose library I&#x27;ve found that helps bring that to the rest of the world has been ZeroMQ, which is great but not as easy and nice to use as core.async.<p>That said, I really don&#x27;t think RabbitMQ is so bad. The default docker configuration for it is fine for most cases you&#x27;re likely to come across for all but the biggest applications, and I do think that having the ability to have jobs restart&#x2F;requeued when a worker crashes out of the box is worth the tradeoffs for slightly increased complexity. I usually just use a single docker compose and glue everything together in one big ol&#x27; YAML.<p>Still, there&#x27;s obvious value in avoiding dependencies, so it of course depends on the size and scope of your project. If you think you&#x27;re going to end up distributing this over 100 nodes, something like RabbitMQ or ActiveMQ might be worth playing with, but if you&#x27;re just doing relatively small (at you appear to be in this project), it&#x27;s probably the correct choice to mimic whatever behavior you need with core.async.
评论 #40476443 未加载
jwr12 个月前
Yes! core.async is a great tool and I&#x27;ve used it to solve a number of problems effectively. I&#x27;m very happy with what I get: reliable, predictable systems.<p>A real &quot;whoa&quot; moment comes when you realize you can also use core.async in ClojureScript :-)
samsquire12 个月前
Thank you for this post Janet.<p>I think my technical perspectives have moved in a similar direction: keep things extremely simple with minimum moving parts.<p>I maintained (automated upgrades) a RabbitMQ cluster and while it is powerful software it is operationally expensive. For a side project you probably just batch process in a cron.<p>If I were to take the approach in this blog post I would want everyone on the team to be extremely familiar with the model of task running: stuck jobs, timeouts, duplicate jobs, client disconnects and retries, stuck &quot;poison&quot; jobs seem like issues you might face.
评论 #40474494 未加载
mark_l_watson12 个月前
I appreciate Janet’s aversion to adding RabbitMQ to the system. I had to use RabbitMQ with Common Lisp a year ago and in was a pain.<p>It looks like Clojure has better RabbitMQ client options than Common Lisp, but still, very cool to build something on core.async and keep things cleaner and simpler.