I have never used Kafka or had to maintain queues with more than a hundred jobs per hour. Most of the jobs queues I've had to maintain are doing simple things like sending emails or updating a third party API.<p>I've always resorted to simpler distributed redis based job queues such as kue [0], bull [1], celery [2] or resque [3].<p>I would like to learn more about distributed queues and architectures from first principles. Where is a good place to go?<p>[0] https://github.com/Automattic/kue
[1] https://github.com/OptimalBits/bull
[2] http://www.celeryproject.org/
[3] https://github.com/resque/resque
A good start is this article by Jay Kreps (Apache Kafka co-creator and Confluent co-founder) :<p>The Log: What every software engineer should know about real-time data's unifying abstraction : <a href="https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying" rel="nofollow">https://engineering.linkedin.com/distributed-systems/log-wha...</a>