TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Three fundamental tricks for developers writing distributed systems

55 pointsby pedrobeloover 12 years ago

6 comments

ameyamkover 12 years ago
Many times the idea behind distributed systems is to avoid single point of failure. by using Database for communication, you are essentially creating another single point of failure in the form of database (unless database is running on some highly reliable elaborate master-master setup). However you can use systems like zookeeper to get similar functionality and to facilitate communication.
i0exceptionover 12 years ago
Isn't this the same as a message queue? Why would you want to rewrite this using a database? Also point/trick 2 seems unnecessary if you are using 3 (idempotent jobs). By queuing job ids you now have a consistency dependency between your message queue and database.
评论 #4868639 未加载
评论 #4868678 未加载
fusiongyroover 12 years ago
The one thing that concerns me about using the database as a queue is that MVCC doesn't really lend itself to writing threading primitives like locks. I'm curious how one would go about writing a queue in an MVCC architecture--off the top of my head, I guess you could have a job assignments table to link processors and jobs, make the job FK unique and interpret forced rollbacks as indicating that another thread grabbed the job before you did. Then again, if your queue is only running idempotent functions it wouldn't matter if you had more than one thread doing the same work, it would just be a waste of time.
评论 #4869009 未加载
评论 #4867812 未加载
评论 #4869610 未加载
NathanKPover 12 years ago
For a while I was using my own custom written job distribution system built on the database but then I discovered Gearman and since I implemented it I have seen increased reliability and productivity with creating new jobs.<p>I do not recommend rolling your own system. Use something that is already built as a server to accomplish the task. Amazon SQS is also a good solution.
tbrownawover 12 years ago
Just what sort of "distributed" is this talking about, that there's a central DB to put the queue in?
评论 #4869312 未加载
mtaubmanover 12 years ago
What pun?
评论 #4873384 未加载