TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

How our users exploited concurrency and how we fixed it

60 pointsby EvilTroutover 12 years ago

12 comments

rlpbover 12 years ago
Even better, rearrange your database to store immutable facts instead of mutable state.<p>In this example, you might have a goal_completion table to which you append a row when the player completes a goal, instead of changing an existing row. The table could have suitable unique constraints to make sure that each player can only complete a goal once.<p>This way, it is much harder for your data to accidentally be modified in a way that it becomes wrong.<p>This also gives you a way to store facts about the completion for free, such as a timestamp for each goal completion. And having data like timestamps is really useful for auditing, debugging and testing.
评论 #5044730 未加载
wulczerover 12 years ago
I believe this won't work on PostgreSQL, which will always say one row has been updated, even if the new value is the same as the old value.<p>There are numerous ways around that, from less fancy to really fancy.<p><pre><code> 1. use SELECT FOR UPDATE, which will lock the row (I'd say that's the normal way) session1&#62; BEGIN; session1&#62; SELECT * FROM goals WHERE player_id = ? FOR UPDATE session2&#62; BEGIN; session2&#62; SELECT * FROM goals WHERE player_id = ? FOR UPDATE # session2 is now hanging session1&#62; UPDATE goals SET completed = true WHERE goal_id = ? session1&#62; UPDATE players SET points = points + 1 WHERE player_id = ? session1&#62; COMMIT; # session2 now proceeds, sees that the goal has been completed, forfeits awarding the reward 2. use suppress_reduntant_updates_trigger (fun) session&#62; CREATE TRIGGER suppress_goal_t BEFORE UPDATE ON goals FOR EACH ROW EXECUTE PROCEDURE suppress_redundant_updates_trigger(); session&#62; UPDATE goals SET completed = true WHERE goal_id = ? UPDATE 1 session&#62; UPDATE goals SET completed = true WHERE goal_id = ? UPDATE 0 # now you can use the approach mentioned in the article 3. use true serialisability session1&#62; BEGIN; session1&#62; SET transaction_isolation TO serializable; session1&#62; SELECT * FROM goals WHERE player_id = ? session2&#62; BEGIN; session2&#62; SET transaction_isolation TO serializable; session2&#62; SELECT * FROM goals WHERE player_id = ? session1&#62; UPDATE goals SET completed = true WHERE goal_id = ? session1&#62; UPDATE players SET points = points + 1 WHERE player_id = ? session1&#62; COMMIT; session2&#62; UPDATE goals SET completed = true WHERE goal_id = ? ERROR: could not serialize access due to concurrent update # session2 now has to rollback the transaction </code></pre> There's a few more, but I ran out of steam typing ;) Yay, Postgres!
评论 #5043576 未加载
vemvover 12 years ago
Really not that much of a subtle bug I'm afraid, the pattern of "mutating the condition itself on success" is error-prone in single-threaded code too.<p>A good example being asynchronous javascript, where you'd do this to prevent issues like the exposed in the article:<p><pre><code> var some_mutable_condition = false; var requested = false; if (!some_mutable_condition &#38;&#38; !requested){ requested = true; // ensure one request is performed at most $.ajax(/*async code that'll set some_mutable_condition to true*/); }</code></pre>
评论 #5045699 未加载
mchermover 12 years ago
I like collecting tales like this one. They are useful for scaring new programmers who think they understand how to safely do multi-thread programming.
emeralddover 12 years ago
If I remember correctly, one of the ways around this is to do a no-op update before querying. So you'd update a field in the goal table to it's original value, which would cause the database to lock the table for the duration of the transaction.<p>Also, if I remember correctly, this is one of the things that can be solved with an MVCC aware engine.<p><a href="http://en.wikipedia.org/wiki/Multiversion_concurrency_control" rel="nofollow">http://en.wikipedia.org/wiki/Multiversion_concurrency_contro...</a><p>With MVCC a transaction should be aborted if another thread modifies the underlying data.
amikazmiover 12 years ago
No need for a workaround, you have Optimistic Locking (via lock_version field)<p>It raises an exception in case you try to update stale data.<p><a href="http://api.rubyonrails.org/classes/ActiveRecord/Locking/Optimistic.html" rel="nofollow">http://api.rubyonrails.org/classes/ActiveRecord/Locking/Opti...</a>
评论 #5043743 未加载
评论 #5043604 未加载
评论 #5043746 未加载
jessaustinover 12 years ago
Another solution would be to have e.g. a "Completions" table indexed on user and on date. Then the RDBMS wouldn't allow duplicate goal completions for the same user and day. If you wanted to allow up to X completions/day, just add a new column to track that, and limit that column to the range [0, X). I often find that updating rows in response to transactions is a code smell. Each transaction should write one or more rows to one or more tables. Of course, if you have denormalized summary tables, they might have to update, but you only do that when you know you have to.
aufreak3over 12 years ago
Umm ... and the 'Player.transaction' doesnt seem to be one then if it allows two transactions to overlap?
评论 #5043587 未加载
评论 #5045706 未加载
voidlogicover 12 years ago
"I was worried that I’d have to resort to semaphores or some kind of other exotic programming construct."<p>I'm not saying a semaphore would have been appropriate here but- For real? A semaphore is exotic? It is an one of the oldest original synchronization methods. Semaphore were developed by Dijkstra in 1965. The author's C-S program failed him.
评论 #5044428 未加载
kaffeinecomaover 12 years ago
Use Optimistic Locking:<p><a href="http://railscasts.com/episodes/59-optimistic-locking-revised?view=asciicast" rel="nofollow">http://railscasts.com/episodes/59-optimistic-locking-revised...</a>
jamiebover 12 years ago
Best to discover the joys of concurrency while writing a game than a bank. =) Welcome! Our playground is a lot of fun!
EGregover 12 years ago
Basically, the correct way is to use transactions. Use a transactional database because it supports MVCC.
评论 #5043922 未加载
评论 #5045210 未加载
评论 #5043927 未加载