At Discourse we maintain a gem for automatic read-only when the main PostgreSQL/Redis is down: <a href="https://github.com/discourse/rails_failover" rel="nofollow">https://github.com/discourse/rails_failover</a>
> "I felt a bit odd using exceptions as the core of this workflow..."<p>i've done something analogous before and it seemed to me the exception framework was the right abstraction to use (though other opinions are welcome!). in this particular case, it seems like the main problem is that the trigger exceptions are not sufficiently granular/appropriate to describe the exact exception condition on which to trigger read-only mode:<p>* ActiveRecord::StatementInvalid<p>* PG::InsufficientPrivilege<p>rather than "DatabaseDownForMaintenance".
A similar approach, for both horizontally scaling your db and automatic read-only mode. Ensure you are following the rule to only attempt writes in POST requests. POST requests get their queries sent to the primary db. GET requests send db queries to a read-only standby, unless the client sent a POST request recently (storing the last POST timestamp in a cookie or a session db, and the value for 'recently' is the current database replication lag). Catch any connection failures for requests using the primary db, and retry on a read-only standby. Catch db write failure to a read-only standby and replace with a 'down for maintenance' page.<p>We retrofitted the above behavior to an ~400k line monolithic Python app with minimal changes a bit over 10 years ago (the web framework was thankfully agreeable), shed over half the load on the master day 1, and could happily bounce the master db or do schema updates at will without needing to touch the appservers. Plug it into your CI system and you can do both automatic deployments and database updates, often with unnoticeable downtime if you are clever about your database patches.
An interesting idea. I know this will cause issues with Devise or something similar when the login token needs to be refreshed and it needs to update a record for that. I guess it depends on how the auth is set up.
I heard Chris talk over this on this The Bike Shed episode: <a href="https://www.bikeshed.fm/262" rel="nofollow">https://www.bikeshed.fm/262</a>, and thought he was going to do something horrible. I was relieved to read the post and see how sensible the solution ended up being! Great to know this is an option now!