What:<p>Feedgrid is a news aggregator for RSS and Atom feeds.<p>It lets you organize your feeds into related topics and view feed items in 2-dimensional grids, rather than a single infinite scroll column.<p>The goal is to have a highly configurable tool to organize diverse feeds in a visual way, and then easily check for new content with a bird's eye view.<p>Why:<p>- there’s a lot of stuff on the web, but you want it in 1 place, organized by categories that make sense to you<p>- RSS is nice in principle but most readers feel like an overloaded email inbox (i.e. kinda stressful—you already have other inboxes to deal with..)<p>- you like breadth-first search rather than depth-first search when you’re on the john and looking for some infotainment, but TikTok is frenetic and unsettling to you because you still like reading<p>How:<p>- we’re running a Django app with some minimal js in the frontend, and serve grid pages as a modest-sized html blob with enough content for one sitting, and offer old-school pagination to get more content<p>- the backend web scraping is entirely separate from the server, and uses scrapy. Scrapers are deployed on VMs, launching scrapy crawler instances dynamically based on feed SLA data in the db; there is no “scheduler”, but rather long-running parent coordinators launching child subprocesses as needed, so SLAs (and other attributes) can be modified by site admins without touching spiders