In iommi we built something imo much smarter: <a href="https://docs.iommi.rocks/en/latest/dev_tools.html#sql-trace" rel="nofollow">https://docs.iommi.rocks/en/latest/dev_tools.html#sql-trace</a><p>The idea is that:<p>1. We store all the stack traces and SQL calls<p>2. At the end of the request we group them to find N+1<p>3. If there are any N+1 (given some threshold), we print a big warning to stdout. Full traceback for one of the SQL calls and a few examples of SQL.<p>This is imo MUCH better. It means the console is <i>silent</i> when there is no issue. It also means the console is <i>loud</i> when there IS an issue. And the output you get is very helpful.<p>This approach is better than the one in the article, and superior to Django Debug Toolbar where you have to actively go look for problems instead of the machine telling you.
Heh, wrote something similar (django-queryinspect) a while ago, then abandoned it since iirc django debug toolbar has it builtin, and I moved to some other stuff.. maybe I should dust it off.<p>N+1 is fairly well understood and documented gotcha, and Django has excellent tools that make it easy to avoid it (as the article mentions).<p>To use Django ORM effectively in nontrivial apps, you do need to understand how relational databases work, and it's helpful to understand SQL. But it does make me a whole lot more productive (and happy!) than writing SQL by hand.
Not knowing anything about web development or Django or whatever, why isn't this just in a real database? Where you can do a join instead of 2000 individual queries?
Database queries are likely to be the most constrained resource in a typical enterprise application. They are essential but also extremely slow.<p>Because of this, that limited resource needs to be carefully managed and it needs to be straightforward to reason about the I/O behavior of your application.<p>Attempting to hide them away and obscure what is actually happening in your application is extremely counter-productive.
One solution is to not use an ORM.<p>These problems are framework problems. I find the cognitive overhead of worrying whether a framework might do the wrong thing or diagnosing problems after they arise often override any cost savings frameworks provide in the first place.
This is the best write-up of tools to use to avoid shooting yourself in the foot with Django N+1s that I have seen.<p>However, having used Django professionally for a while, I can say that it is really starting to show its age even if you deal with the N+1 problems. You still have to deal with:<p>1. Blocking database calls that cannot easily be used in an asyncio context<p>2. No types<p>3. 50% of package code being written without any regard for performance<p>If you're starting a project today and considering Django, I highly recommend you go with Nestjs and Prisma instead.
In Rails you can make this 3 queries. One to retrieve the board, one to get all the report IDs associated with the board and a third to get all the reports with the associated report IDs.<p>An example in Rails would look like:<p><pre><code> board = Board.includes(:reports).find(board_id)</code></pre>
> Django Querycount<p>This seems pretty much like a must have when you think about it: so where’s the equivalents for .NET, Java, Node, Go, Rails and others?<p>I can’t believe that every web framework that has an ORM or just a connection to the database doesn’t provide summaries like this.<p>At best, you can look into some tracing options that might reveal where the the execution time went.<p>I do hope that there are options out there (or might have to try and fail writing something myself) because I can immediately think of a few legacy projects where N+1 is prevalent, maybe even in the new code, depending on who wrote it; some people genuinely believe that using the service pattern and a lot of DB calls for a single API request is okay.
We built Joist, an ActiveRecord-style ORM in TypeScript, on top of Facebook's dataloader library, and haven't had a single N+1 in ~4 years:<p><a href="https://joist-orm.io/docs/goals/avoiding-n-plus-1s" rel="nofollow">https://joist-orm.io/docs/goals/avoiding-n-plus-1s</a><p>Any "query in a loop", whether a manually-written `for` loop, or an "emergent for loop" from 100 `after_update` lifecycle hooks firing at once, are always auto-batched, every time.<p>Joist is also likely why we actually like GraphQL, vs. it generally having a reputation for being a pita to implement (N+1s everywhere w/o tedious, manual batching).
Just orm things...<p><a href="https://wikipedia.org/wiki/Object%E2%80%93relational_impedance_mismatch" rel="nofollow">https://wikipedia.org/wiki/Object%E2%80%93relational_impedan...</a>