TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Surviving Django, if you care about databases

172 点作者 pauloxnet将近 5 年前

33 条评论

ris将近 5 年前
I can&#x27;t say I can recommend following almost any of this as &quot;general&quot; advice. It is, at best, advice from a complete expert <i>for</i> complete experts who are already pushing the edge of what django should be used for, and who never expect to have to hand off their project to an average web developer. I&#x27;ve had to come to terms with the fact that the average web developer <i>is</i> going to screw up e.g. writing their schema by hand in sql. And I know I&#x27;ll likely be the person inheriting this two generations of developer after the original wizard left.<p>For an average web developer, someone just getting into django, or someone who isn&#x27;t yet running into the limitations that the author apparently is, this is a recipe to just make a huge mess.<p>It&#x27;s also very <i>specific</i> advice. Recommending ignoring cross-database concerns.. I&#x27;m sure that&#x27;s fine if you&#x27;re an end-user developer building and running your own product, but if you&#x27;re considering writing a library or reusable app you&#x27;d like to be useful to other django users, it&#x27;s probably worth a second thought.<p>It&#x27;s ironic though that the author makes such a prominent mention of &quot;You Ain&#x27;t Gonna Need It&quot;, because that&#x27;s exactly what I&#x27;d say about most of the pimped-up tweaks mentioned here. 99% of people aren&#x27;t going to run into these limitations and will produce a much more maintainable app by following the standard django doctrine.
评论 #24079958 未加载
评论 #24076016 未加载
评论 #24080468 未加载
评论 #24089998 未加载
nikisweeting将近 5 年前
In my 8 years of working on large Python codebases, I think the Django ORM is one of the best pieces of software I&#x27;ve had the joy of using. I often recommend it independent of the rest of Django (e.g. to people building Flask apps that need an ORM+migrations).<p>Especially when working on a team with many junior developers who don&#x27;t know SQL (myself included in the early days), the Django ORM has been a shining beacon of reliability and consistency allowing us to do complex database schema changes without ever having to worry about messing them up or not being able to revert our changes.<p>I don&#x27;t think I can say the same for any other ORM I&#x27;ve worked with, in particular I have really bad memories of trying to match Django&#x27;s migration reliability and flexibility using SQLAlchemy + Alembic.<p>Also in regards to switching databases, the ability to swap out PostgreSQL for SQLite on smaller projects without having anything break is a feature I&#x27;ve relied on several times, and I&#x27;m extremely grateful maintaining that clean break in the layers of abstraction was prioritized by the Django team.<p>Obviously the author of this article has had different experiences (and has considerable authority in this field), but I caution that their experiences are not universal among Python devs.
评论 #24077622 未加载
评论 #24076004 未加载
评论 #24080533 未加载
评论 #24077528 未加载
评论 #24080003 未加载
评论 #24079532 未加载
评论 #24081326 未加载
bb88将近 5 年前
A couple of real world data points.<p>Two Fortune 500 companies I worked at had projects in Django that used native SQL to do migrations. This led to errors in prod during deployment.<p>The order of the scripts had to run in a proscribed order. They had to be listed on the deployment instructions. So the migrations would run, and then fail during a prod deployment for various reasons. Most notably a dependent script hadn&#x27;t run in prod yet. They had run in the Dev&#x2F;QA environments and everything had run just fine. Or Dev&#x2F;QA had been hand tweaked and that change hadn&#x27;t made it to the script during prod deployment because they were testing it for a month to see performance before moving to prod, etc... etc...<p>It was all ugly.<p>In one of the two companies they finally decided to stop handwriting SQL and started trusting the Django migration system, and the problem with deployments went away.<p><i>AND</i> they still were able to hand write sql to optimize the performance of the DB if needed.
评论 #24076145 未加载
评论 #24077425 未加载
评论 #24076880 未加载
评论 #24076879 未加载
评论 #24075732 未加载
VWWHFSfQ将近 5 年前
&gt; The motivation for writing this article comes from knowledge sharing with the team I&#x27;m currently collaborating with, who are using Django the canonical way. I am convinced they can better use the tools they have. Let&#x27;s see if they can be persuaded to drop Django migrations...<p>I&#x27;ve built and worked on enough large Django projects to know that if you don&#x27;t do Django the canonical way, then I don&#x27;t want any part of the project. If I joined a team and saw this weird hodge-podge of custom migration framework and shadowing ORM fields I would nope-out immediately. It sounds like this post author just doesn&#x27;t want to use Django&#x27;s ORM. And that&#x27;s fine, you don&#x27;t have to.
评论 #24075785 未加载
zzzeek将近 5 年前
Daniele is a great guy and I deal w&#x2F; him regularly as he&#x27;s the author of psycopg2. However the mistake he makes here is in the realm of database abstraction tools having the sole rationale of &quot;database agnostic code&quot;. That&#x27;s really not the main rationale, nor is it really &quot;hide the database so people don&#x27;t have to learn SQL&quot;. The main one is &quot;automation of work&quot;. A lot of the comments here refer to situations where hand-rolling everything always was just too error prone. There&#x27;s a thing we do in computers when a job is very repetitive, tedious, error prone, and follows a complete pattern that you can define algorithmically, which is, <i>write a program to do it!</i>. that said I&#x27;m sort of glad Django took the heat on this one and not SQLAlchemy so I can get to bed without obsessing :)
评论 #24081015 未加载
airstrike将近 5 年前
<i>&gt; Django needs it because a web framework not tied to a single database vendor is more valuable than one tied to a specific one - and that&#x27;s fair enough. But you don&#x27;t: your web program, most likely than not, will not have to switch from one database to another</i><p>This is missing the forest for the trees.<p>Sure, my one app won&#x27;t need to migrate from MySQL to Postgres a year down the road, but as a developer, I&#x27;d much rather be allowed to abstract away each vendor&#x27;s idiosyncrasies and be able to focus on my Python code regardless of whether I&#x27;m using sqlite, PostgreSQL, MariaDB, MySQL, Oracle or even MS-SQL
评论 #24077005 未加载
评论 #24080811 未加载
acdha将近 5 年前
This article is trying too hard to dress the author&#x27;s limited experience up as absolute truth. For example:<p>&gt; Have you got PostgreSQL in production, but you want to test with SQLite because it&#x27;s easier to set up? If so, your tests are just a tick-box exercise: you are not testing anything remotely plausible and resembling your live system.<p>This is not completely wrong but it&#x27;s just telling you that the author has never worked in an environment where it makes sense. Anyone who has knows that this approach can be used to catch a high fraction of errors locally, while not precluding a full slower CI build.<p>&gt; Even if you added some form of manual auditing to each save() method, it will not capture changes made outside Django. It wouldn&#x27;t be very secure either: Django uses a single user to access the database so if someone manages to hijack that user, they would be able to change the data in the database and alter the audit tables to hide their traces.<p>Similarly, while it is true that Django defaults using a single user for everything you can easily configure multiple connections (containing, say, an issue with your normal web code but not your management commands), you can grant access to only SELECT and INSERT on the audit log table but not UPDATE or DELETE, etc. There&#x27;s nothing with the separate approach, of course, but it&#x27;s not like it&#x27;s some great discovery that frameworks are not intended to cover every possible use-case and sometimes you want to turn them off.<p>The extended “I hacked around because understanding the migrations module seemed like more work and then I was too far in to reconsider” section has the same problem. Being surprised that the migration system triggered an update when you renamed a field is, well, exactly what it&#x27;s designed to do. That&#x27;s exactly why there is an entire section in the documentation which tells you exactly what to do in that situation and how to avoid data loss — in that case, if the problem wasn&#x27;t making foo.bar magic (which it almost always is) you&#x27;d create multiple migrations to add the new column, move (and usually convert) the data, and drop the old column. You can even run raw SQL if there&#x27;s something like an efficiency hack which makes it worthwhile. This takes a couple of minutes and is reliable.
评论 #24075641 未加载
评论 #24075590 未加载
评论 #24076912 未加载
vivekseth将近 5 年前
Not mentioned in the article, but the author also built psychopg2 (<a href="https:&#x2F;&#x2F;www.varrazzo.com&#x2F;software&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.varrazzo.com&#x2F;software&#x2F;</a>)
评论 #24075589 未加载
评论 #24075455 未加载
l0b0将近 5 年前
This sounds like a &quot;golden hammer&quot; argument[1], basically saying that SQL can do everything Django can do, and if you know SQL <i>really, really well</i> you can do powerful stuff which would be clunky in Django. Of course that&#x27;s true, because it&#x27;s true of any two tools which have overlapping functionality. But for the vast majority of things you need to do on a daily basis, and for developers with &gt;10x more experience with Python than SQL it just doesn&#x27;t make sense to use raw SQL any more than absolutely necessary. As soon as you use raw SQL in Django most of the <i>maintainability</i> advantages of an ORM disappear.<p>All that said, I totally agree about throwing out DB portability for anything but the simplest apps (or libraries, of course). Portability is extremely costly in terms of which features we have available to and how maintainable the resulting code is, so it should only be a goal when absolutely necessary.<p>[1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Law_of_the_instrument" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Law_of_the_instrument</a>
DangitBobby将近 5 年前
&gt; Much of the complexity of this system is designed to give you a specific feature: migrations abstracted from your database. In my opinion, if there is something less likely to happen than writing a complex app to run on interchangeable databases, it&#x27;s the need to repeat the same history of migrations.<p>Unless you have a dev, staging, and prod setup. Then there&#x27;s a good chance you&#x27;ll encounter a wide range of variation in the current database schema compared to your models. I use the ability to roll back and forward migrations <i>all the time.</i> Django ORM and the migrations (and the admin site) are <i>the</i> features that make it impossible for me to justify the use of other frameworks or languages in the vast majority of cases. The productivity boon is just too much.<p>With that said, I have a personal policy of not letting the ORM get in the way. If I find myself reaching out to the documentation for how to write a particular query (something like window functions, for example), that&#x27;s a good indication that I may be better off writing the raw SQL.
ensignavenger将近 5 年前
I support a very large database, containing product data for a major US retailer. If I understand the history of this database correctly (I am not directly on the team that manages it, but my team supports them in various ways) the database began life as an MS SQL DB. Many years ago it was migrated to MySQL. Now, they are in the process of migrating it to Postgres. So yes, migrating data to different DB&#x27;s does happen, especially when you consider an applications life over decades.
评论 #24077410 未加载
kissgyorgy将近 5 年前
&gt; And with comments! Explaining why a certain index or constraint exist!<p>You can put comments in Python files...<p>&gt; With constraints named meaningfully, available for manipulation, not auth_group_permissions_group_id_b120cbf9_fk_auth_group_id.<p>You can name your migrations with:<p><pre><code> manage.py makemigrations --name &quot;prettier_name&quot; </code></pre> &gt; In my opinion, if there is something less likely to happen than writing a complex app to run on interchangeable databases, it&#x27;s the need to repeat the same history of migrations.<p>Maybe he never worked in a team? Or wrote any integration tests at all? Or implemented rollback? Because those are pretty much basic things for every serious project.<p>&gt; I have a patch_db.py script that I have used, with small variations, in several projects.<p>So you are basically wrote your own schema migrations tool, but inconsistent across projects, and you maintain it yourself. Not a good idea IMO.
评论 #24080704 未加载
BiteCode_dev将近 5 年前
This article misses the point of the django orm and migration system completly.<p>They are meant to allow for the dev of an ecosystem.<p>This is why you have so many django plugins, and why they are so easy to integrate and play together.<p>This is why it&#x27;s easy to have a dev envs, tests, signals and api generation with django.<p>It trades performances and flexibility for this. It&#x27;s nothing more than a classic technical compromise.<p>You may need raw sql later on in your django project. But if you feel like starting with it, just go with flask. Django makes no sense.
评论 #24075553 未加载
评论 #24075799 未加载
评论 #24075557 未加载
评论 #24075710 未加载
评论 #24075539 未加载
orf将近 5 年前
Migrations can, and do, run arbitrary SQL. At the end of the day the migrations framework is just a SQL dependency graph, it can be used without models if that&#x27;s what you wish. By itself that is way better than ad-hoc scripts.
评论 #24075497 未加载
vandahm将近 5 年前
I wish I could work on these magical projects that people with blogs work on, in which everything works like it&#x27;s supposed to do. In my experience, though, it&#x27;s not unusual to be forced by management to migrate to a new RDBMS a year into a project. The last time it happened to me, we were using Rails and had very little difficulty migrating from PostgreSQL to MySQL. We didn&#x27;t need to alter application code or rewrite tests and only had to manage config changes and migrate the data. I imagine the transition wouldn&#x27;t be much different if we had been using Django.
whakim将近 5 年前
Seems like the author should&#x27;ve just used a much less opinionated framework. The last thing I want to do is choose a batteries-included framework and then take on the responsibility of maintaining my own system to manage SQL migrations. Also unclear where the author has worked, but I&#x27;ve had to replay a bunch of migrations in order on many occasions.
tuatoru将近 5 年前
The implication of the article is that if you&#x27;re using a relational database management system, <i>you need to know SQL</i>.<p>I agree. I wholeheartedly recommend Stephane Faroult&#x27;s <i>SQL Success</i> as a beginner book. Read that with attention, and you&#x27;ll be ahead of 95% of people messing around with RDBs.
评论 #24076287 未加载
lewisjoe将近 5 年前
A lot of problems stated are characteristic of any ORM not just Django. People pick ORMs specifically for trading convenience and tooling with loss of control. Any escape hatches that the ORM provides is by definition hacky.<p>These days when I know it’s a long term project, I prefer laying out the domain entities as plain classes and wrapping database operations as class methods with raw database statements.<p>This lets me use the underlying databases to its full potential and gives me enough control without leaking database code into application logic.
some_developer将近 5 年前
&gt; How many times have you worked on a project and, after 1-2 years of development, you have changed to a different database? &gt; &gt; &gt; I can tell you how many times it happened to me, I counted them: exactly never.<p>I guess this is the crux.<p>You hardly know in advance this is going to happen.<p>I experienced this transition (mysql&gt;pgsql), though not with Django.<p>The ORM was perfect. If I recall correctly, only the customized SQL queries (only a few present) had to br adapted, everything else just worked.<p>11&#x2F;10 would chose such a framework again.
mnm1将近 5 年前
I&#x27;ve migrated a production app from mysql to pgsql although not with django but a different orm and language. The app also had some custom sql. The reason was for proper json support as we have a 2tb table with unstructured data (1tb at the time) that was slowing mysql down severely (most used table by far). A later version of mysql with json support was not available on aws so wasn&#x27;t an option. This was about two to three years in so the app was actually almost finished and in maintenance mode.<p>It worked. It took many months and copying the data and doing the migration was a pita. The orm helped in that I had to modify fewer queries by hand, but modifying the queries we did have was trivial also. It&#x27;s one of only two times off the top of my head that the orm was a net benefit rather than a liability in the apps that use it. Would I use an orm in the future in case this comes up? Hell no. I&#x27;d pick a proper db first, aka pgsql (that decision was before my time). But even if I didn&#x27;t, I&#x27;d still prefer to convert the sql one by one from mysql to pgsql rather than incur the horrific penalties and extremely slow run time and development time the orm imposed on us. Developing with the orm was many times slower and the runtime was ten times slower than the actual query time due to hydration. Rewriting a few thousand queries is nothing compared to losing months to the slow development speed and the slow runtime speed which required a completely separate rewrite of most of our functionality alongside the orm just so we could serve api requests in a manner somewhat reminiscent of not completely slow.<p>So yeah, even when switching from mysql to pgsql, the orm wasn&#x27;t worth it. It never is or has been.
zumachase将近 5 年前
We&#x27;ve got a core product running on Django, and one thing the author doesn&#x27;t mention is testing migrations. The migration system is wonderful, except for the difficulty in testing migrations. There&#x27;s no sane&#x2F;official way to do this. And it&#x27;s such an important thing that I don&#x27;t get why the Django crew haven&#x27;t tackled it.
评论 #24075848 未加载
评论 #24075579 未加载
评论 #24076166 未加载
40four将近 5 年前
Pretty good read! But I don&#x27;t like how the author tries to characterize generic ORM&#x27;s like Django&#x27;s as &quot;Portability at all costs&quot;. I don&#x27;t think the point is that you can switch DB&#x27;s a few years down the road if you choose. He&#x27;s right, nobody does that. Ever :) The advantage is that you can <i>start</i> a new project with whatever DB you already have, or are comfortable with. So, if you want to begin a new Django project, you aren&#x27;t locked into using MySql, or Postgres, or what-have-you. It is true you sacrifice being able to leverage the full gamut of features a particular engine might offer, but that is a compromise you make with any ORM right? Or, if the limitations are to much to bear, just don&#x27;t use Django ORM, it&#x27;s not required, it&#x27;s just the path of least resistance.
tabbott将近 5 年前
I agree Django&#x27;s migration system has a few unfortunate quirks (most notably the help-text-change-migrations), but it&#x27;s also worked great for Zulip across over 300 database migrations, and I wouldn&#x27;t recommend anyone do their own migration framework on raw SQL files by hand.<p>The issues mentioned here in this article are real (E.g. we have a lint rule blocking importing other code inside migration files), but the author is also ignoring the benefits of Django&#x27;s system, namely that it&#x27;s a well-documented system that does most of the work of managing migrations for you, and does a good enough job that the vast majority of developers working on a large software project (like Zulip, with 96 people who&#x27;ve contributed changes to database migrations) will do it correctly without special training just from the Django documentation.<p>And the downsides are certainly minor compared to the obvious downsides with rolling one&#x27;s own system with raw SQL (E.g. the fact that nothing validates that the model definitions match the database schema in the author&#x27;s proposal!). I wouldn&#x27;t recommend anyone take the author&#x27;s advice for what to do instead, but this article is helpful reading for anyone curious about what issues one will encounter after a few years working with Django.<p>&gt; How many times have you worked on a project and, after 1-2 years of development, you have changed to a different database?<p>FWIW, Zulip was originally developed with MySQL, and then about a year into developed we switched to Postgres. It also had a phase where we ran SQLite in development environments. Building on Django made our life a lot easier doing this.<p>&gt; Once the migrations have been applied to all your testing and production systems, and all your developers&#x27; databases, just remove them.<p>This is only true if you don&#x27;t care about being able to use tools like `git bisect` where you might want to switch a development environment to older versions and back again (I do this often as part of).<p>&gt; If you do it in a loop, it results in a &quot;ripple load&quot;, meaning you&#x27;ll run 50 queries, each one fetching one record, while you could have run a single query fetching 50 records, or just adding an extra JOIN to another query.<p>This is my biggest complaint about Django&#x27;s ORM -- lazy fetching of foreign keys is great in a management shell but rarely desirable in production, and it results in inexperienced developers doing database queries in loops all the time. We address this with a bit of middleware that formats log lines like this e.g. `41ms (db: 10ms&#x2F;2q)` (showing how much time was spend doing database queries and how many queries were done) -- which lets one easily identify problematic cases of this both in development and in production.<p>But if Django had a setting that just disables the lazy fetching of foreign keys feature (making accessing an unfetched foreign key an error), I&#x27;d turn it on for Zulip.<p>See also <a href="https:&#x2F;&#x2F;zulip.readthedocs.io&#x2F;en&#x2F;latest&#x2F;subsystems&#x2F;schema-migrations.html" rel="nofollow">https:&#x2F;&#x2F;zulip.readthedocs.io&#x2F;en&#x2F;latest&#x2F;subsystems&#x2F;schema-mig...</a>.
评论 #24078407 未加载
评论 #24076148 未加载
thedanbob将近 5 年前
Interesting comparing this to Ruby on Rail&#x27;s ORM, which takes a relatively hands-off approach. ActiveRecord will produce a migration for you when you generate a model from the command line, but otherwise migrations are entirely the developer&#x27;s responsibility. And the way he suggests handling patches sounds a lot like how ActiveRecord does it: keep track of what migrations have already been applied and only apply newer ones, and for a new database just create the up-to-date schema rather than applying the entire history of migrations.
评论 #24075881 未加载
评论 #24078453 未加载
评论 #24075805 未加载
rhema将近 5 年前
I made a custom Django system for my personal website in 2013. It became to old to work with my web host. So, now I just run the django locally long enough to run a wget mirror. Voilà, static content.
luord将近 5 年前
This is tricky, on the one hand keeping everything related to the database close to or in the database is probably cleaner (in the clean architecture sort of way), as it eases a separation between the business rules and the data saving and loading.<p>On the other hand, doing this loses a lot of the sugar and development benefits of using the ORM (either Django or SQLAlchemy, which I prefer), that are often enough—and perhaps preferred—for 95% of projects.<p>If I were to choose, I&#x27;d pick the ORM, though trying to place it as high in the stack as possible.
Jasp3r将近 5 年前
Terrible article, especially the part about migrations. The automatic migration generation of Django of course cannot infer exactly how you want your schema to change if you start renaming or altering columns.<p>The author&#x27;s solution, instead of adding on to migrations then opts for completely abandoning a tried and tested migration framework for arbitrarily executing sql files? This is madness
jordic将近 5 年前
May Django work with multiple tenants on different postgresql schemas? Can Django migrations handle this in an easy way (better than sql).. When projects grow old (+10years), things trend to complicate (lot of hands, lot of changed requirements.. ), and that&#x27;s when the advices on the article had a lot of sense.. on the end data is there and the model is the database..
Humphrey将近 5 年前
History Lesson (if you care about migrations):<p>TLDR; I disagree: Django migrations are the best, and make reusable apps possible!<p>I&#x27;ve been using Django since before they introduced migrations and it was hell! Django only had &quot;sync_db&quot; then which would create your tables initially, but then you were on your own to write and run SQL manually for any model changes. This became especially annoying when using 3rd party apps, because if those apps changed the schema, you&#x27;d have to manually update your database for the 3rd party changes.<p>Then came along django-south which was a reusable app which provided migrations and solved all of the above problems. It became almost the standard approach, and third party apps often provided south migrations.<p>South became so popular, and was such a big improvement to the development workflow, that it&#x27;s codebase was merged into Django (after making some significant API improvements) to become Django migrations. I can still remember having to covert all the files in the migrations directory from South to Django migrations.<p>Now we have a defacto standard of migrations work, and they allow easy use of customer SQL. Seriously, if you hate python based migrations as much as the author does, write custom SQL but put it in a Django migration so you get all the benefits of being able to rollback migrations etc.<p>Summary: when writing all of your own code for a single project then there is only minor benefits from migrations. But they are still worth it! In fact, I write lots raw SQL in my migrations, eg for constraints, triggers, notify, and just modifying my data. If I run SQL, it goes in a migration.<p>But migrations shine the brightest for reusable first and third party apps, which lets admit, we all use in most projects. This allows a single reusable app to be used for all projects regardless of database. Also, as a consumer of a reusable app, I can let migrations do their thing, and not worry if an update will break my database.
jordic将近 5 年前
I would also add to the article, that when you try to alter a busy table add it a timeout (otherwise you will lock the world). SET statement_timeout = 50; Alter .... ;
vbilopav将近 5 年前
My advice is to avoid Django ORM altogether
1337shadow将近 5 年前
Probably they should try django-north instead of their patch_db.py
jordic将近 5 年前
Another thing important that the author doesn&#x27;t mention is that if you try to keep your sql model on the postgres side.. you will be able to introduce other languages (for example to optimize hot paths)<p>I agree 100% with the author, if you check other open source projects (on other languages) they mostly are based on the patch version sql feature. Sorry, the migrations feature from my point of view it&#x27;s absolutely over engineered..<p>I also agree with him, that doesn&#x27;t make too much sense to keep Django+drf, when if you use fastapi+pydantic+asyncpg you got all the real innovation on the python side..<p>Also it&#x27;s not a good argument that following the Django religion will let to proper future maintenance.. think on channels and the big mess around them :)<p>Sorry, I&#x27;m perhaps a bit biased, started with Django 0.9 till 1.9.. and right now I&#x27;m not using it for anything new.. on the way I swapped for go, and later come back to python thanks to asyncio.
评论 #24075902 未加载