TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

One-Process Programming Notes

185 点作者 jaffee将近 7 年前

14 条评论

kodablah将近 7 年前
From experience, before you just spinkle big ass string literals throughout your Go code that communicate with Sqlite, just toss all of your persistence in a separate package if you can. I&#x27;m not saying some crazy abstraction, just put it some place common if you don&#x27;t have a lot of advanced SQL needs, and put in the minor extra work serializing to and from structs. You&#x27;ll be happy when you want to add query logging, find certain usages, need code reuse, and if you ever want to change your persistence approach.<p>Also, devs should be aware of some of the concurrency limitations of Sqlite. Devs need to essentially share a single connection across the app, and serialize the writes.<p>Also, side note, I&#x27;ve found one really neat use case for Sqlite over other DBs: simple disk encryption. I wrote a Go wrapper for one of the popular ones [0], but in general if you distribute your app in other ways you may want to offer your users encryption that you can&#x27;t easily with other DBs. Sure you lose some performance, but sometimes it&#x27;s worth it (again, less in the server case where you control it and&#x2F;or can do fs-level encryption, but definitely in the distributed app case).<p>0 - <a href="https:&#x2F;&#x2F;github.com&#x2F;cretz&#x2F;go-sqleet" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;cretz&#x2F;go-sqleet</a>
评论 #17706023 未加载
pavlov将近 7 年前
<i>&quot;Most problems do fit in one computer, up to a point. Spend some time determining where that point is. If it is years away there is a good chance one computer will do.&quot;</i><p>I&#x27;ve been following this principle for a decade, and 98% of the services I&#x27;ve built have never needed to grow beyond one instance. The ones that did were solved by breaking out specific functions into their own managed services, not by applying a wide horizontal scaling solution.<p>If you build for scaling before you have anything running that can be tested against, it&#x27;s highly likely that you&#x27;re actually scaling the wrong thing. It&#x27;s like optimisation: to get any meaningful benefit, you&#x27;ll want to surgically solve the actual bottleneck rather than sprinkle hope-based gestures all over the place. (I&#x27;m reminded of the article that was posted the other day, where the writer experienced a Firebase slowdown and tried to solve it by upgrading hundreds of npm packages to their latest versions. That&#x27;s what I mean by hope-based gestures. Premature &quot;web scale&quot; architectural tripping is essentially the same kind of shotgun approach.)
评论 #17705155 未加载
pjc50将近 7 年前
&quot;One process programming&quot; is a really under-rated philosophy. Communication overhead can be substantial and concurrency issues are harder to debug. A single process that you can just stop and inspect in the debugger is easier to work on. Especially if you&#x27;re only one developer as well.<p>Two big pieces of software I worked on used this philosophy. One was <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Zeus_Web_Server" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Zeus_Web_Server</a> - nowadays its niche is occupied by nginix, but back then its advantage was being a one-process-per-core select()-based solution, compared to Apache&#x27;s process-per-concurrent-request solution. It was also absurdly, unnecessarily portable, being written in vanilla POSIX. Ran on a dozen different flavours and architectures of UNIX almost all of which are now dead.<p>Another was chip design software for a startup. Eventually it would have been useful to parallelise computation, but in the meantime it was <i>so</i> much faster to develop it and just buy big machines and leave it running overnight.<p>See also <a href="https:&#x2F;&#x2F;adamdrake.com&#x2F;command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html" rel="nofollow">https:&#x2F;&#x2F;adamdrake.com&#x2F;command-line-tools-can-be-235x-faster-...</a>
interfixus将近 7 年前
Unsurprisingly, comments start pointing the way to added complexity like Postgres and containers and whatnot, but this is <i>exactly</i> the right approach for probably the large majority of webapps out there. One executable, one db-file, up and running anywhere within a very few minutes. With a modicum of intelligence thrown into the design, you&#x27;re never ever going to see SQLite acting bottleneck. Few sites have request rates on the scale of tens of thousands per second.<p>Developers of would-be federated services to decentralize the web, please take note. This is how you go about ensuring wide deployment. Simplicity, if anyone remembers the word.
oavdeev将近 7 年前
If you&#x27;re using AWS anyway, I think using their managed Postgres &#x2F; RDS could make this a lot easier operationally. No need for custom syncing&#x2F;backup scripts, easier to scale the storage up.<p>To the larger point, while I totally agree with the premise -- most apps will never need to scale beyond one large instance, I&#x27;m not exactly sure what&#x27;s the actual tradeoff is. If you&#x27;re writing a simple CRUD app, it is not really controversial that it shouldn&#x27;t need any complex distributed stuff anyway, it is just a simple python&#x2F;Go app and a data store.<p>Most &quot;fancy&quot; things outside that single process paradigm, such as avoiding using local disk and using S3&#x2F;RDS&#x2F;document stores, using containers for deployment, etc. usually have more to do with making the app operationally simpler and easy to recover from instance failures than scaling per se.
评论 #17702774 未加载
评论 #17702775 未加载
erdaniels将近 7 年前
&quot;<i>Typical concerns about the impact on software quality of adding C code to Go do not apply to SQLite as it has an extraordinary degree of testing. The quality of the code is exceptional.</i>&quot;<p>It could be worth noting the thread concurrency limit imposed by cgo calls locking the OS thread to the calling goroutine. For example, 128 concurrent requests running long statements would take up 128 OS threads.<p>Of course any language not using a lightweight threading model has this issue either way if each request is mapped to one thread but it seems worth it to point out that you may want to limit your own concurrency at the request level or with properly sizing a connection pool (like the one present in the sqlite driver).<p>Edit: whatever the solution to concurrency is, benchmarking it is a good idea for any sized product going into a production environment. There&#x27;s a good article about this by the people at Cockroach labs: <a href="https:&#x2F;&#x2F;www.cockroachlabs.com&#x2F;blog&#x2F;the-cost-and-complexity-of-cgo&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.cockroachlabs.com&#x2F;blog&#x2F;the-cost-and-complexity-o...</a> (see Cgoroutines != Goroutines)
评论 #17706079 未加载
hathawsh将近 7 年前
This is a good article, but I feel like using Postgres instead of SQLite on the server side would achieve his goals even better:<p>- No need to link with C code. There&#x27;s a pure go driver for Postgres.<p>- You get the &#x27;psql&#x27; command to inspect and change the database, even while the web app is running.<p>- You get a lot more database features.<p>- You can keep the deployment simple by running Postgres on the web app server until you need to scale up.<p>- Conversion to a full blown cloud database and horizontal scaling is just a matter of configuration.<p>I understand the desire to keep the stack as simple as possible, but it looks to me like Postgres would actually be simpler than SQLite for the server side.
评论 #17702375 未加载
评论 #17702427 未加载
评论 #17702629 未加载
评论 #17704206 未加载
chj将近 7 年前
While reading this, please keep in mind this is for indie developers, who needs to do all kinds of jobs. Chances are SQLite is the one thing that everyone must learn to use, if your business involves building apps. So if you can use it on server side, it would save a lot of needless learning. Same reason as how nodejs got started.<p>Some may say, well, sql servers are not that hard to learn. Well, that is because you live with it day and night. Try going back to it after 1 year of break.
cabaalis将近 7 年前
&gt; I claim typical services do not hit this scaling limit.<p>This seems to be the main push of the article, and I for one agree with that statement.
justinholmes将近 7 年前
I&#x27;ve done something similar to keep it small with potential to go multi region for one of my new projects using Bitbucket and Google Cloud.<p>Bitbucket pipelines that create an image of the app, create instance template with a container &quot;gcloud beta compute instance-templates create-with-container&quot;, create instance group, create DNS, global load balancer, SSL certs and rolling update of the instance groups with a new version (if applicable).<p>Using preemptible g1-small to keep costs down but deployed in at least two zones per region.
评论 #17705647 未加载
bvinc将近 7 年前
Just today, I finished writing my own low level go sqlite driver. I haven&#x27;t published it yet. I really thought there was a need for it. I hate the database&#x2F;sql integration that the other drivers have. Now I&#x27;ll have to see if my driver has any need to exist.<p>This always seems to happen when I write something.
评论 #17705474 未加载
评论 #17706357 未加载
评论 #17704216 未加载
mikhailfranco将近 7 年前
<i>Scalability! But at what COST?</i><p>Single process performance and the COST principle, from the iconoclastic (some might say bombastic) McSherry:<p>[1] blog <a href="http:&#x2F;&#x2F;www.frankmcsherry.org&#x2F;graph&#x2F;scalability&#x2F;cost&#x2F;2015&#x2F;01&#x2F;15&#x2F;COST.html" rel="nofollow">http:&#x2F;&#x2F;www.frankmcsherry.org&#x2F;graph&#x2F;scalability&#x2F;cost&#x2F;2015&#x2F;01&#x2F;...</a><p>[2] pdf <a href="https:&#x2F;&#x2F;www.usenix.org&#x2F;system&#x2F;files&#x2F;conference&#x2F;hotos15&#x2F;hotos15-paper-mcsherry.pdf" rel="nofollow">https:&#x2F;&#x2F;www.usenix.org&#x2F;system&#x2F;files&#x2F;conference&#x2F;hotos15&#x2F;hotos...</a>
TooBrokeToBeg将近 7 年前
Why is half the blog post about SQLite? Use SQLite, because everyone has or will. Yea, got it.
gf263将近 7 年前
I really enjoy the minimalism.