TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Two Tier Architectures Are Anachronistic [video]

40 点作者 amcaskill超过 1 年前

4 条评论

mike_hearn超过 1 年前
It&#x27;s a tech talk so I watched it and made some notes. It&#x27;s an argument for using in-process databases when doing data science, rather than an external DB. The speaker pitches DuckDB as a concrete example, which seems to be an in-process DB for Python data frames.<p>The speaker presents measurements showing how much overhead the wire protocols for various DBs have. MySQL is the best, Postgresql is orders of magnitude worse due to a very inefficient binary format design. The best is still 10x worse than netcat.<p>Apache Arrow is trying to design a universal protocol for DB access that&#x27;s more efficient than what&#x27;s out there currently.<p>Speaker asserts that scale-out is usually not needed in data analytics, no need to use Spark etc unless you want it on your CV.<p>Audience member asks &quot;what about multi-user&#x2F;multi-process access&quot;, speaker admits DuckDB basically doesn&#x27;t do that.<p>Speaker pitches for using embedded in-proc DBs inside AWS Lambda functions. Not practical to install Oracle RDBMS in something that only runs for 100msec.<p>A web shell for DuckDB is demonstrated, it uses WASM.<p>Decentralization is pitched as a reason to avoid 2-tier architecture (separate db engine w&#x2F; client protocol).
评论 #38407827 未加载
评论 #38412448 未加载
评论 #38407496 未加载
0xbadcafebee超过 1 年前
One of the major failures of the modern computer science age (among others) is a lack of direction away from traditional i&#x2F;o. We still are stuck on files and directories and tcp sockets. Yet what we actually want to do with i&#x2F;o is not read a file from a local disk, or connect to a server and transmute the contents of the file over some additional protocol.<p>What we really want is to store some data somewhere, and later be able to retrieve it, without necessarily knowing what it was we stored or where or how. And we don&#x27;t want to think about what server it&#x27;s on, or what hard drive, or what folder. And we don&#x27;t want to think about client protocols or query languages.<p>All of that would be possible if we reinvented i&#x2F;o. Basically, just imagine what you want your experience to be, and then start making up names for functions that do that. Stuff that in a kernel, or a standard library. Now you have i&#x2F;o that&#x27;s based on how you really want to use data. The backend implementation of it can vary, but the point is to make the user experience what we actually want rather than what somebody else thinks is practical. Make the data interface you want to use, and make it a standard.
评论 #38408473 未加载
评论 #38408375 未加载
评论 #38408793 未加载
评论 #38414368 未加载
评论 #38407945 未加载
blibble超过 1 年前
that chart of the &quot;inefficiency of client protocols&quot; tripped my bullshit alarm<p>the paper is here: <a href="https:&#x2F;&#x2F;15721.courses.cs.cmu.edu&#x2F;spring2023&#x2F;papers&#x2F;15-networking&#x2F;p1022-muehleisen.pdf" rel="nofollow noreferrer">https:&#x2F;&#x2F;15721.courses.cs.cmu.edu&#x2F;spring2023&#x2F;papers&#x2F;15-networ...</a><p>it&#x27;s a super-contrived example that&#x27;s not using any of the functionality of the database and is just using it as &quot;cat&quot;<p>basically just doing cat over localhost, well, what a surprise, if you add a layer of serialisation of course it&#x27;s slower that just doing memcpy()<p>if you&#x27;re using your database to store files... maybe don&#x27;t do that
评论 #38406695 未加载
评论 #38408500 未加载
评论 #38414977 未加载
sertbdfgbnfgsd超过 1 年前
Starts by saying &quot;i sent the abstract drunk, then i had to create a talk&quot;, basically admitting he started with the conclusion and then built the argument.
评论 #38409383 未加载