TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

My Browser WASM't Prepared for This. Using DuckDB, Apache Arrow and Web Workers

126 点作者 jjp大约 1 个月前

7 条评论

mcraiha大约 1 个月前
Tip for all the blog authors, do NOT post code as image. Specially do not add fake editor UI and drop shadow to the image.<p>In this case 25 lines of code is 50 kB of image binary.<p>Also it cannot be searched via search engine. Nor can it be read with screen reader.
评论 #43601150 未加载
评论 #43601049 未加载
评论 #43614878 未加载
评论 #43601887 未加载
评论 #43602256 未加载
评论 #43607498 未加载
评论 #43615976 未加载
评论 #43601521 未加载
jasmcole大约 1 个月前
We use the WASM build of DuckDB quite extensively at Count (<a href="https:&#x2F;&#x2F;count.co" rel="nofollow">https:&#x2F;&#x2F;count.co</a> - 2-3m queries per month). There are a couple of bugs we&#x27;ve noticed, but given that it&#x27;s pretty much maintained by a single person seems impressively reliable!
评论 #43602548 未加载
pmm大约 1 个月前
Author here. Thank you all for the comments. I take full responsibility for stupidly using an image for posting the code snippet. Sorry for that! Also, the article was originally posted almost 2 years ago (and &quot;resurrected&quot; with the recent migration to Medium). This is why a fairly old DuckDB version is referenced there. Some of the issues I observed are now gone too.<p>Obviously, many things have changed since then. We&#x27;ve experimented extensively and moved back and forth with using DuckDB for our internal cloud processing architecture. We eventually settled on just using it for reading the data and then handling everything else in custom workers. Even using TypeScript, we achieved close to 1M events&#x2F;s per worker overall with very high scalability. However, our use-case is quite distinct. We use a custom query engine (for sequence processing), which has driven many design decisions.<p>Overall, I think DuckDB (both vanilla and WASM version) is absolutely phenomenal. It also matured since my original blog post. I believe we&#x27;ll only see more and more projects using it as their backbone. For example, MotherDuck is doing some amazing things with it (e.g., <a href="https:&#x2F;&#x2F;duckdb.org&#x2F;2023&#x2F;03&#x2F;12&#x2F;duckdb-ui" rel="nofollow">https:&#x2F;&#x2F;duckdb.org&#x2F;2023&#x2F;03&#x2F;12&#x2F;duckdb-ui</a>) but there are also many more exciting initiatives.
azakai大约 1 个月前
&gt; [wasm] is executed in a stack-based virtual machine rather than as a native library code.<p>Wasm&#x27;s binary format is indeed a stack-based virtual machine, but that is not how it is executed. Optimizing VMs convert it to SSA form, basic blocks, and finally machine code, much the same as clang or gcc compile native library code.<p>It is true that wasm has some overhead, but that is due to portability and sandboxing, not the stack-based binary format.<p>&gt; On top of the above, memory available to WASM is limited by the browser (in case of Chrome, the limit is currently set at 4GB per tab).<p>wasm64 solves this, by allowing 64-bit pointers and a lot more than 4GB of memory.<p>The feature is already supported in Chrome and Firefox, but not everywhere else yet.
评论 #43608202 未加载
tobilg大约 1 个月前
Not sure why the comparisons were made with pretty outdated versions to be honest.<p>I‘m using a (older) v1.29.1 dev version with <a href="https:&#x2F;&#x2F;sql-workbench.com" rel="nofollow">https:&#x2F;&#x2F;sql-workbench.com</a> w&#x2F;o any bigger issues.
__mp大约 1 个月前
I’ve been toying with the idea of implementing a distributed analytics engine on top of Cloudflare workers and DuckDB.<p>I’m not sure if this goes against the CloudFlare TOS tough (last time I checked they had some provisons against processing images).
评论 #43600908 未加载
评论 #43601283 未加载
评论 #43602687 未加载
评论 #43604191 未加载
canadiantim大约 1 个月前
How persistent can you make this data?