TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

RaptorQ and performance optimization in Rust

127 点作者 raccoonone大约 6 年前

10 条评论

lower大约 6 年前
When comparing a Rust rewrite of a C++ codebase, I noticed that Rust's default HashMap and HashSet were noticably slower than unordered_map and unordered_set. The reason is that Rust uses a hash function that has better properties, but that is a bit slower. If the performance of a HashMap is a bottleneck, one can use a simpler hash function to get the same performance as in C++. With FnvHashMap, the performance was the same as unordered_map.
评论 #19532040 未加载
评论 #19532941 未加载
评论 #19534511 未加载
Luker88大约 6 年前
I have played with a C++ implementation of RFC6330, and while I have to say I never went for performance, I find this benchmark a bit... pointless.<p>RaptorQ is kind of a gaussian elimination of a matrix, so it all depends on the block size (=&gt;matrix size). The algorithm has basically cubic complexity on the number of symbols in a block. RFC6330 is made to work on files, which are divided into blocks with a certain number of symbols, and the bytes are interleaved.<p>This implementation does not do the (complex and almost pointless) interleaving, which is fine, even OpenRQ does not.<p>The bench seems to be done on a.... 10kb file? It all fits in the L2. We are not given the symbol size (which determines the block size!) and I assume all of this fits in a 10x10 matrix.<p>You are benchmarking operations on a matrix that is (more or less) a 10x10 byte matrix.<p>The biggest part of this benchmark might almost be the generation of repair symbols (was it even done?), since that would require multiple xoring of the above-mentioned symbols.<p>This is much closer to micro-benchmarking than an actual benchmark, imho. It would have been more interesting to see what happens with files at least larger than the L3 cache.<p>You can also cache intermediate results, (which he does not do) which is especially useful for encoding, but only when working on matrix &gt;= 100x100, otherwise just searching the cache, getting from memory (my implementation optionally did LZ4 compression&#x2F;decompression) and doing a matrix multiplication is slower than just computing the matrix again.<p>Still, it&#x27;s nice to see implementations of the RFC, which is a real pain to read...
sudhirj大约 6 年前
Does anyone know of a good (preferably pure) Go implementation? <a href="https:&#x2F;&#x2F;github.com&#x2F;google&#x2F;gofountain" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;google&#x2F;gofountain</a> has mad a good start and implemented the older plain Raptor standard, but I haven&#x27;t been able to find a good Go library.<p>If anyone is qualified enough to take a crack at it I can try and arrange for sponsorship as well.
nullc大约 6 年前
RaptorQ, even in this implementation, appears to be a lot slower than wirehair: <a href="https:&#x2F;&#x2F;github.com&#x2F;catid&#x2F;wirehair" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;catid&#x2F;wirehair</a>
评论 #19534545 未加载
mike00632大约 6 年前
For anyone interested in how this code works, here is Qualcomm&#x27;s technical overview of RaptorQ: <a href="https:&#x2F;&#x2F;www.qualcomm.com&#x2F;media&#x2F;documents&#x2F;files&#x2F;raptorq-technical-overview.pdf" rel="nofollow">https:&#x2F;&#x2F;www.qualcomm.com&#x2F;media&#x2F;documents&#x2F;files&#x2F;raptorq-techn...</a>
eridius大约 6 年前
I don&#x27;t know much about SIMD but are there no alignment concerns with reinterpreting a slice as an AVX vector?
评论 #19532453 未加载
tgtweak大约 6 年前
Very cool efforts here. Curious what the speed is without those instructions present on the cpu, and what hardware this was run on.<p>The concept of a fountain seems interesting but what is a good use case? Variable strength error correction?
评论 #19531616 未加载
评论 #19531417 未加载
minimax大约 6 年前
Can anyone suggest a good book that would serve as an introduction to finite field arithmetic? I keep randomly running into it (e.g. this post), but don&#x27;t understand it well enough to follow the discussion.
评论 #19531740 未加载
评论 #19530991 未加载
empath75大约 6 年前
Anybody want to comment on whether all of his unsafe code was actually necessary? Seems bad for rust that safe code is 25x slower.
评论 #19531106 未加载
评论 #19531845 未加载
评论 #19531131 未加载
评论 #19530969 未加载
评论 #19531056 未加载
layoutIfNeeded大约 6 年前
What is the patent situation with Raptor codes nowadays?
评论 #19531571 未加载
评论 #19531499 未加载