TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Stuff Your Logs

118 点作者 r4um大约 4 年前

4 条评论

ChrisFoster大约 4 年前
This is really neat. In the past I&#x27;ve used similar techniques to decode binary data from a third party lidar system in parallel. In a way that the manufacturers probably didn&#x27;t intend or expect.<p>The system generated large data files which we wanted to process in parallel without any pre-indexing. It turned out that these streams contained sync markers which were &quot;unlikely&quot; to occur in the real data, but there wasn&#x27;t any precise framing like COBS. Regardless, the markers and certain patterns in the binary headers were enough to synchronize with the stream with a very high degree of reliability.<p>So for parallel processing we&#x27;d seek into the middle of the file to process a chunk of data, synchronize with the stream, and process all subsequent lidar scanlines which started in that chunk. Exactly the algorithm they describe here.<p>Amusingly this approach gave reasonable results even in the presence of significant corruption where the manufacturer&#x27;s software would give up.
c0l0大约 4 年前
Having skimmed the article (because $dayjob and all), I wonder how&#x2F;if their scheme can cope with write(2) producing a short write, with not all the data in the buffer being atomically committed to their POSIX-compliant backing store?<p>I don&#x27;t see any mechanism described that makes sure that never happens (by forcing records not to exceed a given length that can always be written atomically - which I am not sure even exists...), so I am wondering how often that kind of thing even happens on contemporary systems, and - if it does - how often they wreck a good number of stored records that way.
评论 #26511108 未加载
评论 #26513961 未加载
评论 #26511076 未加载
评论 #26518024 未加载
codetrotter大约 4 年前
This is a wonderful article! This is exactly what I need, I can’t wait to implement this! Thank you both author and person that posted this ^^
评论 #26513569 未加载
mef大约 4 年前
site appears to be down... archive link <a href="https:&#x2F;&#x2F;archive.ph&#x2F;https:&#x2F;&#x2F;pvk.ca&#x2F;Blog&#x2F;2021&#x2F;01&#x2F;11&#x2F;stuff-your-logs&#x2F;" rel="nofollow">https:&#x2F;&#x2F;archive.ph&#x2F;https:&#x2F;&#x2F;pvk.ca&#x2F;Blog&#x2F;2021&#x2F;01&#x2F;11&#x2F;stuff-your...</a>
评论 #26515687 未加载