TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Purely Functional Sliding Window Aggregation Algorithm

70 pointsby agnishom3 months ago

7 comments

mlochbaum3 months ago
The queue method is popular, but there&#x27;s a much faster (branch-free) and in my opinion simpler way, known as the van Herk&#x2F;Gil-Werman algorithm in image processing. It splits the input into windows and pairs up a backward scan on one window with a forward scan on the next. This works for any associative function. I was very surprised when I learned about it that it&#x27;s not taught more often (the name&#x27;s not doing it any favors)! And I wrote a tutorial page on it for my SIMD-oriented language, mostly about vectorizing it which I didn&#x27;t quite finish writing up, but with what I think is a reasonable presentation in the first part: <a href="https:&#x2F;&#x2F;github.com&#x2F;mlochbaum&#x2F;Singeli&#x2F;blob&#x2F;master&#x2F;doc&#x2F;minfilter.md">https:&#x2F;&#x2F;github.com&#x2F;mlochbaum&#x2F;Singeli&#x2F;blob&#x2F;master&#x2F;doc&#x2F;minfilt...</a><p>I also found an interesting streaming version here recently: <a href="https:&#x2F;&#x2F;signalsmith-audio.co.uk&#x2F;writing&#x2F;2022&#x2F;constant-time-peak-hold&#x2F;" rel="nofollow">https:&#x2F;&#x2F;signalsmith-audio.co.uk&#x2F;writing&#x2F;2022&#x2F;constant-time-p...</a><p>EDIT: On closer inspection, this method is equivalent to the one I described, and not the one I&#x27;m used to seeing with queues (that starts my tutorial). The stack-reversing step is what forms a backwards scan. The combination of turning it sequential by taking in one element at a time but then expressing this in functional programming makes for a complicated presentation, I think.
farazbabar3 months ago
This is similar to an approach I use but instead of a queue, I accomplish this using a ring buffer that wraps around and overwrites entries older than window size. We maintain a global window aggregate, subtract ring buffer slot aggregate for entries dropping out and accumulate new entries into new slot aggregate while adding it to the global aggregate. Everything is o(1) including reads, which just returns the global window aggregate.
评论 #43161157 未加载
评论 #43161543 未加载
codebje3 months ago
That was a well written and easily approachable blog post on what I found to be an interesting topic. Aside from the topic itself, I think I also learned a bit about structuring technical articles.
agnishom3 months ago
This is a very interesting algorithm which is more or less known in the folklore, but is still relatively obscure. I have used it as a part of temporal logic monitoring procedure: <a href="https:&#x2F;&#x2F;github.com&#x2F;Agnishom&#x2F;lattice-mtl&#x2F;blob&#x2F;master&#x2F;src&#x2F;Monitor&#x2F;AggQueue.v">https:&#x2F;&#x2F;github.com&#x2F;Agnishom&#x2F;lattice-mtl&#x2F;blob&#x2F;master&#x2F;src&#x2F;Moni...</a>
评论 #43155988 未加载
GistNoesis3 months ago
I have made a quick c++ implementation for those unfamiliar with Haskell :<p><a href="https:&#x2F;&#x2F;gist.github.com&#x2F;unrealwill&#x2F;5ca4db9beefafaa212465277b6918779" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;unrealwill&#x2F;5ca4db9beefafaa212465277b...</a>
brianberns3 months ago
I translated this to F# for my own edification. It&#x27;s more verbose, but perhaps easier to understand for non-Haskellers.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;brianberns&#x2F;AnnotatedStack">https:&#x2F;&#x2F;github.com&#x2F;brianberns&#x2F;AnnotatedStack</a>
belter3 months ago
Competitive Programming in Haskell...I can only define this as Masochistic Aesthetics...
评论 #43160508 未加载
评论 #43160717 未加载
评论 #43159622 未加载