TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Outlier Detection in Time-Series Signals using FFT and Median Filtering

6 点作者 ivoflipse大约 11 年前

2 条评论

kastnerkyle大约 11 年前
Nice, simple explanation of this concept. +1000 for using IPython notebook, though the left-right layout of the cells is a little hard to follow. Shamelessly, I prefer my own layout based on pelican-bootstrap: <a href="http://kastnerkyle.github.io/blog/2014/02/15/polyphase-signal-processing/" rel="nofollow">http:&#x2F;&#x2F;kastnerkyle.github.io&#x2F;blog&#x2F;2014&#x2F;02&#x2F;15&#x2F;polyphase-signa...</a> - but that moving swarm logo is pretty great!<p>However, I have to say that a high-pass filter (in this case, basically a brickwall applied using an FFT? might not be a great idea due to &quot;ringing&quot; in the time-domain though it works here...) is typically <i>NOT</i> the right answer for real datasets. Think of detecting outliers on a square wave, or a pinched sine wave, or any &quot;non-pure&quot; tone, really... Non-gaussian noise&#x2F;interference is also way more common than you would be lead to believe :). Onset detection is a complex problem, but an easy to understand solution is a tunable lag-lead moving average filter.<p>One small window calculates the moving average, while a larger window also calculates the moving average. For &quot;noise&quot; or typical data, the averages should be similar given that both windows encompass more than a few cycles of the signal. When the small window average is &gt; than the large window average by some margin, an event has just occurred. The two windows should then get to about the same average for the event duration (in this case, there is not really much duration). Once the small window average is &lt; the large window average by some margin, then the signal event is over. There are better approaches from music information retrieval which use phase information as well...<p>The hard part is tuning the &quot;above and below&quot; thresholds and two window sizes from the signal you wish to catch - this approach is used as a simple alternative to wavelets for earthquake detection.<p>Great illustration, and if this technique works for your data that is awesome! Unfortunately, I have never found a dataset with such an easy solution for outlier detection...
lutusp大约 11 年前
It&#x27;s a shame that the linked article, which uses the new IPython blog-authoring features, isn&#x27;t identified as such. It&#x27;s a nice example of IPython&#x27;s interactive authoring abilities that mix normal text with technical content and graphics created using the IPython web interface.<p><a href="http://blog.fperez.org/2012/09/blogging-with-ipython-notebook.html" rel="nofollow">http:&#x2F;&#x2F;blog.fperez.org&#x2F;2012&#x2F;09&#x2F;blogging-with-ipython-noteboo...</a>