TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Show HN: Simple Log Alerts to Slack

26 点作者 tradrich大约 2 年前
There are many log alerting systems on the market. The best known is probably Datadog. There’s also Logtail, Papertrail, Splunk, Logstash and others.<p>These are well put together products with a host of great features, such as excellent UIs, sophisticated live searching via web interfaces and sometimes query languages and alerting. They require various levels of installation and they have costs, either through volume-based tiered systems or monthly payments.<p>For a bootstrapped business, this can be problematic, for instance when a surge of logs - indicating a possible important problem that needs to be solved - pushes volume on to another tier. Should the “log ransom” be paid?<p>Instead, I recalled from earlier times surely the simplest log watcher: Swatchdog [1]. It is rather venerable software. Its file history from its source download shows dates in 2015, but it was written much earlier - the 90s or possibly 80s by Todd Atkins [2].<p>We wanted to have alerts in Slack - the blog explains how we did it. In short: *very simply*. The code is available [3].<p>[1]: <a href="https:&#x2F;&#x2F;github.com&#x2F;ToddAtkins&#x2F;swatchdog">https:&#x2F;&#x2F;github.com&#x2F;ToddAtkins&#x2F;swatchdog</a><p>[2]: <a href="https:&#x2F;&#x2F;www.linkedin.com&#x2F;in&#x2F;toddatkins&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.linkedin.com&#x2F;in&#x2F;toddatkins&#x2F;</a><p>[3]: <a href="https:&#x2F;&#x2F;github.com&#x2F;profitviews&#x2F;swatchdog">https:&#x2F;&#x2F;github.com&#x2F;profitviews&#x2F;swatchdog</a>

9 条评论

PuffinBlue大约 2 年前
This looks very interesting. It seems to work on a regex match, so you&#x27;d set what you want to see alerted on and then you receive alerts if that match is made?<p>I like the opposite method - default alert on everything and develop an allowlist that quietens things down you don&#x27;t want to hear. This is great for alerting you to unexpected things. And once in a while you actually want to know about some of those things :-)<p>It may sound very noisy but it&#x27;s not too bad, especially once you&#x27;re allowlist is setup. Logcheck[0] is a good tool for this and it runs by default at 2 minutes past each hour, emailing in a report of everything that isn&#x27;t allowed. I think it matches some regex to what it deems higher threat events and those are always alerted on.<p>I&#x27;ll conceed that this method isn&#x27;t stellar for cattle! And we don&#x27;t bother with it for things like kubernetes clusters or servers with semi-regular turnover for instance.<p>For pets and long lived servers that need looking after it&#x27;s a good tool.<p>[0]<a href="https:&#x2F;&#x2F;logcheck.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;logcheck.org&#x2F;</a>
Jedd大约 2 年前
An interesting approach, especially the hybrid python + perl bundling. Embrace the &#x27;you can&#x27;t please everyone, all of the time&#x27; mantra.<p>At work we&#x27;re using a combination of Splunk, Microsoft Sentinel, and Grafana Loki (can really recommend the latter combined with promtail).<p>There&#x27;s some curious artefacts in this repo - the systemd service file calling usr&#x2F;bin&#x2F;env to a $PYTHON_EXECUTABLE (set in the env file to &#x2F;usr&#x2F;bin&#x2F;python3) but the python script being called starts with &#x27;#!&#x2F;usr&#x2F;bin&#x2F;env python&#x27; (which on my Debian unstable system launches python2).<p>Throwing executables under &#x2F;etc&#x2F; is a .. contentious manoeuvre.<p>And the - so far as I can see - lack of support for handling journald output I suppose is a reflection of this sentiment:<p>&gt; There’s endless criticism of systemd in the Linux community and I expect it will be properly superseded in the next few years. [...] It’s a curiously un-robust system.
slyall大约 2 年前
You could probably also use fluentd to do the same thing. It can process the logs locally and sent alerts instead of forwarding logs to a central source.<p>How to grep the logs for errors and send an email<p><a href="https:&#x2F;&#x2F;docs.fluentd.org&#x2F;how-to-guides&#x2F;splunk-like-grep-and-alert-email" rel="nofollow">https:&#x2F;&#x2F;docs.fluentd.org&#x2F;how-to-guides&#x2F;splunk-like-grep-and-...</a><p>The slack plugin you could use instead of email:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;sowawa&#x2F;fluent-plugin-slack">https:&#x2F;&#x2F;github.com&#x2F;sowawa&#x2F;fluent-plugin-slack</a>
giraffer大约 2 年前
If most of your stack is written in Python you could also consider setting up a custom log handler (with additional filters or formatters) such as <a href="https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;slacker-log-handler&#x2F;" rel="nofollow">https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;slacker-log-handler&#x2F;</a><p>The Python logging framework is quite flexible
NovemberWhiskey大约 2 年前
Hrmm, I&#x27;m not sure I&#x27;d call Datadog or Splunk &quot;log alerting systems&quot;. That&#x27;s one of <i>n</i> capabilities that involves logs in each of those platforms. If you just want to fire an alert based on a match in a log line, you definitely do not need those platforms, I agree.
iroddis大约 2 年前
I really like this, the simplicity is really refreshing.<p>Nothing to do with the article, but it would be nice if Slack (or IOS&#x2F;Android) offered better customization of notifications. Errors and warnings are indistinguishable from regular messages on mobile, which means they often get lost in the shuffle.
yabones大约 2 年前
Very nice. I think a lot of admins jump straight to the most complicated Elasticsearch&#x2F;Datadog&#x2F;NewRelic stack and don&#x27;t even take a second to build something simpler.<p>For &quot;unfussy&quot; machines, I like to just set up rsyslog to send high priority syslogs right to email. [1]
评论 #35048693 未加载
jesterson大约 2 年前
Not trying to break your party, but GrayLog2 solves all those problems. Most businesses&#x2F;startups do not need well marketed DataDog or Logtail.<p>Free, fast, versatile, adaptable to almost every case
评论 #35066736 未加载
troysk大约 2 年前
Why use Slack? Slack has rate limits and one will not get the notifications when something real goes down as there will be 100s of messages overloading Slack API.
评论 #35055586 未加载