TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Surviving the front page of Hacker News on a 50 Mbps uplink

261 点作者 hddherman超过 3 年前

28 条评论

invalidator超过 3 年前
I haven&#x27;t been hugged by HN yet, but I&#x27;ve survived being featured on a couple other front pages long ago.<p>It&#x27;s easy if it&#x27;s a lightweight page and you have something faster than a 486. If it&#x27;s heavy (lots of images, videos, downloads, etc) you can run out of bandwidth pretty quickly. However, that doesn&#x27;t necessarily sink the ship!<p>Traffic shaping makes a big difference. Without it, the modem buffer quickly fills causing multi-second ping times. This easily leads to congestion collapse. TCP grows the receive window to large values, and the congestion avoidance algorithm doesn&#x27;t do well when the feedback is delayed due to the latency. TCP starts sawtoothing, and you end up with occasional moments where a few pages get served, interspersed with so much loss that connections get dropped.<p>We used htb [1] which tightly controls the outbound traffic rate, preferring to drop packets instead of delay them. That prevents the modem&#x27;s buffer from filling up, which keeps the latency down. This creates a smoother flow. Pages load slowly, but at least they keep loading, instead of wasting bandwidth on retries. It also prioritizes traffic: other services are guaranteed their minimum rates. We were using ssh through the line and didn&#x27;t even realize we were getting hit until someone asked why the site was running slowly.<p>Newer algorithms are even better, especially for keeping jitter down, but htb is simple and vastly better than letting your router blast packets at full speed. OpenWRT and all the other open source router firmwares can do it these days. It improves things for even normal end-user uses, but if you&#x27;re running a server at home, you really want to set up some QOS before someone hugs you to death.<p>[1] <a href="https:&#x2F;&#x2F;linux.die.net&#x2F;man&#x2F;8&#x2F;tc-htb" rel="nofollow">https:&#x2F;&#x2F;linux.die.net&#x2F;man&#x2F;8&#x2F;tc-htb</a>
评论 #30270721 未加载
评论 #30275060 未加载
marginalia_nu超过 3 年前
I&#x27;ve been hugged twice on 100 mbit domestic broadband, and it&#x27;s honestly not been all that bad.<p>In general I&#x27;ve seen somewhere between 1-5 visitors per second, a couple of million over a day. I think keeping the number of requests per visit down is a big key to surviving this traffic with low resources.<p>The first hug was search.marginalia.nu, which used to load fonts and had a load of about 50 Kb&#x2F;visit; but still only generated in maybe 4-5 GETs per visit. I measured about 2 Mbps.<p>The second time was search.marginalia.nu&#x2F;browse&#x2F;random, which loads 75 images at a resolution of 512x348 (mostly WebP; so anywhere between 2 and 30 Kb). These were randomly picked from a very large pool, so they can&#x27;t be meaningfully cached. I presciently set them to lazy loading and I think that made a big difference, especially from mobile visitors as they&#x27;d only load a few at a time. I didn&#x27;t measure the bandwidth usage at this point, but it stayed up and stayed responsive throughout the day.<p>I think something that possibly also makes a difference is that I don&#x27;t do any sort of session cookies or mutable non-ephemeral state in my backend. I can only imagine having to validate sessions and manage access&#x2F;roles for every request would make for a lot more work.<p>I don&#x27;t think it&#x27;s necessarily the bandwidth that&#x27;s going to be the bottleneck, rather than something like contested locks in the database, resulting in back pressure in turn causing traffic jam of lingering connections. Reducing mutable server state helps a lot with that.
评论 #30271565 未加载
评论 #30270639 未加载
评论 #30272032 未加载
wink超过 3 年前
Author named ~70k hits per day. If we calculate 72000&#x2F;12 that&#x27;s 6000 - which is 1.6 req&#x2F;s. Why are we talking about &quot;surviving&quot; anything here?<p>Here some numbers from a Wordpress installation on a 3 EUR Scaleway VPS where zero time was spent optimizing anything:<p><pre><code> $ siege -c 3 -r 10 https:&#x2F;&#x2F;example.org ** SIEGE 4.0.4 ** Preparing 3 concurrent users for battle. The server is now under siege... Transactions: 360 hits Availability: 100.00 % Elapsed time: 13.89 secs Data transferred: 24.65 MB Response time: 0.11 secs Transaction rate: 25.92 trans&#x2F;sec</code></pre>
评论 #30271288 未加载
评论 #30271817 未加载
评论 #30273038 未加载
moreentropy超过 3 年前
I&#x27;m in the same situation (50 Mbps uplink at one place, 100Mbps at another) and that&#x27;s enough to do all hosting for hobby projects at home, which I really love.<p>Instead of exposing my home directly using DynDNS, I got a <i>really</i> cheap low end server (currently a VM w&#x2F; 1 CPU, 512MB RAM and 400mbps traffic flat) for 1€&#x2F;month that proxies all traffic to the (hidden) servers hosted at home.<p>The spec is enough to reliably run a HAproxy that can max out the available bandwidth w&#x2F;o sweat and it allows me to do failover between servers in my two &quot;home datacenters&quot; + possibly cache assets.
评论 #30272001 未加载
评论 #30271840 未加载
评论 #30270669 未加载
评论 #30270401 未加载
hddherman超过 3 年前
Wayback Machine link, just in case: <a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20220209050643&#x2F;https:&#x2F;&#x2F;ounapuu.ee&#x2F;posts&#x2F;2022&#x2F;02&#x2F;09&#x2F;hn-stats-analytics&#x2F;" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20220209050643&#x2F;https:&#x2F;&#x2F;ounapuu.e...</a>
tyingq超过 3 年前
Having correct cache headers helps as well, as a single visitor probably comes back to the page fairly often, or visits another page with shared assets. It&#x27;s surprising how often some things that should be cached come back with headers that make it uncacheable by the browser. KeyCDN has a pretty comprehensive guide: <a href="https:&#x2F;&#x2F;www.keycdn.com&#x2F;blog&#x2F;http-cache-headers" rel="nofollow">https:&#x2F;&#x2F;www.keycdn.com&#x2F;blog&#x2F;http-cache-headers</a>
calmingsolitude超过 3 年前
I&#x27;ve had the experience of being on the opposite end of the spectrum when I made a joke site[0] that kept the connection open to every connected browser. Needless to say, it got hugged to death in the very first ten minutes.<p>What got it back up was switching from my homebrew webserver to Nginx and increasing the file descriptor limits. Switching to a battle-tested webserver made all the difference. I&#x27;m still not sure how many connections it can keep open now, but I don&#x27;t have to worry about the static parts of the site going down.<p>[0] <a href="https:&#x2F;&#x2F;youcantdownloadthisimage.online&#x2F;" rel="nofollow">https:&#x2F;&#x2F;youcantdownloadthisimage.online&#x2F;</a>
评论 #30270321 未加载
评论 #30270141 未加载
评论 #30270736 未加载
timw4mail超过 3 年前
I wish I had 50Mbps uplink. My web server (<a href="https:&#x2F;&#x2F;timshome.page" rel="nofollow">https:&#x2F;&#x2F;timshome.page</a>) runs on a ~15Mbps uplink. The hardware is still overpowered for it&#x27;s usage, though, with a Ryzen 9 3900x (the server gets my gaming rig hand-me-downs) and 16GB of ram.<p>In my case, the limited bandwidth probably protects my server in some respect.
zuzun超过 3 年前
You can get the web fonts below 20 KB each if you subset them. You can also check if they are present on the system with `local()` before `url()` in the font-face declaration. You could also give the Apple system fonts a higher priority than Inter, because Inter and San Francisco are pretty similar.
bawolff超过 3 年前
I wonder how much bandwidth a hn hug normally is. 50 mbps sounds like a lot to me unless you have heavy images&#x2F;movies&#x2F;not text, but i&#x27;m out of my area of expertise.
评论 #30270102 未加载
kragen超过 3 年前
A million hits from 75k views, seems pretty reasonable.<p>Taking as an example page <a href="http:&#x2F;&#x2F;canonical.org&#x2F;~kragen&#x2F;memory-models&#x2F;" rel="nofollow">http:&#x2F;&#x2F;canonical.org&#x2F;~kragen&#x2F;memory-models&#x2F;</a>, the last thing of mine that hit the front page, that&#x27;s 248.7 KiB, including all the images and other liabilities, so if that were distributed over a single day, that would be about 18 GiB, or 1.8 megabits per second, maybe with a peak of a few times that. The numbers provided here seem pretty comparable, maybe a factor of 2 or 3 bigger.<p>Things can vary by an order of magnitude or more from this. The HTML of that page is 17 KiB, so bandwidth could be an order lower, and as Õunapuu points out, it&#x27;s also easy to drop in a few multi-megabyte photos or videos that bulk up a page to 25 or 25 megabytes instead of 0.25. <a href="http:&#x2F;&#x2F;canonical.org&#x2F;~kragen&#x2F;larvacut.webm" rel="nofollow">http:&#x2F;&#x2F;canonical.org&#x2F;~kragen&#x2F;larvacut.webm</a> is 640×480 × 13 seconds and encodes to 1.8 megabytes.
tsak超过 3 年前
I&#x27;m using the exact same setup as yourself. I have a script that runs every 10 minutes that checks if my external IP has changed and then updates my DNS records via the GoDaddy API.<p>I also decided to use Hugo to generate my blog (<a href="https:&#x2F;&#x2F;tsak.dev" rel="nofollow">https:&#x2F;&#x2F;tsak.dev</a>)<p>And I use `goaccess` to check my logs as well.
评论 #30271674 未加载
jfkimmes超过 3 年前
Quick Tip for your &lt;figure&gt; inconvenience with markdown: Just write a script that copies whatever is in your clipboard to a .&#x2F;images&#x2F; directory and replace the content of your clipboard with the markdown you usually write pointing to that image. See my hacky version, here [1]. I have that on a keyboard shortcut but you can also just alias&#x2F;source it and run it when needed.<p>[1]: <a href="https:&#x2F;&#x2F;gitlab.com&#x2F;jfkimmes&#x2F;dotfiles&#x2F;-&#x2F;blob&#x2F;master&#x2F;scripts&#x2F;markdown_image_include.sh" rel="nofollow">https:&#x2F;&#x2F;gitlab.com&#x2F;jfkimmes&#x2F;dotfiles&#x2F;-&#x2F;blob&#x2F;master&#x2F;scripts&#x2F;m...</a> (this requires an `activate` command that symlinks my current project directory to a path in my home directory so that all my scripts can work there. You can probably make this more elegant but it works for me.)
thomasfromcdnjs超过 3 年前
Sidenote: You pretty much generated traffic analytics anyway, and nginx logs have referrer information if you want to see where the traffic is coming from.
asiekierka超过 3 年前
Regarding image compression, have you considered using jpegrescan on top of mogrify resizing? In my experience, it can shave off a few percent losslessly.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;kud&#x2F;jpegrescan" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;kud&#x2F;jpegrescan</a>
at0mic22超过 3 年前
If you struggle with bandwidth, why not serving gzipped content? By response headers it looks like you are serving plain html.
评论 #30271630 未加载
评论 #30270622 未加载
adam-a超过 3 年前
I did run a few semi-private sites from my home but I soon found the incoming traffic was causing my outbound connection to fall over. Lots of dropped video calls and spotty vpn connections. I assume because my ISP provided router was not handling the incoming requests very well. I wonder if anyone here has any good advice to get this sort of setup to run well?
评论 #30270624 未加载
miyuru超过 3 年前
Any story on why the blog does not support IPv6? Your ISP seems to support it.<p><a href="https:&#x2F;&#x2F;stats.labs.apnic.net&#x2F;ipv6&#x2F;AS2586" rel="nofollow">https:&#x2F;&#x2F;stats.labs.apnic.net&#x2F;ipv6&#x2F;AS2586</a><p>I am asking because you did not even mention it in the &quot;Residential connections and DNS&quot; section. Depending on the ISP you might get a static IPv6 prefix.
评论 #30270147 未加载
skilled超过 3 年前
Ok, what is the color theme you&#x27;re using for the terminal?
评论 #30269900 未加载
评论 #30269792 未加载
评论 #30269808 未加载
ComputerGuru超过 3 年前
This is a joke, right? We colo at a location where the standard is 100mbit and the websites hosted there are doing just fine. Just slim down your pages and stop generating 25MiB landing page bundles.
charcircuit超过 3 年前
There didn&#x27;t seem to be any data in the article analyzing how much bandwidth was being used under load. I&#x27;m not sure if for example compressing those images made it possible to now survive.
评论 #30269824 未加载
louwrentius超过 3 年前
I’ve had a few visits on my solar powered Pi3b+ on a 50 mbit uplink and because it’s a static site both pi and my internet had no issues whatsoever (looking at Grafana).
评论 #30277294 未加载
sT370ma2超过 3 年前
I&#x27;ve survived the front page of Hacker News more than once with an uplink speed that averages only 0.75 Mbps. You just have to know how to do it.
rufname超过 3 年前
Since it appears to be a statically generated site it might not matter, but do you use any caching in your NGINX config?
评论 #30270172 未加载
apoland超过 3 年前
When I began hosting web sites around 2002 we started with a 1.5mbit leased line. It&#x27;s amazing how many requests we handled with what is now considered so little bandwidth. Of course the media attached to web sites has grown larger since then.
notyourday超过 3 年前
The problem is pretty much never the bandwidth. It is nearly always the boneheaded way nearly static content is generated.<p>If you don&#x27;t bake your pages into static files, just throw Varnsh in front of your site and make dynamically generated static content static.
评论 #30271688 未加载
zild3d超过 3 年前
not mentioned in the article, but since its a static blog the answer in how to survive a hug of death is: stick a CDN in front of it
评论 #30271835 未加载
Havoc超过 3 年前
Bit confused as to why set up a static website and then host it from home? Could have just used one of the CDNs?
评论 #30276040 未加载