TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: What's state of the art for screen scraping these days?

9 pointsby nateover 5 years ago
Any recommendations for super user friendly tools out there for screen scraping? Mostly looking for hosted services that have the experience dialed in.

3 comments

teerayover 5 years ago
If you like Go, I’ve had good success on a few projects with <a href="http:&#x2F;&#x2F;go-colly.org&#x2F;" rel="nofollow">http:&#x2F;&#x2F;go-colly.org&#x2F;</a> . It’s significantly easier than using the <a href="https:&#x2F;&#x2F;godoc.org&#x2F;golang.org&#x2F;x&#x2F;net&#x2F;html" rel="nofollow">https:&#x2F;&#x2F;godoc.org&#x2F;golang.org&#x2F;x&#x2F;net&#x2F;html</a> package directly where you would otherwise have to juggle cookies and whatnot to perform authentication tasks.
dhruvkarover 5 years ago
kimono was pretty good. these guys are trying to build a replacement:<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=21417965" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=21417965</a>
girishsoover 5 years ago
Any suggestions for scraping SPAs?