TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: What are the best tools for web scraping for non-programmers?

1 pointsby searchersover 7 years ago

1 comment

Ihateusernames2over 7 years ago
Chrome Devtools.<p>That sounds silly, and it is. But really, in many cases the data you want to scrape is already loaded to the page in a useful format. For example, take Politico&#x27;s 2016 election results data: <a href="https:&#x2F;&#x2F;www.politico.com&#x2F;2016-election&#x2F;results&#x2F;map&#x2F;president&#x2F;colorado&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.politico.com&#x2F;2016-election&#x2F;results&#x2F;map&#x2F;president...</a><p>If you go to devtools &gt; network &gt; XHR, you&#x27;ll see the page for Colorado loads a file containing all of the results in a format you should be able to parse. This example is a bit tricky, as parsing it as a non-programmer might still be a bit challenging, but some sites will load data as a CSV.<p>I know this doesn&#x27;t exactly answer your question, but I think it would help non-programmers to know this.