TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: Can we make side projects by web scraping?

2 pointsby tpkahlonover 4 years ago

2 comments

pnwhycover 4 years ago
Yes. So long as you do it morally. Your project must extract novel insights from the scraped content, not steal it. Some of my most successful projects have been a result of web scraping.<p>However, there are caveats. 1. If the scraped website receives frequent updates to its html structure, you will need to update your scraper frequently. This can turn your codeitonceandforgetit side project into a monthly maintenance headache. 2. You are entirely dependent on that site. If they shut their doors, or start blocking bots, you have to find a workaround or shut down your project completely<p>Ultimately, I would recommend it as a means for quickly attracting users, or as automation for part of a larger project. Ensure that the value is &gt; the maintenance.
tpkahlonover 4 years ago
Recently, I created a project (<a href="https:&#x2F;&#x2F;cs1000.vercel.app" rel="nofollow">https:&#x2F;&#x2F;cs1000.vercel.app</a>) which is refined and ad free version of an article. Is it fine to build such side projects via web scraping? Do we need approval from site&#x2F;entity involved?
评论 #25790739 未加载