TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Scraping content issues

4 pointsby andrealmost 18 years ago

2 comments

snorkelalmost 18 years ago
Anyone can legally complain if you copy their content. So they send a C&D letter and you remove whatever offended them, no harm done. <p>If you want to prevent bots from scraping your content then take advantage of the fact that most bots don't do Javascript: in your server code render the content of each page with some simple encoding that makes text unreadable then add a piece of javascript to window.onload() thats decodes and displays the content.
评论 #25499 未加载
评论 #25500 未加载
andrealmost 18 years ago
If a company has no terms of use or any other kind of policy on their site, what are the issues in scraping the content? any way to prevent it?
评论 #25480 未加载