TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Show HN: Copy from URL – A tiny tool to bypass robots.txt for AI chatbot

2 pointsby ngkw11 months ago
Hello HN! I&#x27;d like to share a very small project I&#x27;ve built called &quot;Copy from URL&quot; (<a href="https:&#x2F;&#x2F;copyurl.io" rel="nofollow">https:&#x2F;&#x2F;copyurl.io</a>). It&#x27;s a minimal tool designed to solve a common frustration when working with AI chatbots like ChatGPT or Claude.<p>The Problem: Many of us need to copy website content for AI queries. However, popular sites often use robots.txt to block AI crawlers, resulting in errors when trying to use these sites with AI. The Solution: Copy from URL is a simple, no-frills web tool that lets you extract text content from any webpage, bypassing robots.txt restrictions. Here&#x27;s all it does:<p>- You enter a URL - Click &quot;Go!&quot; - Get clean, copyable text content - Use it in your AI interactions<p>Key Features:<p>- No registration - Open-source (GitHub link in comments) - Completely free - Bare-bones functionality<p>Tech Stack: Built with minimal dependencies using Next.js, React, and Tailwind CSS. Uses Cheerio for HTML parsing. Privacy &amp; Security: We don&#x27;t collect or store any data. Everything happens client-side. This is a very small project born out of personal necessity. It&#x27;s not feature-rich or complex, just a simple tool that does one thing. I thought others might find it useful too. I&#x27;d love to hear your thoughts or suggestions, especially if you&#x27;ve encountered similar issues with AI interactions!

3 comments

ngkw11 months ago
Github codes: <a href="https:&#x2F;&#x2F;github.com&#x2F;Amunzen&#x2F;CopyFromURL">https:&#x2F;&#x2F;github.com&#x2F;Amunzen&#x2F;CopyFromURL</a><p>Any comment is welcome! Thanks.
ngkw11 months ago
haha I have mistaken the URL in the comments. Here&#x27;s the right one: <a href="https:&#x2F;&#x2F;copyfromurl.com" rel="nofollow">https:&#x2F;&#x2F;copyfromurl.com</a>
JSDevOps11 months ago
Or you know you can just ignore robots.txt like everyone else. What do you think robots.txt does exactly?
评论 #40861708 未加载
评论 #40861718 未加载