TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: YouTube – how to batch scrape comments and details for 300 videos?

2 pointsby pcfover 1 year ago
If I have a CSV file with links for over 300 YouTube videos, what are some good ways to safely save all comments below each one plus the uploader details?<p>It&#x27;s super important to do it in a way that either a) prevents the Google account and&#x2F;or IP address to be marked as a spammer or something, or b) can be done anonymously without a Google account.<p>There&#x27;s no rush, so the process can be done slow enough to not trigger anything at YouTube&#x2F;Google. It can e.g. be done in batches over some weeks, if that&#x27;s necessary.<p>I&#x27;m not a great programmer, so the simpler&#x2F;more automated, the better.<p>Any and all ideas are welcome. Thanks!

3 comments

Leftiumover 1 year ago
Use: `yt-dlp --write-comments --no-download --batch-file FILE`<p>- FILE is a text file with a list of YouTube id&#x27;s&#x2F;URL&#x27;s<p>- <a href="https:&#x2F;&#x2F;superuser.com&#x2F;a&#x2F;1732443&#x2F;4390" rel="nofollow">https:&#x2F;&#x2F;superuser.com&#x2F;a&#x2F;1732443&#x2F;4390</a><p>- <a href="https:&#x2F;&#x2F;github.com&#x2F;yt-dlp&#x2F;yt-dlp">https:&#x2F;&#x2F;github.com&#x2F;yt-dlp&#x2F;yt-dlp</a>
kevindammover 1 year ago
I would echo the advice of using a test framework. An alternative would be a browser extension, and using that to query the element ids while manually visiting each site.<p>The requirement to make it not tied to a Google account rules out what would be my preferred method of getting these via the YT API.<p>I think there are some open source git repos that already do what you&#x27;re asking (e.g. <a href="https:&#x2F;&#x2F;github.com&#x2F;egbertbouman&#x2F;youtube-comment-downloader">https:&#x2F;&#x2F;github.com&#x2F;egbertbouman&#x2F;youtube-comment-downloader</a>) but I haven&#x27;t personally tried any of these.
austin-cheneyover 1 year ago
Use a browser test automation tool like playwright or puppeteer and go to each page. On each page wait for comments to dynamically appear and then walk the DOM to extract that content and transform it to any format of your choosing.