TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Using NLP on raw HTML from scrapers More efficient than training them directly?

2 点作者 robert1er超过 2 年前
I have used rudimentary scrapers for years but always found that the process of training them doesn’t scale as well as I expect. In a scenario where one needs hundreds of scrapers all grabbing the same five or 10 pieces of information (dates, names, addresses, and ID numbers) is it possible to just grab the raw HTML and train an NLP model to find the relevant data rather than selecting specific selectors&#x2F;paths?<p>It seems to me that finding these items in an HTML doc would be a trivial task for NLP and this would allow us to forgo scraper maintenance in the event that websites change. Assuming that I already have hundreds of thousands or millions of records in structured data that could be used for training, would modern tools make this a more efficient way of ingesting data?

1 comment

PaulHoule超过 2 年前
It’s worth trying.
评论 #34762663 未加载