TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: Is spaCy still relevant in the LLM era?

3 pointsby next_xibalba6 months ago
What are the use cases in which spaCy (and other similar libraries) still shines? When/how will LLM improvements obsolete spaCy?

3 comments

dvisca6 months ago
Earlier this year I published a paper [1] that was partly about recognizing place references from text. Various NLP libraries and gpt-3.5-turbo were used in the comparison. The comparison was not the focus of the paper and newer LLMs are probably better, but in the specific case, gpt had a lower precision score than most of the tested NLP libraries and was also a bit more difficult to handle when trying to force machine-readable output.<p>[1] <a href="https:&#x2F;&#x2F;www.mdpi.com&#x2F;1999-5903&#x2F;16&#x2F;3&#x2F;87" rel="nofollow">https:&#x2F;&#x2F;www.mdpi.com&#x2F;1999-5903&#x2F;16&#x2F;3&#x2F;87</a>
segmondy6 months ago
A lot of old stuff are still very relevant, they are much resource efficient, smaller amount of RAM, CPU, performance that&#x27;s much faster. Easily 100x cheaper. I mean, if you have a few things you are trying to sort out, then using LLM to solve is okay, but if you have lots and lots of computation, then it&#x27;s worth going classic.
authorfly6 months ago
I don&#x27;t think there are any more which is a shame, because spaCy was an amazing library and probably the library I most ever enjoyed working with, it truly felt like a craftsmans belt for intelligent text transformation&#x2F;insight. Some things like topic clouds can still be useful for creative work but this is not where spaCy shines.<p>But ChatGPT can derive better insights, doesn&#x27;t need pipelines, doesn&#x27;t need hard coded approaches with their issues. And the (NLTK&#x2F;Stanford parser-like) dependency views are still interesting for linguistic purposes.