TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Show HN: An adaptive classifier that detects hallucinations in LLM/RAG outputs

1 点作者 codelion2 个月前

1 comment

codelion2 个月前
I built an open-source hallucination detector that identifies when LLM outputs contain information not present in the source context. The tool is particularly useful for RAG systems where ensuring factual accuracy is critical.<p>Unlike most hallucination detection approaches that require separate LLM calls (which add cost and latency), this is a lightweight classifier built on HuggingFace transformers. It&#x27;s adaptive, meaning it continuously improves as it processes more examples.<p>Technical approach:<p>- Uses a prototype memory system that maintains class examples for quick adaptation<p>- Combines transformer embeddings with an adaptive neural layer<p>- Trained on the RAGTruth benchmark dataset across QA, summarization, and data-to-text tasks<p>- Achieves 80.7% recall overall (51.5% F1), with strongest performance on data-to-text generation<p>Example usage:<p>from adaptive_classifier import AdaptiveClassifier<p># Load pre-trained detector<p>detector = AdaptiveClassifier.from_pretrained(&quot;adaptive-classifier&#x2F;llm-hallucination-detector&quot;)<p># Format input with context, query and response<p>input_text = f&quot;Context: {your_context}\nQuestion: {your_question}\nAnswer: {llm_response}&quot;<p># Get prediction<p>prediction = detector.predict(input_text)<p># Returns: [(&#x27;HALLUCINATED&#x27;, 0.72), (&#x27;NOT_HALLUCINATED&#x27;, 0.28)]<p>Current limitations:<p>- Performance varies by task type (stronger on data-to-text, weaker on summarization precision)<p>- Initial version focuses on binary classification; token-level detection is planned<p>- The model is relatively small, so it won&#x27;t catch subtle nuanced hallucinations that require deep domain knowledge<p>The library&#x27;s wider goal is to enable adaptive classification for use cases where models need to continuously learn from new examples. We&#x27;ve also built LLM routers and configuration optimizers with it.<p>Would love feedback from anyone working on RAG systems or LLM evaluation. What metrics or capabilities would be most useful to you in a hallucination detector?<p>Project: <a href="https:&#x2F;&#x2F;github.com&#x2F;codelion&#x2F;adaptive-classifier">https:&#x2F;&#x2F;github.com&#x2F;codelion&#x2F;adaptive-classifier</a><p>Docs: <a href="https:&#x2F;&#x2F;github.com&#x2F;codelion&#x2F;adaptive-classifier#hallucination-detector">https:&#x2F;&#x2F;github.com&#x2F;codelion&#x2F;adaptive-classifier#hallucinatio...</a>