TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

A review of research trials finds they are rigged to exaggerate AI performance

6 点作者 stuartbman大约 4 年前

1 comment

alisonkisk大约 4 年前
Libelous editorialized title violates HN guidelines.<p>Article actually says<p>-----<p>There were 81 papers in total, of which only 9 were prospective studies. All of the studies were in the area of medical imaging.<p>The study design often meant AI performance was exaggerated. For example, the human comparison group averaged only 4 doctors (too small!). Often expert and non-expert performance was combined together — making the expert group perform worse on average.<p>The reporting quality of the papers was typically pretty low, with low scores on the CONSORT and TRIPOD checklists. Papers often made claims of being better than clinicians in their abstract, without including the appropriate caveats.<p>Most of the studies didn&#x27;t have their code or data publicly available. This raises questions around whether the research can be reproduced by other scientists - a key principle of science.