TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Goodhart’s Law: Are Academic Metrics Being Gamed?

71 pointsby ubacover 5 years ago

10 comments

acganover 5 years ago
Glad to see this important piece here (disclosure: I am one of the editors of The Gradient).<p><a href="https:&#x2F;&#x2F;ieeexplore.ieee.org&#x2F;document&#x2F;5089308" rel="nofollow">https:&#x2F;&#x2F;ieeexplore.ieee.org&#x2F;document&#x2F;5089308</a>, from RCIS 2009 (Beel and Gipp) noted that &quot;Google Scholar seems to be more suitable for searching standard literature than for gems or articles by authors advancing a view different from the mainstream.&quot;<p>Unrelated, but interesting: scraping Google Scholar is remarkably annoying if you want to actually use the data. The easiest way (in my experience) seems to be regex hacking on the BibTeX files, but this seems truly broken.
评论 #21073936 未加载
评论 #21076786 未加载
radioactivistover 5 years ago
Something seems a bit wrong with the graph &quot;Publication rate by career length&quot; -- should the y-axis be &quot;Average number of published papers <i>per year</i>&quot;? (I can&#x27;t imagine that someone whose first paper was in the 1950s only published 1 additional paper in the next 30 years)
评论 #21076082 未加载
评论 #21074697 未加载
评论 #21074147 未加载
adipandasover 5 years ago
Fantastic over view of current trends in academia. There is truly a huge bias in the research publications. I think blogging is a better way to put forth your ideas and research rather than getting a publication in some cases.
评论 #21074763 未加载
xorandover 5 years ago
Excellent article. A relevant link would be the San Francisco Declaration on Research Assessment (DORA) <a href="https:&#x2F;&#x2F;sfdora.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;sfdora.org&#x2F;</a><p>Back to the article, lots of gems, like:<p>&gt;Today&#x27;s researchers can publish not only in an ever-growing number of traditional venues, such as conferences and journals, but also in electronic preprint repositories and in mega-journals that offer rapid publication times.<p>Did I just read a very normal point of view of a researcher putting on equal footing electronic preprint repositories and mega-journals?<p>The BOAI Open Access preachers surely can&#x27;t believe their eyes :) Heresy! (No researcher was involved in the BOAI flawed definition of green OA as archiving and gold OA as publishing.)
tgvover 5 years ago
Do people benefit from gaming the system? Then it surely is being gamed. And they do. Funding and tenure depend on these metrics.<p>&gt; Overwhelmed by the volume of submissions, editors at these journals may choose safety over risk and select papers written by only well-known, experienced researchers.<p>There is a bit of a &quot;circle jerk&quot; in this process: if you know the right people, you can get better reviews. In return, you review their papers or requests favorably. That also leads to repeating authors.
评论 #21076450 未加载
评论 #21078518 未加载
评论 #21075127 未加载
acollins1331over 5 years ago
Yes. But a good department will know you for who you are. Plenty of great science takes a long time do do, and it is known it&#x27;s hard to get funding for long-term monitoring experiments. Most grant money is for new and innovative ideas and no one is pushing out one of those every year. And if you are then you&#x27;re not doing 90% of the work on any of them.
buboardover 5 years ago
They should have limited the analysis to the most popular journals. There are tons more journals nowadays because its so easy to run one - but it s more important to know what’s happening at the well known ones. The lesser know are largely ignored
spodekover 5 years ago
Tough call: Goodhart&#x27;s Law says yes, metrics will be gamed, but Betteridge&#x27;s law says the answer is no.
评论 #21074734 未加载
caoilteover 5 years ago
No. We&#x27;re all being gamed. Academics were just nearer the front of the queue.
dr_dshivover 5 years ago
I wish there was a Google n grams for Google scholar...