TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Research Debt

508 pointsby wwilsonabout 8 years ago

33 comments

beambotabout 8 years ago
A lot of this seems to be a problem with academic publishing, peer review, and the pedantry contained therein.<p>I tried (twice!) to publish a paper that was scientifically sound, but written such that it could be understood by a lay audience. It was rejected; quoting a reviewer: &quot;This paper lacks math.&quot; That sentiment made me lose a lot of faith in academia. Now, I simply refuse to contribute any more time to writing papers or performing peer review (esp. in non-open publications). I&#x27;m sure I&#x27;m not alone; I know of at least one &quot;seminal&quot; robotics paper that was rejected from top venues multiple times for &quot;simplicity&quot; (lack of mathematics content) that went on to become a foundational paper in the field after appearing in a lower-tier venue years later.<p>The irony: it takes researchers a lot of time to make a paper dense &amp; concise. If they &quot;showed all steps&quot;, it would probably improve researcher productivity &amp; make the material more approachable to newcomers. Instead publishers enforce length restrictions... for which authors dedicate upto 25% to related work (some of which is useful; much of which is pandering to likely peer reviewers in small, niche fields). Length restrictions seem equally foolish in the age of digital publishing. And again, inadvertent pedantry is the only explanation I can imagine... but happy to be wrong.
评论 #13935196 未加载
评论 #13934430 未加载
评论 #13934910 未加载
评论 #13936855 未加载
评论 #13936179 未加载
评论 #13934558 未加载
评论 #13940008 未加载
评论 #13934466 未加载
评论 #13937287 未加载
评论 #13934447 未加载
评论 #13938266 未加载
评论 #13935563 未加载
btownabout 8 years ago
&quot;What is the role of human scientists in an age when the frontiers of scientific inquiry have moved beyond the comprehension of humans?&quot;<p>The above quote is from Ted Chiang&#x27;s short story &quot;The Evolution of Human Science,&quot; originally published in <i>Nature</i> as &quot;Catching crumbs from the table&quot; [0]. It&#x27;s a brilliant depiction of this very problem: when new developments contribute to an <i>increasing</i> gap between those who can <i>make</i> new developments, and those attempting to understand the state of the art, the entire process of scientific inquiry becomes less efficient. In fact, the scenario depicted is one where the majority of researchers become &quot;distillers,&quot; to use the language of the original post.<p>While Chiang posits a science-fiction reason for the divide, &quot;normal&quot; research&#x2F;technical debt is insidious as well. Without incentives to reduce debt, the knowledge gap widens until only a handful of experts can make significant contributions. It&#x27;s a problem that needs to be tackled head-on in both research and engineering. I&#x27;d love to see more initiatives like Distill.<p>[0] <a href="http:&#x2F;&#x2F;www.nature.com&#x2F;nature&#x2F;journal&#x2F;v405&#x2F;n6786&#x2F;full&#x2F;405517a0.html" rel="nofollow">http:&#x2F;&#x2F;www.nature.com&#x2F;nature&#x2F;journal&#x2F;v405&#x2F;n6786&#x2F;full&#x2F;405517a...</a> - a highly recommended companion piece to the original post.
评论 #13936290 未加载
评论 #13939433 未加载
cs702about 8 years ago
Yes. Yes. Yes. A million times yes.<p>I can&#x27;t count how many times I&#x27;ve invested meaningful time and effort to grok the key ideas and intuition of a new AI&#x2F;DL&#x2F;ML paper, only to feel that those ideas and intuitions could have been explained much better, less formally, with a couple of napkin diagrams.<p>Alas, authors normally have no incentive (or time, for that matter!) to publish nice concise explanations of their intuitions with easy-to-follow diagrams and clear notation... so the mountain of research debt continues to grow <i>to the detriment of everyone</i>.<p>I LOVE what Olah, Carter et al are trying to do here.
akyuabout 8 years ago
I really love this effort. Research papers are low bandwidth way to get information into our brains. They take a lot of effort to read, even if the ideas are not particularly complicated. Often when reading complicated material, I have to come up with metaphors in my mind to make sense of it. This is somewhat of a wasted effort, as the author who wrote the material surely had metaphors for their own mind when writing, but too often they don&#x27;t share these metaphors, and stick to purely technical writing. I think this is one reason why ideas like general relativity are so popular, even though the material is actually quite complicated. The average educated person can give a reasonable explanation of general relativity because the metaphors used to explain it are so powerful, even though its very unlikely they understand any of the math involved.
评论 #13933733 未加载
评论 #13934537 未加载
pcrhabout 8 years ago
I like the ideas put forth in this article. I wonder, though, if &quot;distillation&quot; is a re-casting of &quot;scholarship&quot; as considered by the humanities.<p>People studying topics ranging from Biblical Studies to History to Literature often do not create new source material, unlike in STEM. Yet there is a large degree of effort taken to &quot;distill&quot; existing facts through new lenses, producing novel concepts and interpretations. These efforts can transform our understanding of many areas of human endeavour.
评论 #13933314 未加载
adamseaabout 8 years ago
One thing I believe to be of great value which is not made explicit in this article (which I think is an awesome article), is that research debt, as they describe it, is basically _education_.<p>In other words, improve the educational resources for complex subjects.<p>In our age where we&#x27;re blessed with cheap printing of books and the possibility of creating complex interactive media, I think the question of designing user-friendly, powerful, and beautiful educational resources is a huge opportunity and pressing question.<p>Not just for people seeking to achieve a research-level understanding of a complex subject, but for all subjects and all people.<p>Consider the social value of beautiful, well-designed and nontrivial educational material for mathematics or basic science being widely available for all classes of people at all ages.<p>I&#x27;d argue that when news organizations use infographics or interactive journalism at its best, they are also performing this educational function.<p>Sorry for the long post, but to summarize, I think it&#x27;s useful to recognize research debt as a specific case of the art and practice of creating media for education.
dluanabout 8 years ago
There are &#x27;distillers&#x27; of large bodies of scientific research. Traditionally, they are science communicators, and more specifically science journalists.<p>The goal of a practicing scientist is very much at odds with someone whose job is to translate science into larger audiences. I&#x27;ve had very well-intentioned rational research scientists tell me with a straight face that &quot;my job is to produce science results, not to communicate it. that&#x27;s someone else&#x27;s job&quot;, usually with the attitude that it&#x27;s less respected or somehow self-aggrandizing. &quot;The best science will be self-evident&quot; attitude that all researchers secretly aspire for, not realizing that 99% of impactful science has had effort spent to promote, frame, or distribute it.<p>This weird stereotype is somehow beaten into scientists from the very beginning, and I haven&#x27;t been able to figure out where this comes from. Obviously, yes, it&#x27;s a lack of tools and accessibility into letting scientists also become distillers themselves. But the motivations and incentives at the center of the whole system is what&#x27;s making this whole imbalance. I think there are parts of our research system that actually say &quot;No, you cannot and should not distill your science&quot;.<p>Ultimately, for me, it gets back to funding. If review articles and outreach weighed just as much as citation count in tenure and grant committees, then maybe this could start to change. Yet, these committees still don&#x27;t value open access, and look how tough that battle has been.<p>Also - this solution is really great and commendable, but I don&#x27;t see how this works outside of ML&#x2F;CS where research outputs are more like software development - gists, snippets, prototypes that are immediately shared, pushed, forked. More science fields like ecology, synthetic biology, anthropology, will look like this, but it will take a few human generations.
评论 #13934897 未加载
评论 #13938914 未加载
tominousabout 8 years ago
Isaac Asimov anticipated the idea of a research distiller in &quot;The Dead Past&quot; with the character of Nimmo, a professional science writer:<p>&quot;Nimmo received his first assignment at the age of twenty-five, after he had completed his apprenticeship and been out in the field for less than three months. It came in the shape of a clotted manuscript whose language would impart no glimmering of understanding to any reader, however qualified, without careful study and some inspired guesswork. Nimmo took it apart and put it together again (after five long and exasperating interviews with the authors, who were biophysicists), making the language taut and meaningful and smoothing the style to a pleasant gloss.&quot;<p>In the story Nimmo has less prestige than a &quot;real researcher&quot; but the role pays well and he is in high demand.
cingabout 8 years ago
I don&#x27;t see anything wrong with the &quot;textbook&quot; -&gt; &quot;review&quot; -&gt; &quot;article&quot; strategy to climbing a mountain of debt. A good review paper should hit a sweet spot in terms of exposition, digestion, abstraction, and noise filtering. Transformative new ways of visual thinking, well, that&#x27;s a different story.<p>The problem is just that the pace of research in the authors field is too fast at the moment. What&#x27;s the hurry? Over time, the citation graph will reveal the most significant work, and the community will naturally distill that research for maximum effect. The danger is that beautiful distillation of an extremely &quot;niche topic&quot; will not change the fact that it has limited scope and may even limit abstraction. Of course, I say danger with tounge-in-cheek...
评论 #13933418 未加载
评论 #13933946 未加载
评论 #13935376 未加载
评论 #13936428 未加载
PaulHouleabout 8 years ago
There is no research distillation because scientists don&#x27;t get grants to do it.
评论 #13933772 未加载
eddotmanabout 8 years ago
Pretty compelling arguments.<p>I do think that the ML &#x2F; CS &#x2F; etc. community is actually more open than other academic fields, and so this is definitely the right subfield to start in. Putting open access preprints online is not common practice in all disciplines, although it really should be.<p>I wonder if it makes sense for Distill to also publish on fields outside of pure ML - e.g. as applied to specific problems in other domains. I work in materials informatics, and I suspect that research in such fields (ML + applied sciences) might benefit quite a bit from having key results &#x27;distilled&#x27; in this format.
评论 #13933640 未加载
CogitoCogitoabout 8 years ago
Article is really good, but it&#x27;s reference to the pi vs tau debate is kind of silly. It really isn&#x27;t a big deal. When I did my math phd I wrote 2 pi all the time, but this didn&#x27;t matter. The tiny convenience gained by changing to tau is totally trivial and doesn&#x27;t even deserve a mention compared to the rest of the things in the article.
评论 #13933817 未加载
评论 #13946916 未加载
mballantyneabout 8 years ago
When I&#x27;ve talked to senior researchers about these problems, they say that they have no problem finding distilled information about new results; they get it from in-person conversations at conferences that they attend frequently. The publishing of distilled research would most benefit low-status, newer researchers (like Ph.D. students), but it needs to be valued by senior researchers to make it into the incentive systems of hiring and grant funding. It seems like a tricky problem to fix the incentives here.
ameliusabout 8 years ago
I think one of the biggest things missing is a means to communicate openly about published research. What I&#x27;d like to see is a forum for every paper, and this forum should be properly moderated (perhaps by a peer-review system).<p>Such a forum could make it much easier to decipher published work, and to fill in details which were missing. Also, errors in publications become clear more quickly.
评论 #13938899 未加载
haddrabout 8 years ago
Brilliant thing! As a person pursuing a PhD I&#x27;d something that is in my opinion the best way to avoid research debt: have a good tutor or a good group (peers). This way you can learn new things (or ask for something you don&#x27;t understand) and really get to the peak sooner that doing everything alone.
killjoywashereabout 8 years ago
3blue1brown&#x27;s Essence of Linear Algebra series on YouTube should get some sort of honorary inclusion in Distill. There have been many &quot;visualization of algorithms&quot; posts on HN over the years. A collection of those would be good as well.
erikbabout 8 years ago
What I think is funny is that so many people complain about how bad science has become and how the papers and funding institutions enforce a very ineffective way of doing it. Why is nobody attempting or proposing new ways to make money with science? E.g. have people tried to do science in a subscription based model (like artists)? Or using free pappers+consulting on how to replicate the experiments or apply them in real projects (like open source)?
RangerScienceabout 8 years ago
Ooooh. I&#x27;ve actually been working on a blog post where I try reading a science paper (knowing only a little about the science) and learn &#x2F; explain as I go.<p>This seems like the better way to do it (my way is taking way to many words) - something like better science reporting.<p>My concern is the act of writing - Medium has a real nice web editor. What can I use to write an article using this HTML&#x2F;CSS&#x2F;JS without literally writing the HTML?
评论 #13937535 未加载
评论 #13933261 未加载
评论 #13933917 未加载
评论 #13933671 未加载
Drupabout 8 years ago
As an (aspirant) programming language theorist, we have a really great advantage in this field: advances in the field of prog lang have a natural distillation process: getting into a &quot;real&quot; programming language.<p>And it&#x27;s great, it means I can toy with features and consider &quot;what could I do if it was in a real language ?&quot;. Also, we can observe idioms and usage that gets developed when some advanced feature start being used by &quot;normal&quot; programmers.<p>Computer science, in general, does have the advantage that the distance to applications is quite often much shorter than math, which forces part of this distillation process to proceed a bit quicker. On the other hand, the formalization is highly non-uniform, due to how young it is as a science.
fmapabout 8 years ago
In my experience this is already being practiced. Maybe it&#x27;s different in machine learning, but in type theory simpler explanations are well worth publishing and are published all the time even in high impact venues.<p>As far as I can tell, progress towards eliminating &quot;research debt&quot; takes two forms: on the one hand there is a place for good exposition, typically in the form of textbooks, and on the other hand concepts get better understood over time and people come up with simpler explanations. In both cases the results can already be published...<p>Is the situation in machine learning really so bad that nobody is going to publish simpler explanations of known results?
intrasightabout 8 years ago
How about we just abstract to &quot;Intellectual Debt&quot;? A subclass that I&#x27;ve recently encountered is &quot;Learning Debt&quot;. I think all these types of debt can equally apply to learning - as in formal education. Came to mind while reading &quot;Undigested Ideas&quot; because I recently had a conversation with my daughter (sophomore at Yale) that a bad test grade means that you may have Intellectual Debt that needs to be retired by taking time after the test to make sure you understand what you did wrong.
euskeabout 8 years ago
When I hear things like this, I always think of Richard Feynman. He was simply a distilling genius to me.
评论 #13937103 未加载
nnonynousabout 8 years ago
This is nonsense.<p>Here is a (well-publicized) post which only serves to hype up and stake a claim on what is already valued and practiced, the exposition of research. The message seeks to capture an ignorant readership and practitioners with short-term memory, and have them walk away with the thought &quot;This is the place where good research expositions will be.&quot;<p>The machine learning learning community has already benefited from the myriad great expositions provided online for free, in addition to the locations where even source code is given alongside research findings. I will not link to these sites, in hopes to not seem a salesman, but if you&#x27;ve taken an interest in the field and taken some time to search on a topic (for example neural nets), you will have likely already found one of several free, helpful resources. This includes online courses, online books, blogs, videos, &amp;c.<p>These individuals give examples of already well-written expositions going back years (associating these well-received expositions with their own endeavour), yet the theme is one of &quot;newness&quot;, helped by the usage of terms like &quot;distillers&quot; and &quot;research debt&quot;. In Silicon Valley, it seems that if something&#x27;s been given enough press and sounds new, you should at least hop on the bandwagon for a while, lest you risk missing out on being an &quot;early adopter&quot;.<p>Publishers are not a new conception either. They&#x27;re a funnel which selects what you see. In an era where the larger population is beginning to see how much power publishers have over what they think, new efforts should strengthen decentralization. Playing to the tune of publishers has gotten us into the mess we&#x27;re in.<p>Peer review is a mainstay of science, and I am in complete support of it, but peer review and publishers do not need to coexist. I and others will happily use a decentralized system with peer review, additionally and crucially providing transparency.<p>Good exposition should occur. Good exposition does occur. Indeed, people are learning: Ask yourself when you last learned from someone&#x27;s writing. Audiences get their information from many sources, and if they can&#x27;t understand those sources, they don&#x27;t go to them.<p>Addendum. I will not discuss at length the repercussions of teaching the population how to create intelligent systems, but that is a dangerous road for all of us, and not one easily traversed. Companies and other powerful persons have a strong interest in guaranteeing they have a large pool of subordinates who have skills that they can profit greatly from, so it&#x27;s obvious why the push for software engineering and artificial intelligence teaching is so strong (the promise and hype has been strong). But this is heavy-handed and short-sighted. Businesses have functioned with this approach in the past however, so I wager they assume past performance can be used for prediction in this case as well. If anyone&#x27;s doing any thinking at all.
评论 #13950715 未加载
tomkat0789about 8 years ago
My scientific writing was transformed by Rafeal Luna&#x27;s &quot;Art of Scientific Storytelling&quot;. I&#x27;m on my phone, so I can&#x27;t do it justice here, but I think that style will be crucial for the distillation TFA is discussing.
stuartaxelowenabout 8 years ago
One take away I got from this is the immense value of great content marketing.<p>Especially for software, the best content marketing is an immense value to the community, helping it understand core ideas and adopt new technology more easily.
therealdrag0about 8 years ago
In terms of clear communication, I highly recommend &quot;The Sense of Style: The Thinking Person&#x27;s Guide to Writing in the 21st Century&quot; by Steven Pinker
safeandsoundabout 8 years ago
Perhaps distillers should be the reviewers of top publishers so they can have more power.
jeyoorabout 8 years ago
Kudos to this team for dogfooding their platform. That choice makes me far more likely to evaluate and eventually use it.
PeterStuerabout 8 years ago
Unfortunately arcane notations are an effective tool to obfuscate the shallowness of some ideas.
gburtabout 8 years ago
This is a really awesome perspective.
mfschabout 8 years ago
I agree 100% that it is way too hard to climb the mountains of knowledge, that it could be much easier, and that there is little incentive and cultural support in academia to work on these topics. Time and time again I see examples of where outcomes are not limited by the existing knowledge, but by the accessibility of said knowledge. I would gladly devote most of my time to distilling knowledge if I saw a viable career path for it.<p>Over the last years, I gave a lot of thought to the topic of conveying information. I see research as the process in which knowledge is created. This knowledge then has to be “encoded” in a format that allows for the transmission to other people. This encoding can be optimized in different ways. Research articles have the advantage that they are close to “lossless” in that they are supposed to contain all information necessary to build up that knowledge. This makes them well suited for archival, especially as they can be stored as a stack of paper.<p>However, research articles are often not optimized for building up that knowledge in an efficient way. I believe that the “encoding” optimized for learning &amp; understanding should be more like a progressive image codec, in that it provides a comprehensive view as soon as possible, filling in further details along the way. This also makes it possible to stop whenever you have reached the level of detail that is relevant to you. The challenge in creating these encodings is to extract the information that provides the most clear &amp; useful picture as soon as possible. I like the word “to distill” for that process, as it is really about extracting the essence of a body of information.<p>Doing this work for research articles is how I understand the goal of distill.pub, which seems extremely valuable to me. However, I think this is just the first step. What is the most useful distillation of all of deep neural networks? Of all of machine learning? Of all of computer science? As others mentioned, there are some forms of publications (review articles, textbooks) that do part of this distillation process, but they only cover part of the spectrum. Preciously few textbook contain a well thought-out summary of their contents and not just an introduction. In my experience, often the least amount of thought is given to the highest level of abstraction (e.g. what is the essence of mathematics?), even though they are the most fundamental ones.<p>It would be great if there was more focus on extracting useful understanding from the ocean of knowledge we already have (useful both in the sense of being applicable on its own as well as being a solid foundation to build more knowledge on). It looks like distill.pub is a step in that direction, and I really hope it will bring more attention and recognition to this kind of work.
RSchaefferabout 8 years ago
Chris, will Distill support community editing of articles like Wikipedia does?
评论 #13934018 未加载
debtor123about 8 years ago
This abstraction of debt doesn&#x27;t really work for me...
评论 #13933635 未加载
评论 #13933844 未加载