We published a paper a while back in a genetic programming conference, and before the paper had been published (only the title was announced) Schmidhuber guessed that we had not cited his work. He very publically put us through the wringer for not citing him as the inventor of the concept were were examining. In fact we had <i>not</i> cited him: but among our dozen or so previous examples, we had cited two <i>others</i> which long predated his work and were the actual seminal papers. He had failed to cite them himself.
List of "famous" ML people not to waste time on:<p>- Gary Marcus<p>- Juergen Schmidhuber<p>- Pedro Domingos<p>- Max Tegmark<p>- Eliezer Yudkowsky<p>Some context for people unfamiliar with ML research:
the author, Schmidhuber, is well known for claiming that he should get credit for many ML ideas. Most ML researchers think that:<p>- He doesn't deserve the credit he claims, in most if not all cases.<p>- There's a few cases where his papers should have been cited and weren't. That's fairly common.<p>- People do not get much credit for formulating an abstract idea in a paper or implementing it on a toy problem. Credit belongs to whoever actually makes it work.<p>- Credit assignment in ML is not perfect but roughly works.
This whole comment section is full of absolutely unacceptable ad-hominem attacks on Schmidhuber, from people who most likely haven't even read any of the works in question and are certainly not showing any of the "intellectual curiosity" this site is supposed to be about.<p>Anyone who cares about academic integrity should at least not attack someone complaining of plagiarism. That sort of attack is the academic equivalent of blaming the victim. If even half of Schmidhuber's accusations have a basis that's still a major academic scandal of epic proportions.
I can see some angry comments here, but so far I have not seen any facts that refute his claims. Once I spent a long time reviewing a related paper on Hacker News, and I think he is right about disputes B1, B2, B5, H2, H4, H5. I'd have to study the others more closely:<p>B: Priority disputes with Dr. Bengio (original date v Bengio's date):
B1: Generative adversarial networks or GANs (1990 v 2014)
B2: Vanishing gradient problem (1991 v 1994)
B3: Metalearning (1987 v 1991)
B4: Learning soft attention (1991-93 v 2014) for Transformers etc.
B5: Gated recurrent units (2000 v 2014)
B6: Auto-regressive neural nets for density estimation (1995 v 1999)
B7: Time scale hierarchy in neural nets (1991 v 1995)<p>H: Priority disputes with Dr. Hinton (original date v Hinton's date):
H1: Unsupervised/self-supervised pre-training for deep learning (1991 v 2006)
H2: Distilling one neural net into another neural net (1991 v 2015)
H3: Learning sequential attention with neural nets (1990 v 2010)
H4: NNs program NNs: fast weight programmers (1991 v 2016) and linear Transformers
H5: Speech recognition through deep learning (2007 v 2012)
H6: Biologically plausible forward-only deep learning (1989, 1990, 2021 v 2022)<p>L: Priority disputes with Dr. LeCun (original date v LeCun's date):
L1: Differentiable architectures / intrinsic motivation (1990 v 2022)
L2: Multiple levels of abstraction and time scales (1990-91 v 2022)
L3: Informative yet predictable representations (1997 v 2022)
L4: Learning to act largely by observation (2015 v 2022)
Reading the article and some of the links make me feel like the author Jürgen Schmidhuber is the academic version of the patent troll.<p>It sounds like he published some theoretical musings back in 1990s without any real practical implementation that did anything useful and since then has run around accusing AI researchers who actually produced concrete research and techniques to get are actually in use today of plagiarism.
The machine learning field as a whole has a huge credit assignment problem. This post seems to encourage other ML researchers to come out with their own priority disputes. Tomas Mikolov just aired his grievances:<p>> I wanted to popularize neural language models by improving Google Translate. I did start collaboration with Franz Och and his team, during which time I proposed a couple of models that could either complement the phrase-based machine translation, or even replace it. I came up (actually even before joining Google) with a really simple idea to do end-to-end translation by training a neural language model on pairs of sentences (say French - English), and then use the generation mode to produce translation after seeing the first sentence. It worked great on short sentences, but not so much on the longer ones. I discussed this project many times with others in Google Brain - mainly Quoc and Ilya - who took over this project after I moved to Facebook AI. I was quite negatively surprised when they ended up publishing my idea under now famous name "sequence to sequence" where not only I was not mentioned as a co-author, but in fact my former friends forgot to mention me also in the long Acknowledgement section, where they thanked personally pretty much every single person in Google Brain except me. This was the time when money started flowing massively into AI and every idea was worth gold. It was sad to see the deep learning community quickly turn into some sort of Game of Thrones. Money and power certainly corrupts people...<p>Reddit post: "Tomas Mikolov is the true father of sequence-to-sequence"
<a href="https://www.reddit.com/r/MachineLearning/comments/18jzxpf/d_tomas_mikolov_is_the_true_father_of/" rel="nofollow noreferrer">https://www.reddit.com/r/MachineLearning/comments/18jzxpf/d_...</a>
It's nice of Schmidhuber to point out the quality papers with theoretical advancements and actual validations that fix the intellectual problems with his weak musings.
As an outsider to this space, it seems suspect that all of the people he has a beef with (like LaCun) will gladly cite others and predecessors, but not Schmidhuber.<p>Has he explained his reasons for thinking there is some conspiracy? Otherwise it reflects badly on his assessment of himself, or possibly his mental state.
Copying and pasting Urban Dictionary's definitions of "to schmidhuber"[a], "schmidhuber"[b] and "schmidhubered"[c]:<p>---<p>to schmidhuber<p>When you publicly claim that someone else's idea that is remotely resembling your own is stolen from you.<p>---<p>schmidhuber<p>1) To interject for a moment and explain how one's recent popular idea is a few transformations away from your 1991 paper.<p>2) To miraculously produce fifty years of relevant literature after someone claims to trace the origin of an idea in a particular work.<p>---<p>schmidhubered<p>Being "schmidhubered" looks something like this:<p>1) Invent something brilliant that no one cares about. Experience derision.<p>2) That thing becomes popular years later. Someone else is given credit for inventing it. That person appears in the New York Times and is declared smartest person alive.<p>3) Go on a campaign explaining the situation and how you are the rightful inventor and thus the rightful Smartest Person Alive.<p>4) Everyone accuses you of being a sore loser and no one takes you seriously.<p>5) A verb is named after you.<p>---<p>[a] <a href="https://www.urbandictionary.com/define.php?term=to+schmidhuber" rel="nofollow noreferrer">https://www.urbandictionary.com/define.php?term=to+schmidhub...</a><p>[b] <a href="https://www.urbandictionary.com/define.php?term=schmidhuber" rel="nofollow noreferrer">https://www.urbandictionary.com/define.php?term=schmidhuber</a><p>[c] <a href="https://www.urbandictionary.com/define.php?term=schmidhubered" rel="nofollow noreferrer">https://www.urbandictionary.com/define.php?term=schmidhubere...</a>