TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

AI is currently just glorified compression

77 pointsby kklisuraover 1 year ago

22 comments

rodletteover 1 year ago
This is covered in &quot;Information Theory, Inference, and Learning Algorithms&quot; by David MacKay ( <a href="https:&#x2F;&#x2F;www.inference.org.uk&#x2F;itprnn&#x2F;book.pdf" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.inference.org.uk&#x2F;itprnn&#x2F;book.pdf</a> ):<p>&gt; Why unify information theory and machine learning? Because they are two sides of the same coin. In the 1960s, a single field, cybernetics, was populated by information theorists, computer scientists, and neuroscientists, all studying common problems. Information theory and machine learning still belong together. Brains are the ultimate compression and communication systems. And the state-of-the-art algorithms for both data compression and error-correcting codes use the same tools as machine learning.<p>* In compression, gzip is predicting the next character. The model&#x27;s prior is &quot;contiguous characters will likely recur&quot;. This prior holds well for English text, but not for h264 data.<p>* In ML, learning a model is compressing the training data into a model + parameters.<p>It&#x27;s not a damning indictment that current AI is just compression. What&#x27;s damning is our belief that compression is a simpler&#x2F;weaker problem.
评论 #38400938 未加载
评论 #38400370 未加载
评论 #38401036 未加载
评论 #38400230 未加载
评论 #38400368 未加载
评论 #38401299 未加载
评论 #38400299 未加载
评论 #38400243 未加载
Bnichsover 1 year ago
&quot;Jet engines are just glorified oil lamps.&quot;<p>Not that I think the current AI is as life-changing as is purported, but this comparison is terrible. Almost all complex software is made up of a bunch of other previous simpler technologies.
评论 #38400632 未加载
furyofantaresover 1 year ago
I love how I&#x27;ve seen Ilya talk about it. If we could find the shortest program to reproduce the training set, that would be optimal compression. But it&#x27;s an intractable problem, we can&#x27;t even come close, there&#x27;s just no way to come at it.<p>But with deep learning we can instead find a circuit that approaches reproducing the training set.<p>This <i>is</i> lossy compression. There&#x27;s nothing &quot;just glorified&quot; about it though; the result is astounding.<p>A more appropriate takeaway might be that sufficient compression is mind-bendingly more powerful than intuition might otherwise guess.<p>Does calling it &quot;just glorified&quot; guide any intuition that in order to compress amazon reviews a neural net is gonna have weights that correspond to sentiment? Does it tell you that such compression also ends up producing an artifact that can be put to work in a generative framework? And that it&#x27;ll be a very useful generative framework because such compression required weights that correspond to all sorts of useful ideas, a compression of something more meaningful than just text?<p>Calling it &quot;just glorified X&quot; is clickbait. It&#x27;s compression alright, and it&#x27;s either 1) also whole lot more, or 2) compression is a whole lot more wild and impressive than you thought, or both.
评论 #38406278 未加载
评论 #38402224 未加载
booleandilemmaover 1 year ago
As usual, when compression is brought up in the context of AI, it seems relevant to mention the Hutter Prize:<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Hutter_Prize" rel="nofollow noreferrer">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Hutter_Prize</a>
IMTDbover 1 year ago
We also use to say that compression is the science of prediction.<p>Scientific models are also tools used to predict the results of experiments.<p>So if AI is able - through « compression » - to build models that help us predict experiments and understand the world better; it fully deserves the « intelligence » suffix.
评论 #38402254 未加载
baneover 1 year ago
This wasn&#x27;t obvious to me a while ago, but I&#x27;ve come around to it. To me the important thing is that it&#x27;s not only compression, it&#x27;s <i>lossy</i> compression.<p>Models (any kind) are basically an imperfect approximation of some system, and that definition is precisely what&#x27;s identified here.<p>You can demonstrate it by assuming some sequential process that gets an &quot;AI&quot; like an LLM to generate as much content as it can[1], then train a new generation model on this content. Then use the new model to generate as much content as <i>it</i> can, train a third generation model, and so on.<p>LLM-&gt;generate-&gt;LLM&#x27;-&gt;generate&#x27;-&gt;LLM&#x27;&#x27;-&gt;generate&#x27;&#x27;-&gt;...-&gt;LLM&#x27;...&#x27;<p>Since each generation may not produce content with <i>every</i> possible token it has been trained on, it stands to probability that eventually some tokens, especially low probability tokens, will simply be left out of the generated content. The next model will therefore <i>lose</i> that token and it will never be part of the content <i>it</i> generates, and so on.<p>After enough generations of this process, you eventually end up with a model with a single, or no, tokens it can use to generate text.<p>The problem is that the models are trained on <i>so</i> much information that they&#x27;re effectively models of the textual content of our civilization. If we hand off the reigns to these systems, they may work well for a while, and may even seem to be produce novel ideas (but which are secretly just old ideas in combinations we haven&#x27;t seen before), but they&#x27;ll also produce content which will inevitably be used to train later models. As the content these models produces starts to dominate the ratio of human-text:model-text, there will be less of a reason to preserve the human text and we&#x27;ll end up with the scenario above.<p>Things may work well for a while, perhaps a <i>long</i> time, but even if we don&#x27;t end up with models trained on one token, we will end up with fewer and fewer ideas-as-text represented within them. Civilization will stagnate, the singularity will not commence, and things will slowly regress.<p>Of course this all presumes research stops, and somehow LLMs become religion and we turn society over to them. Which of course isn&#x27;t what&#x27;s going to happen. Is it?
评论 #38400765 未加载
kklisuraover 1 year ago
Nitter: <a href="https:&#x2F;&#x2F;nitter.net&#x2F;ChombaBupe&#x2F;status&#x2F;1727720756371403131" rel="nofollow noreferrer">https:&#x2F;&#x2F;nitter.net&#x2F;ChombaBupe&#x2F;status&#x2F;1727720756371403131</a>
simbolitover 1 year ago
think about video compression for a &quot;zoom&quot; call.<p>You could just send a lot of bitmap files. Or, you could save bandwidth and zip them before send.<p>Or you could integrate some image-specific compression into the file format, as in motion-jpeg. And in mpeg2&#x2F;h264&#x2F;h265 you supercharge this with temporal compression, not just adjacent pixels, not just blocks within the frame, but also pixels and blocks from adjacent frames are used to predict each pixel.<p>And now think about not sending video, just a single initial jpeg, some face shape data, and now a continuous stream of position data of a dozen or so points on your face and the facial movements are reconstructed. (Nvidia research project from last year)<p>And now think about no longer sending images at all, just coordinates for points in latent space (aka very abstract image descriptions) and a super-fast diffusion model &quot;generates&quot; the right frames on the fly.<p>Where does &quot;compression&quot; end, where does &quot;AI&quot; start? Ultimately, the marketing people decide.<p>Don&#x27;t get hung up on the term &quot;AI&quot;. &quot;AI&quot; is a bullshit hype buzzword. Calling it such serves no scientific-practical purpose, it is solely meant to impress venture capital.
2OEH8eoCRo0over 1 year ago
Once again, so does the human brain. You aren&#x27;t remembering a high bandwidth stream of raw sensory input but a distilled essence of that.
评论 #38400145 未加载
评论 #38400148 未加载
3cats-in-a-coatover 1 year ago
So is our brain. “Model” by definition is an approximation of something else (or the thing we’re modeling… it’s a very overloaded term all right).<p>But if you think you can use this “realization” to dismiss AI or claim what it can or cannot do… you’re missing the forest for the trees.
Arntover 1 year ago
I chatted briefly to overtone.ai a few weeks ago, heading back to the hotel from a conference. What they do is train an existing LLM model to detect things about the text (overtones, I suppose you might say). What&#x27;s interesting in this context is that they train the AI using an English corpus, but once trained the AI is able to detect the same traits in other languages.<p>This sounds quite different from compression.
评论 #38403237 未加载
jasfiover 1 year ago
I once read that intelligence is compression (or similar). An abstract way of thinking about it, but if true, then AI is on the right track.
评论 #38400693 未加载
xeckrover 1 year ago
Ok. Let me know when WinRAR learns to speak.
TeMPOraLover 1 year ago
Yes. AI is just compression. So is understanding, <i>fundamentally</i>. That&#x27;s literally what it&#x27;s about.
评论 #38404522 未加载
jesuslopover 1 year ago
Liked this view always (look for Hutter prize rationale), but I think it needs to be accomodated in a general perception&#x2F;action loop that optimizes a lower level fitness&#x2F;utility&#x2F;reward (for instance an inner sense of pleasure&#x2F;pain).
thealistraover 1 year ago
Isn’t childbirth a glorified decompression of parents’ DNA?<p>This is a tweet level of discourse
kklisuraover 1 year ago
Relevant paper: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2311.13110" rel="nofollow noreferrer">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2311.13110</a>
评论 #38400380 未加载
RecycledEleover 1 year ago
It seems this X post is from someone trying to claim LLMs are copyright violations.<p>The argument that LLMs are only a new compression algorithm is nonsense.
glimsheover 1 year ago
&quot;Humans are just glorified monkeys&quot;...
ShamelessCover 1 year ago
tl;dr Tweet removes any potentially boring nuance from the research in favor of sensationalism and upvotes.
评论 #38400041 未加载
dgudkovover 1 year ago
Is generalization also a form of compression?
Racing0461over 1 year ago
compression is just glorified flow of electrons. Still usefull non the less.