Anyone who has iterated on trained models for long enough knows that feedback loops can be a serious problem. If your models are influencing the generation of data that they are later retrained on, it gets harder and harder to even maintain model performance. The article mentions one experiment in this direction: "With each generation, the quality of the model actually degraded." This happens whenever there aren't solid strategies to avoid feedback loop issues.<p>Given this, the problem isn't just that there's not enough new content. It's that an ever-increasing fraction of the content in the public sphere will be generated by these models. And can the models detect that they are ingesting their own output? If they get good enough, they probably can't. And then they'll get worse.<p>This could have a strange impact on human language / communication as well. As these models are increasingly trained on their own output, they'll start emulating their own mistakes and more of the content we consume will have these mistakes consistently used. You can imagine people, sometimes intentionally and sometimes not, starting to emulate these patterns and causing shifts in human languages. Interesting times ahead...
AlphaZero demonstrates that more human-generated data isn't the only thing that makes an AI smarter. It uses zero human data to learn to play Go, and just iterates. As long as it has a way of scoring itself objectively (which it obviously does with a game like Go), it can keep improving with literally no ceiling to how much it can improve.<p>Pretty soon ChatGPT will be able to do a lot of training by iterating on its own output, such as by writing code and analyzing the output (including using vision systems).<p>Here's an interesting thing I noticed last night. I have been making a lot of images that have piano keyboards in them. DALL-E 3 makes some excellent images otherwise (faces and hands mostly look great), but it always messes up the keyboards, as it doesn't seem to get how black keys are in alternating groups of two and three.<p>But I tried getting chatgpt to analyze an image, using its new "vision" capabilities, and the first thing it noticed was that the piano keys were not properly clustered. I said nothing about that, I just asked it "what is wrong with this image" and it immediately found that. What if it could feed this sort of thing back in, using similar logic to Alpha Zero?<p>That's just a tiny hint of what is to come. Sure, it typically needs human generated data for most things. It's already got thousands of times more than any human has looked at. It will also be able to learn from human feedback, for instance a human could tell it what it got wrong in a response (whether regular text, code, or image), and explain in natural language where it deviated from what was expected. It can learn which humans are reliable, so it can minimize the number of paid employees doing RLHF, using them mostly to rate (unpaid) humans who choose to provide feedback. Even if most users opt out of giving this sort of feedback, there will be plenty to give it new, good information.
>As a rule, chatbots today have a propensity to confidently make stuff up, or, as some researchers say, “hallucinate.” At the root of these hallucinations is an inability to introspect: the A.I. doesn’t know what it does and doesn’t know.<p>The last bit doesn't seem to be true. There's quite a lot of indication that the computation can distinguishing hallucinations. It just has no incentive to communicate this.<p>GPT-4 logits calibration pre RLHF - <a href="https://imgur.com/a/3gYel9r" rel="nofollow noreferrer">https://imgur.com/a/3gYel9r</a><p>Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - <a href="https://arxiv.org/abs/2305.14975" rel="nofollow noreferrer">https://arxiv.org/abs/2305.14975</a><p>Teaching Models to Express Their Uncertainty in Words - <a href="https://arxiv.org/abs/2205.14334" rel="nofollow noreferrer">https://arxiv.org/abs/2205.14334</a><p>Language Models (Mostly) Know What They Know - <a href="https://arxiv.org/abs/2207.05221" rel="nofollow noreferrer">https://arxiv.org/abs/2207.05221</a><p>Also even if we're strictly talking about text, there is still a ton of data left to train on. We've just barely reached what is easily scrapable online and are nowhere near a real limit yet. And of course, you can just train more than one epoch. That said, it's very clear quality data is far more helpful than sheer quantity and sheer quantity is more likely than not to derail progress.
> Yelp caught Google scraping their content with no attribution. ... A similar thing happened at a company I once worked for, called Genius. We sued Google for copying lyrics from our database into the OneBox; I helped prove that it was happening by embedding a hidden message into the lyrics, using a pattern of apostrophes that, in Morse code, spelled “RED HANDED.<p>Ah, the old aphorism, don't put anything on the web you don't want Google to take.
I have the entirely unrefined notion, that, surely, lack of data is not what is keeping us from creating much, much better LLMs.<p>I understand with how training is done right now that more data makes things scale really well without having to come up with new concepts, but it seems completely obvious that better processing of already available knowledge is the way to make the next leaps. The idea is that, what is keeping me from having expert level knowledge in 50 different fields and using that knowledge to draw entirely new connections between all of them, in addition to understanding where things go wrong, is not lack of freely available expert level information.<p>And yet, GPT4 barely reaches competency. It feels like computers should be able to get much more out of what is already available, specially when levering cross discipline knowledge to inform everything.
> These Web sites want chatbots to give credit to their contributors; they want to see prominent links; they don’t want the flywheel that powers knowledge production in their communities to be starved of inbound energy.<p>But this is ultimately impossible right? That’s the one thing I really hate about what is happening right now with ChatGPT.<p>I can’t tell you how many people are worried about their future because of AI because I don’t know the exact number, but I know I am worried about it because it can already do so much, and I fail to see a scenario in which attribution alone is going to make things better.<p>Writing and digital art more than code, but not even code is safe. It is merely safe by the extent that OpenAI is willing to drip feed its future releases.
I think next stage in AI training is as the authors said, synthetic data. I am not worried about the G.I.G.O. curse, you can do synthetic data generation successfully today with GPT-4. For example in the TinyStories dataset, or the Phi-1 & 1.5 models, or the Orca dataset we have seen big jumps in competency on the small models. Phi punches 5x above its weight class.<p>So how can you generate data at level N+1 when you have a model at level N?<p>You amplify the model - give it more tokens (CoT), more rounds of LLM interaction, tools like code executor and search engine, you use retrieval to bring in more useful context, or in some cases you can validate by code execution.<p>But there is a more general framework - by embedding LLMs in larger systems, they act as sources of feedback to the model. From the easiest - a chat interface, where the "external system" is a human, to robotics and AI agents that interact with anything, or simulations. We need to connect AI to feedback sources so it can learn directly, not filtered through human authored language.<p>From this perspective it is apparent that AI can assimilate much more feedback signal than humans. The road ahead for AI is looking amazing now. What we are seeing is language evolving a secondary system of self replication besides humans - LLMs. Language evolves faster than biology, like the rising tide, lifting both humans and AI.
A bit nitpicking. I do not think it is quite right to say that current large language models learn, we infuse them with knowledge. On the one hand it is almost just a technicality that the usage of large language models and the training process are two separate processes, on the other hand it is a really important limitation. If you tell a large language model something new, it will be forgotten once that information leaves the context window. Maybe to be added back later on during a training run using that conversation as training data.<p>Building an AI that can actually learn the way humans learn instead of slightly nudging the output in one direction with countless examples would be a major leap forward, I would guess. I have no good idea how far we are away from that, but it seems not the easiest thing to do with the way we currently build those systems. Or maybe the way we currently train these models turns out to be good enough and there is not much to be gained from a more human like learning process.
The problem is that ai doesnt learn as such. Therefore it depends on continuously ingesting data to maintain token databases up to date. Naturally at some point a ceiling will be hit and the quality of generic token databases will stagnate.
The article seems to suggest that humans, esp human linguistic output, are the best sources of knowledge.<p>Let's just say that they often aren't.
Compare the size in MB of a book to the size in GB of a movie. There's so, so much more data available. Multimodal models are not just the next step, they're already happening. AI will get better.
We will have people hooked up to Neuralink.<p>We will call them Psykers.<p>The Machine God has blessed them with the ability to take existing knowledge and fill the void.<p>No RAG. No vector databases. Pure willpower and biologics combined with the blessings of the Machine God.