I think interpretable is a overloaded term. If you use RAG, sure you can look at the retrieved text and understand what is being used, but at that point you're just interpreting the retriever.<p>LLMs need to be interpreted so that they can be edited have their biases understood in a systematic way. However just as people aren't "interpretable" these algorithms are not going to be able to display their inner workings with 100% confidence. It's going to remain probabilistic, which might be fine for the majority of use cases. I think we're coming from an age where everything was 100% interpretable because we knew what was going on inside the machine (e.g. in a knowledge graph).<p>There needs to be some definition of what we want to achieve with interpretability for us to understand what standards we need to keep.
I like this point of view, and I'll even go a step further...<p>If we're going to use LLMs as the basis for anything resembling general intelligence, it won't be through one-shot invocation of the model. It'll be through some kind of chain/tree/graph of thought where the model invokes itself recursively.<p>In this scenario, we have an exact transcript of the model's thought process, exactly as it occurred, <i>written for us in english.</i> The model can't even have a private "thought", everything needs to be in the visible context.<p>You can't get more interpretable than that.
A great article - just a technical nitpick for the author: "in-context learning" is not what you do with RAG. "In context learning" is a really confusing name for reasoning by analogy. In RAG you provide source information in the prompt - in ICL you provide examples of how the task should be accomplished. :<p>"""<p>In-context learning in language models, also known as few-shot learning or few-shot prompting, is a technique where the model is presented with prompts and responses as a context prior to performing a task. For example, to train a language model to generate imaginative and witty jokes.<p>We can leverage in-context learning by exposing the model to a dataset of joke prompts and corresponding punchlines:<p><pre><code> Prompt 1: “Why don’t scientists trust atoms?” Response: “Because they make up everything!
Prompt 2: “What do you call a bear with no teeth?” Response: “A gummy bear!”
Prompt 3: “Why did the scarecrow win an award?” Response: “Because he was outstanding in his field!”
</code></pre>
By training in different types of jokes, the model develops an understanding of how humor works and becomes capable of creating its own clever and amusing punchlines.<p>"""<p>from <a href="https://www.techopedia.com/from-language-models-to-problem-solvers-the-rise-of-in-context-learning-in-ais-problem-solving-journey" rel="nofollow noreferrer">https://www.techopedia.com/from-language-models-to-problem-s...</a>