I think knowing what part of the knowledge base to delete — to get to adequately small reasoning model — is the hard part.<p>Doesn't "reasoning" rise from the knowledge? How much of a brain can you cut away before you affect the reasoning? When do you know what you've cut away, and what aspects did you miss / forget about?<p>We can probably train / fine-tune, w/ synthetic data, and we'll get reasonably close, but the "reasoning" will always hit rough patches, bc our training didn't include <i>that</i> kind of reasoning... and if we had to give it examples of every single kind of reasoning, then it can't move past all the already-established kinds of reasoning, so it's still pattern matching