My puny version of ChatGPT.<p>This was based on the excellent LLM lecture series by Andrej Karpathy: <a href="https://www.youtube.com/watch?v=kCc8FmEb1nY">https://www.youtube.com/watch?v=kCc8FmEb1nY</a><p>The main points of differentiation are that my version is token-based (tiktoken) with code to load up multiple text files as a trining set. Plus, it has a minimal server which is a drop-in replacement for the OpenAI REST API.<p>So you can train the default tiny 15M parameter model, and use that in your projects instead of ChatGPT.<p>I trained it on 20Mb of Project Gutenberg encyclopaedias, then fine-tuned it on 120 dad jokes, to get a Q: A: prompt format.<p>This model + training set is so small that the results are basically a joke; it's for entertainment purposes only. The code is also very rough, and the server only has the minimum functionality filled in.<p>I embodied this model in my talking LLM-driven hexapod robot, and it could give very silly answers to spoken questions.
This is a great idea.
I want to make a 'pet' for my kid.
I can't get them a real dog, so why not a tinyLLM?<p>Training on guttenberg data is a great idea. What I would do is train it on all the e-books I have that are suitable for kids (I managed to find quite a lot online).<p>The dad jokes idea is great, please keep doing things along this line.