I want to make a Discord bot that impersonates all my friends and continues to refine the model as the conversations continue. Basically this [1] post, but with a more modern model and, ideally, reinforcement learning. Seems like this would fit the bill.... Is there anything else that would make this easier?<p>[1] <a href="https://www.izzy.co/blogs/robo-boys.html" rel="nofollow">https://www.izzy.co/blogs/robo-boys.html</a>
From the title I misunderstood what it does. However, now I'm wondering if what I thought is was (don't ask my why I thought it) is possible:<p>I have a PC that is able to run e.g. Mistral Instruct 7B Q4 inference with around 30 token/s.<p>How (computation and memory) expensive would it be to also run backpropagation in addition to inference?<p>I'm aware that the models are typically fed with much more and better data than what is typically provided during normal conversations but on the other hand if I could finetune my local model a teeny tiny bit during during / after each conversation I have with it anyways, it would after a while be perfectly customize for me.<p>I'm also aware that this could be problematic for models that are used by multiple users but my intended use case would be personal use by a single user.