i gave Llama 3 Instruct 7B Q8 a try using LM Studio on a 16 GB Macbook Pro w/ M2. not sure what do at first I asked it to be a dungeon master to get some creative conversation going and i'm satisfied with both the performance and creativity if not impressed (link: https://pastebin.com/raw/iM4U8skk if you're interested)<p>configuration:<p>- laptop: macbookpro m2 w/ 16 gb ram<p>- context length: 8192 (max)<p>- gpu layers: 33 (max)cpu threads: 8<p>response:<p>- time to first token: ~2s by the end of the conversation (4892 total token count)<p>- speed: ~7-8 tok/s<p>- memory usage: 13GB (system total)<p>- memory pressure: slightly over 50% (>90% when coding with containers)<p>now, these results are on par with ChatGPT 4. i compared a few general knowledge questions and coding problems, including some niche libraries, and it seem to do very well against ChatGTP 4 as well.<p>i want to compare your experience and wonder your opinions. is it possible to run copilot alike with a local server?