What's the easiest way to run this assuming that you have the weights and the hardware? Even if it's offloading half of the model to RAM, what tool do you use to load this? Ollama? Llama.cpp? Or just import it with some Python library?<p>Also, what's the best way to benchmark a model to compare it with others? Are there any tools to use off-the-shelf to do that?
[dupe] <a href="https://news.ycombinator.com/item?id=39986047">https://news.ycombinator.com/item?id=39986047</a><p>Which has the link to the tweet instead of the profile:<p><a href="https://twitter.com/MistralAI/status/1777869263778291896" rel="nofollow">https://twitter.com/MistralAI/status/1777869263778291896</a>
Out of topic but are we now back at the same performance than ChatGPT 4 at the time people said it worked like magic (meaning before the nerf to make it more politically correct but making his performance crash)?
Here is the unofficial benchmark:
<a href="https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1/discussions/4" rel="nofollow">https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1/...</a>
Weird, the last post I see at that link is from the 8th of December 2023 and it's not about this.<p>Edit: Ah, it's the wrong link. <a href="https://news.ycombinator.com/item?id=39986047">https://news.ycombinator.com/item?id=39986047</a><p>Thanks SushiHippie!
It's weird that more than a day after the weights dropped, there still isn't a proper announcement from Mistral with a model card. Nor is it available on Mistral's own platform.
To this day 8x7b Mixtral remains the best model you can run on a single 48GB GPU. This has the potential to become the best model you can run on two such GPUs, or on an MBP with maxed out RAM, when 4-bit quantized.
What is the excitement around models that arent as good as llama?<p>This is clearly an inferior model that they are willing to share for marketing purposes.<p>If it was an improvement over llama, sure, but it seems like just an ad for bad AI.
The lack of a corresponding announcement on their blog makes me worry about a Twitter account compromise and a malicious model. Any way to verify it’s really from them?