Getting Started with Mistral-7b-Instruct-v0.1

How is this different than ollama (<a href="https:&#x2F;&#x2F;github.com&#x2F;jmorganca&#x2F;ollama">https:&#x2F;&#x2F;github.com&#x2F;jmorganca&#x2F;ollama</a>)? I would argue it&#x27;s even simpler to run LLMs locally with ollama

Mistral have created a docker image which hosts their model in vllm. Vllm creates an openai like http API interface.<p><a href="https:&#x2F;&#x2F;docs.mistral.ai&#x2F;quickstart&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;docs.mistral.ai&#x2F;quickstart&#x2F;</a>

Does anyone have any feedback on using those open-source models with any language except English? Particularly non-western group of languages like korean&#x2F;japanese&#x2F;chinese?<p>Will assess myself but wonder if anyone tried.

What are Mistrals strengths and weaknesses? I tried it for infrastructure as code and it wasn&#x27;t able to output more than the most basic examples, let alone modify them.

Thanks for sharing this!<p>I read somewhere that Mistral 7B had a similar performance to GPT-3, but it seems to be miles behind it unfortunately.

llama.cpp can run the Q4 variant of the same at 30tok&#x2F;s on an M1 Pro unlike the 20tok&#x2F;s being quoted

I really like the post that they mention (<a href="https:&#x2F;&#x2F;www.secondstate.io&#x2F;articles&#x2F;fast-llm-inference&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.secondstate.io&#x2F;articles&#x2F;fast-llm-inference&#x2F;</a>). The reasons for avoiding python all resonate with me. I&#x27;m excited to play with WASI-NN (<a href="https:&#x2F;&#x2F;github.com&#x2F;WebAssembly&#x2F;wasi-nn">https:&#x2F;&#x2F;github.com&#x2F;WebAssembly&#x2F;wasi-nn</a>) and that rust code is very readable to load up a GGUL model.

It seems a lot slower than ollama &#x2F; LM Studio. Is that likely because it’s not optimised as much or a limitation of some sort?

The demo looks too slow for practical usage. How much it will cost if i host it in cloud to get instant response similar to the speed of openAI?

Impressive use of wasmedge, great to see ML projects getting away from python for efficiency&#x27;s sake, thanks for sharing!

The WasmEdge README gives me the heebie-jeebies. Starry-eyed emojis, highlights use-case for today&#x27;s most trendy thing even though it&#x27;s general-purpose, mentions blockchain. This reeks of former cryptobros chasing the next big thing. I&#x27;d trust Wasmtime more.