LM Studio – Discover, download, and run local LLMs

This looks great!<p>If you&#x27;re looking to do the same with open source code, you could likely run Ollama and a UI.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;jmorganca&#x2F;ollama">https:&#x2F;&#x2F;github.com&#x2F;jmorganca&#x2F;ollama</a> + <a href="https:&#x2F;&#x2F;github.com&#x2F;ollama-webui&#x2F;ollama-webui">https:&#x2F;&#x2F;github.com&#x2F;ollama-webui&#x2F;ollama-webui</a>

I really like LM Studio and had it open when I came across this post. LM Studio is an interesting mixture of:<p>- A local model runtime<p>- A model catalog<p>- A UI to chat with the models easily<p>- An openAI compatible API<p>And it has several plugins such as for RAG (using ChromaDB) and others.<p>Personally I think the positioning is very interesting. They&#x27;re well positioned to take advantage of new capabilities in the OS ecosystem.<p>It&#x27;s still unfortunate that it is not itself open-source.

Curious about this and I just download it.<p>Want to try uncensored models.<p>I have a question, looking for the most popular &quot;uncensored&quot; model I just find &quot;TheBloke&#x2F;Luna-AI-Llama2-Uncensored-GGML&quot;, but it has 14 files to download between 2 to 7 GB, I just download the first one: <a href="https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;DE2byOB" rel="nofollow noreferrer">https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;DE2byOB</a><p>I try the model and it works: <a href="https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;2vtPcui" rel="nofollow noreferrer">https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;2vtPcui</a><p>I should download all the 14 files to get better results?<p>Also, asking how to make a bomb it looks that at least this model isn&#x27;t &quot;uncesored&quot;: <a href="https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;iYz7VYQ" rel="nofollow noreferrer">https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;iYz7VYQ</a>

This app could use some simple UI improvements:<p>- The chatbox field has a normal &quot;write here&quot; state, when no chat is really selected. I thought my keyboard broke until I discovered that<p>- I didn&#x27;t find a way to set cuda acceleration before loading a model, only managed to set gpu offloaded layers and using &quot;relaunch to apply&quot;<p>- Some HugginFace models are simply not listed and there&#x27;s no indication about why. I guess models are really curated, but somehow presented as a HuggingFace browser?<p>- Scrolling in the accordion parts of the interface seems to be responding to mouse wheel scroll only. I have a mouse with a damaged one and couldn&#x27;t find a way to reliably navigate to bottom drawers<p>That said, I really liked the server tab, which allowed for initial debugging very easily

I don&#x27;t mean this as a criticism, I&#x27;m just curious because I work in this space too: who is this for? What is the niche of people savvy enough to use this who can&#x27;t run one of the many open source local llm software? It looks in the screenshot like it&#x27;s exposing much of the complexity of configuration anyway. Is the value in the interface and management of conversation and models? It would be nice to see info or even speculation about the potential market segments of LLM users.

Amusing qualifications for the senior engineering roles they’re hiring for:<p>“Deep understanding of what is a computer, what is computer software, and how the two relate.”<p>Right after the senior ML role that requires people understand how to write “algorithms and programs.”<p>Kinda hard to take those kinds of requirements seriously.

For my experiments with new self-hostable models on Linux, I&#x27;ve been using a script to download GGUF-models from TheBloke on HuggingFace (currently, TheBloke&#x27;s repository has 657 models in the GGUF format) which I feed to a simple program I wrote which invokes llama.cpp compiled with GPU support. The GGUF format and TheBloke are a blessing, because I&#x27;m able to check out new models basically on the day of their release (TheBloke is very fast) and without an issue. However, the only frontend I have is console. Judging by their site, their setup is exactly the same as mine (which I implemented over a weekend), except that they also added a React-based UI on top. I wonder, how they&#x27;re planning to commercialize it, because it&#x27;s pretty trivial to replicate, and there&#x27;re already open-source UI&#x27;s like oogabooga.

This works, but I&#x27;ve noticed that my CPU use goes up to about 30 percent, all in kernel time (windows), after installing and opening this, even when it&#x27;s not doing anything, on two separate machines... I also hear the fan spinning fast on my laptop.<p>Killed the LM studio process and re-opened it and the ghost background usage is down to about 5%.

<a href="https:&#x2F;&#x2F;github.com&#x2F;enricoros&#x2F;big-agi">https:&#x2F;&#x2F;github.com&#x2F;enricoros&#x2F;big-agi</a> seems better and is open source

M1 is only 3 years old and no one cares to support intel macs any more. There are surely a lot of them out there. Are they that much worse to run LLMs on?

I’m late to the game about this. So I’ll ask a stupid question.<p>As a contrived example, what happens if you feed the LoTR books, the Hobbit, the Silmarillion, and whatever else is germane, into an LLM?<p>Is there a base, empty, “ignorant” LLM that is used as a seed?<p>Do you end up with a Middle Earth savant?<p>Just how does all this work?

LMStudio is great, if a bit daunting. If you’re on Mac and want a native open source interface, try out FreeChat <a href="https:&#x2F;&#x2F;www.freechat.run" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.freechat.run</a>

This is what I use on Windows 10.<p>I have an HP z440 with an E5-1630 v4 and 64GB DDR4 quad channel RAM.<p>I run LLMs on my CPU, and the 7 billion parameter models spit out text faster than I can read it.<p>I wish it supported LMMs (multi modal models.)

Disappointing, no proper Linux support. Just &quot;ask on discord&quot;.

1. Mistral<p>2. Llama 2<p>3. Code Llama<p>4. Orca Mini<p>5. Vicuna<p>What can I do with any of these models that won&#x27;t result in 50% hallucinations&#x2F;it recommending code with APIs that don&#x27;t exist&#x2F;it recommending basically regurgitated StackOverflow historical out of date answers (that it was trained on) for libraries that have had their versions&#x2F;APIs change, etc?<p>Can somebody share one real use case they are using any of these models for?

If you just want to try something quick. You can try AskCyph LITE <a href="https:&#x2F;&#x2F;askcyph.cypherchat.app" rel="nofollow noreferrer">https:&#x2F;&#x2F;askcyph.cypherchat.app</a>. It runs AI model natively on browser without having to do any installation, etc.

For those looking for an open source alternative with Mac, Windows, Linux support check out GPT4All.io

Why purple or some shade of purple is the color of all AI products? For some reason, the landing pages of AI products immediately remind of Crypto products. This one does not have Crypto vibes but the colour is purple. I don&#x27;t get why.

Also, if you don&#x27;t know what all the toggles are for, this is a simpler attempt by me: <a href="https:&#x2F;&#x2F;www.avapls.com&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.avapls.com&#x2F;</a>

LMStudio is great to run local LLMs, also support OpenAI-compatible API. In the case you need more advance UI&#x2F;UX, you can use LMStudio with MindMac(<a href="https:&#x2F;&#x2F;mindmac.app" rel="nofollow noreferrer">https:&#x2F;&#x2F;mindmac.app</a>), just check this video for details <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=3KcVp5QQ1Ak" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=3KcVp5QQ1Ak</a>.

Newbie question... Is this purely for hosting <i>text</i> language models? Is there something similar for image models? i.e., upload an image and have some local model provide some detection&#x2F;feedback on it.

I was working on something like this by myself, but ADHD deleted all my motivation. I&#x27;ll definitely want to try this soon, especially if it&#x27;s free!<p>I hope it supports my (3060) GPU, though.

After the latest Chatgpt debacles, the poor performance I&#x27;m getting from 4 turbo, I&#x27;d really like a local version of chatgpt4 or equivalent. I&#x27;d even buy a new pc if I had too.

The settings say saving chats can take 2 GB. Why? What states do chat LLMs have? Isn&#x27;t the only state the chat history text?

How does this App make money?

Wow this is sleek, good job.

The Linux version is not loading models. What’s your experience?

Is this an alternative to privategpt or GPT4All?

RIP Intel users

So if you suspect I am using this for business related purposes you may take any action you want to spy on me? Such Terms. Very Use.

Considering the code is closed source and they can change the ToS anytime to send conversation data to their servers whenever they want, i would like to know what would be the benefit of using this over ChatGPT?

Am I missing something here? I&#x27;m on a recent M2 machine. Every model I&#x27;ve downloaded fails to load immediately when trying to load it. Is there some way to get feedback on the reason for failure, like a log file or something?<p>EDIT: The problem is I&#x27;m on macOS 13.2 (Ventura). According to a message in Discord, the minimum version for some (most?) models is 13.6.

Is anyone using open source models to actually get work done or solving problems in their software architecture? So far I haven&#x27;t found anything near the quality of GPT-4.