Microsoft Phi-3 Cookbook

Looks like some of the docs are generated by an llm. I see pictures with typos and imagined terms, incomplete texts etc., I wonder to what extent we can trust rest of the docs.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;microsoft&#x2F;Phi-3CookBook&#x2F;blob&#x2F;main&#x2F;md&#x2F;04.Fine-tuning&#x2F;LetPhi3gotoIndustriy.md">https:&#x2F;&#x2F;github.com&#x2F;microsoft&#x2F;Phi-3CookBook&#x2F;blob&#x2F;main&#x2F;md&#x2F;04.F...</a>

You can interact with the new Phi-3 vision model on this page (no login required): <a href="https:&#x2F;&#x2F;ai.azure.com&#x2F;explore&#x2F;models&#x2F;Phi-3-vision-128k-instruct&#x2F;version&#x2F;1&#x2F;registry&#x2F;azureml" rel="nofollow">https:&#x2F;&#x2F;ai.azure.com&#x2F;explore&#x2F;models&#x2F;Phi-3-vision-128k-instru...</a>

&quot;We are introducing Phi Silica which is built from the Phi series of models and is designed specifically for the NPUs in Copilot+ PCs. Windows is the first platform to have a state-of-the-art small language model (SLM) custom built for the NPU and shipping inbox. Phi Silica API along with OCR, Studio Effects, Live Captions, Recall User Activity APIs will be available in Windows Copilot Library in June. More APIs like Vector Embedding, RAG API, Text Summarization will be coming later.&quot;<p>2024: the year of personal computers with neural processing units running small language models<p>How do NPU&#x27;s work? Who builds them and how are they built? Are they capable of running a variety of SLM-like firmware?

The initial model release had a terrible, frequent, issue with emitting the wrong &quot;end of message&quot; token, or never emitting one.[1] That is a <i>very</i> serious issue that breaks chat.<p>The ones from today still have this issue.[2]<p>Beyond that, they&#x27;ve been pushing new ONNX features enabling LLMs via Phi for about a month now. The ONNX runtime that supports it still isn&#x27;t out, much less the downstream integration of it into the iOS&#x2F;Android runtimes. Heck, the Python package for it isn&#x27;t supported anywhere but Windows.<p>It&#x27;s absolutely wild to me that MS is pulling this stuff with ~0 discussion or reputation repercussions.<p>I&#x27;m a huge ONNX fan and bet a lot on it, it works great. It was clear to me about 4 months ago that Wintel&#x27;s &quot;AI PC&quot; buildup meant &quot;ONNX x newer Phi&quot;<p>It is very frustrating to see an extremely late rush, propped up by potemkin blog posts that I have to waste time to find out are just straight up lying. Burnt a lot of goodwill that they worked hard to earn.<p>I am virtually certain that the new Windows AI features previewed about yesterday are going to land <i>horribly</i> if they actually try to land them this year.<p>[1] <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;microsoft&#x2F;Phi-3-mini-4k-instruct-gguf&#x2F;discussions&#x2F;8#662e705ea47b4da4b295db25" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;microsoft&#x2F;Phi-3-mini-4k-instruct-gguf...</a> [2] <a href="https:&#x2F;&#x2F;x.com&#x2F;jpohhhh&#x2F;status&#x2F;1793003272187351195" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;jpohhhh&#x2F;status&#x2F;1793003272187351195</a>

It looks like the Phi-3 Vision model isn&#x27;t available in GGUF or ONNX. I was hoping there was a GGUF I could use with llamafile.

The bigger news is that Phi-3-Small, Phi-3-Medium and Phi-3-Vision were finally released

I installed Phi:medium last night on my Mac using Ollama and, subjectively, it looks good. I was surprised of the claim the it was better than mistral-8x7B.<p>I largely ignore benchmarks now, but on the other hand, while trying many models myself is easy for simple tests, really using a LLM for an application is a lot of work.

Slightly off topic: what’s the reasonably smallest LLM model i can use to do language processing and rewriting of a large library of word documents? For the purposes of querying information and regurgitating out summaries or detailed information?<p>My use case is very simple: take 1000 word documents filled with two to three pages of information and pictures. And then output a set of requested information via prompting. Is there something off the shelf? Or do I have to make this?

Wow, actually this cookbook is really bad? I expected something like the OpenAI or Anthropic cookbooks, but this seems to be some AI generated low-quality content without any code examples or interesting examples?<p>The Phi-3 models are great though, especially the vision model has great potential for low latency applications (like robotics?)...

<a href="https:&#x2F;&#x2F;huggingface.co&#x2F;collections&#x2F;microsoft&#x2F;phi-3-6626e15e9585a200d2d761e3" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;collections&#x2F;microsoft&#x2F;phi-3-6626e15e9...</a>, all of these models except Phi-3 mini are new.

Was playing around with this model - why does it return 2 or 3 responses when I ask it for one? I asked it for a json response and it generates 2 or 3 at a time. What&#x27;s with this.