The "Mistral Pixtral multimodal model" really rolls off the tongue.<p>> It’s unclear which image data Mistral might have used to develop Pixtral 12B.<p>The days of free web scraping especially for the richer sources of material are almost gone, with anything between technical (API restrictions) and legal (copyright) measures building deep moats. I also wonder what they trained it on. They're not Meta or Google with endless supplies of user content, or exclusive contracts with the Reddits of the internet.
Couple notes for newcomers:<p>1. This is a VLM, not a text-to-image model. You can give it images, and it can understand them. It doesn't generate images back.<p>2. It seems like Pixtral 12B benchmarks significantly below Qwen2-VL-7B [1], so if you want the best local model for understanding images, probably use Qwen2. If you want a large open-source model, Qwen2-VL-72B is most likely the best option.<p>1: <a href="https://qwenlm.github.io/blog/qwen2-vl/" rel="nofollow">https://qwenlm.github.io/blog/qwen2-vl/</a>
Mistral being more open than 'openai' is kind of a meme. How can a company call itself open while it refuses to openly distribute it's product and when competitor are actually doing it.
Related earlier:<p><i>New Mistral AI Weights</i><p><a href="https://news.ycombinator.com/item?id=41508695">https://news.ycombinator.com/item?id=41508695</a>
I’d love to know how much money Mistral is taking in versus spending. I’m very happy for all these open weights models, but they don’t have Instagram to help pay for it. These models are expensive to build.
A question for sd lora trainers, is this usable for making captions and what are <i>you</i> using, apart from BLIP?<p>Also, can your model of choice understand your requests to include/omit particular nuances of an image?
Could this be used for a selfhosted handwritten text recognition instance?<p>Like writing on an ePaper tablet, exporting the PDF and feed this into this model to extract todos from notes for example.<p>Or what would be the SotA for this application?
12B is pretty small, so I’m doubting it’ll be anywhere close to internvl2 however mistral does great work and likely this model is still useful for on device tasks