ARIA: An Open Multimodal Native Mixture-of-Experts Model

&gt; outperforms Pixtral-12B and Llama3.2-11B<p>Cool, maybe needs of a better name for SEO though. ARIA has meaning in web apps.

In an MoE model such as this, are all &quot;parts&quot; loaded in Memory at the same time, or at any given time only one part is loaded? For example, does Mixtral-8x7B have the memory requirement of a 7B model, or a 56B model?

I’m curious how it compares with recently announced Molmo: <a href="https:&#x2F;&#x2F;molmo.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;molmo.org&#x2F;</a>

Model should be available for testing here [0], although I tried to upload a video and got an error in Chinese, and whenever I write something it says that the API key is invalid or missing.<p>[0] <a href="https:&#x2F;&#x2F;rhymes.ai&#x2F;" rel="nofollow">https:&#x2F;&#x2F;rhymes.ai&#x2F;</a>

This looks worth a try. Great test results, very good example output. No way to know if it’s cherry picked &#x2F; overtuned without giving it a spin, but it will go on my list. Should fit on an M2 Max at full precision.

<i>&quot;Here, we provide a quantifiable definition: A multimodal native model refers to a single model with strong understanding capabilities across multiple input modalities (e.g. text, code, image, video), that matches or exceeds the modality specialized models of similar capacities.&quot;</i>