I'm reading a lot of criticism here, some I understand, but honestly, this sounds pretty good for a free LLM service: "Orbit doesn't require account creation or save your session data."<p>Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service - for free. The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default.<p>Not to speak of all the other providers who are either paid or free-but-mine-your-data.
I guess I'm really not this extension's target audience, because somehow Mozilla managed to make me uninstall it in just a few minutes.<p>First, they have forced telemetry. Okay, it's an early release, it's very basic information, they want to understand how they're doing - I don't like it, but I can understand it. Sets a wrong vibe, though - I had to check if it was from Microsoft and not Mozilla. ;-)<p>Then, I figured there is no option to use locally-hosted LLMs, which can be something as minimal as simply allowing to configure custom API URL. Somehow, less and less things about Firefox are tinkerer-friendly than they used to be.<p>That made me wonder if Mozilla used OpenAI-like API, or if they invented their own unique thing for some reason. So I went to look and according to the extension page, it's proprietary ("All Rights Reserved") and I'm too lazy to bother deminifying code from the xpi or remembering how to debug extensions.<p>Finally, '00s have called and said they wanted their weird floating round thingy UI back, and so I had to return it to the store. (I'm kidding, I don't really mind the visual style. The real issue with that thingy was how it floats on the page. Like, why on Earth it isn't normal right-click menu option that doesn't obstruct the view until it's needed? Or a menu on that toolbar button? It's not even a paperclip to be worth it.)<p>And then I realized I somehow missed the big "AI you can trust" header, which should've already been a huge red flag.
Info from the FAQ:<p>- Currently using Mistral 7B, but ability (by Mozilla) to swap the model used to another open source at any point.<p>- Hosted "by Mozilla" on their GCP instance.<p>- No obvious info about what it will cost them to run this since it is free to use.<p>- No training on user data.<p>Like others here, I'm very curious about the cost for Mozilla to run this service. It may be less than it initially appears given the 7B model they chose. I do wish they would focus their efforts on creating a very long-term endowment to pay devs for continued Firefox development in lieu of projects like this given the tenuous situation with their Google funding.<p>I'm not against this kind of thing in theory, but I hope it's being done in a cost-sustainable way.
When running Orbit on the comments in this page:<p>> The Orbit add-on by Mozilla is a new AI-powered tool that summarizes and answers queries about web content, including articles and videos. It uses a Mistral LLM (Mistral 7B) hosted on Mozilla's GCP instance. The add-on is free to use and works on various websites, including Gmail, Wikipedia, NY Times, YouTube, and more. However, some users have raised concerns about the size of the model and its privacy implications, as well as the fact that it requires an internet connection to function. Additionally, some users have suggested that Mozilla should focus on improving the browser itself rather than developing new add-ons.
Some more information from when this was launched 3 months ago on the Mozilla connect board:<p><a href="https://connect.mozilla.org/t5/discussions/try-orbit-by-mozilla-a-new-ai-productivity-tool/m-p/71853" rel="nofollow">https://connect.mozilla.org/t5/discussions/try-orbit-by-mozi...</a>
I installed it and it's a giant floating popup that's permanently on your screen. You can enable the "minimal" theme in the settings to turn it into a smaller-but-still-big pill shape. It doesn't look like there's a way to hide it in a context menu, at least not one I can see.<p>I don't fancy having a random floaty object in the way of my webpages, no thank you.<p>Edit: It appears to go away occasionally. This UX is unclear to me.
Related. This is the bookmark I use to summarize websites using ChatGPT:<p><pre><code> javascript:window.open('https://chatgpt.com/?q=summarize this page in 100 words: '+encodeURIComponent(window.location.href),'_blank');
</code></pre>
Basically it opens a new tab on chatgpt.com with the prompt: "summarize this page in 100 words: URL"<p>Tested on Firefox and Chrome.<p>Some websites block ChatGPT and can't be summarized this way.<p>Works in incognito/anonymous mode and doesn't require a ChatGPT account.<p>You can probably use another AI service with this idea.
I'm encouraged that they're actively exploring this and not shying away from experiments. It seems clear to me that there are areas of Mozilla[0],[1] pushing closer and closer to great local AI integrations doing the kinds of things that I, a browser user, find useful. I went to some of the articles on the hn frontpage and had questions (and followups!) that begat reasonable starting points for further learning.<p>Hopefully they continue to iterate on this with better integration (for instance, moving to a toolbar icon instead of persistent badge on every page) and then make it ~truly privacy respecting by moving locally.<p>[0] <a href="https://github.com/Mozilla-Ocho/llamafile">https://github.com/Mozilla-Ocho/llamafile</a>
[1] <a href="https://github.com/mozilla/translations">https://github.com/mozilla/translations</a>
Huge heading:<p>> <i>Commitment to privacy</i><p>Buried as the last sentence in a collapsed box at the bottom of the page:<p>> <i>For the current version, we are using a Mistral LLM (Mistral 7B) hosted within Mozilla’s GCP instance.</i><p>And why is it "...Mozilla's GCP instance", not "We quietly send all your data to Google servers, and everyone pinkie-swears that's totally privacy-respecting"?
Sounds extremely expensive. How is this paid for?<p>Also, does anyone know if we'd be able to point it to our own LLM instance for the guarantee of our data being secure?
I'm skeptical of a lot of Firefox side quests, but I encourage this one. Orbit could be a stepping stone to an Open Source version of Google Project Mariner. A n un-nerfed built-in AI could turn browsers into true user agents that work on your behalf.
Unfortunately no BYOLLM. Brave supports bringing your own LLM e.g. through Ollama<p>Besides that I'm using AI Summary Helper plugin for Chromium-based browsers <a href="https://philffm.github.io/ai-summary-helper/" rel="nofollow">https://philffm.github.io/ai-summary-helper/</a> which also allows using Ollama (or OpenAI / Mistral), asking questions to articles and inserting summaries right into the DOM (which is perfect for hoarding articles / forwarding them to Kindle)
Installing the extension enables a floating widget in the webpage. Horrible implementation. Couldn't it have been integrated into the browser better, like the Reader button?
Looks like there is no way i can use it without giving up on screen real estate. I'd love to have it pinned in the browser toolbar and I click it when I need it. But it wouldn't allow itself to be hidden and needs to have a floating circle taking up page real-estate.<p>Disabled it.
Am I the only person who isn't hugely interested in summarizing emails? I don't <i>want</i> emails summarized because that bypasses a tell-tale sign that said email isn't worth reading in the first place.
> How does Orbit work?<p>> Orbit is a Firefox add on that uses AI to summarize and answer queries about web content such as articles and videos.<p>> When a user asks Orbit to summarize or query content, Orbit gathers the context (eg. text, images, videos, etc.) of the page the user is viewing and provides a summary or answer. Orbit works on websites including Gmail, Wikipedia, NY Times, YouTube, and more.<p>> For the current version, we are using a Mistral LLM (Mistral 7B) hosted within Mozilla’s GCP instance.<p>It's interesting they're going with Mistral 7B. Is anybody else using Mistral 7B in production? And in what role?<p>I've considered using it for general knowledge type questions, and as a way to classify information, but would have never considered it for summarization type tasks due to it's limited context size (8k).
Mozilla has one job: make the standards compliant browser and work with the relevant groups to foster those standards.<p>Everything else is a waste of time and money and energy.
The only useful (or even paid) browser-integrated AI service I can imagine using would be a browsing history-aware AI chatbot. Essentially, it would just spit out a link from my history based on the context or prompt I give. Since privacy will be a crucial factor, I can imagine building an extension that reads page contents, stores them in a database, and connects to a self-hosted LLM.
I don't understand all the pessimism.<p>- It's clear from user share graphs that Firefox as just a browser is tending towards irrelevance. No amount of "improving the browser" is going to solve the problem.<p>- More fundamentally, the browser is just <i>one</i> portal to the internet / world wide web. With technology <i>outside the browser</i> getting increasingly sophisticated, Mozilla necessarily needs to expand their mandate beyond Firefox in order to serve user needs and influence the landscape. Otherwise we might easily end up in a future where the browser becomes irrelevant and everybody interacts with proprietary large models.<p>- As far as innovation in browser features goes, this seems like a breath of fresh air. Internet users at large deserve access to AI services in a secure and privacy-friendly, and as a pillar of the free web Mozilla is well-placed as a distribution channel to serve these needs. Therefore, this seems like a very good stepping stone / experiment for Mozilla.<p>There will be execution challenges that need to be figured out. AFAIK Mozilla doesn't have the talent+budget for training large AI models, or even for doing intensive product research. So they're going to have to team up with some other AI expertise -- either explicitly or implicitly, by depending on open source models. Regardless, IMHO this is a risk they have to take and figure it out as they go along.
There is a (to me very surprising) typo in the section 'Focus on what matters.' where the AI summary states "... 11 out off[sic] 100 products ..."<p>I don't think I've ever encountered a typo in any of the LLM output I've seen, seems like the exact sort of thing an LLM would be more or less perfect at.
Am I wrong to take this as an indication that this text is actually written by a human as a concise marketing example?
Can I just be blunt here and ask, why the F? No one asked for this. No one wants this. We just want a good, standards-compliant browser that doesn't eat up 8gb of RAM watching a single YouTube video (something that seems to be quite a challenge, though not entirely the browser engine's fault but rather the runaway train that is "capitalistic motives dictate the browser is now an OS and needs all features thereof"))…
For those wondering, this uses cloud-hosted AI models. But it's completely free to use so Mozilla is just paying the cloud bills out of pocket. Maybe not the best idea given their increasingly precarious financial situation?
I have to ask: why, exactly, does it make sense for Mozilla to invest heavily into running expensive servers to run vanity chat bots for people? What's the path here to something which improves their financial situation or browser market share? How isn't this just yet another random service they'll throw money into for a couple of years before shutting down?
Interesting product, very relevant in our time and age.<p>My guess is this could be useful to many "knowledge workers" who constantly have to crawl, translate and find the meaning of the sugar coated landfill that has become most of the web.<p>We are right in the middle of the Tower of Babel story.<p>Seriously if it works reasonably well on legal fine prints I am in.
Mozilla first needs to regain the trust of its users. They aren't privacy-first at all, they run a lot of telemetry on their users. Have you ever checked the number of DNS queries and IP requests that go to Firefox servers every minute? If you haven't, I have, and it is a lot. They literally ping home every minute in the name of network connectivity and other things, if you believe it. I don't mind using Mozilla products, but I just don't trust their motives and data practices.<p>If you want privacy first AI in the browser here are the tools<p><a href="https://ollama.com">https://ollama.com</a><p><a href="https://github.com/n4ze3m/page-assist">https://github.com/n4ze3m/page-assist</a>
Related, I made a small extension for chrome that talks to ollama: <a href="https://github.com/tobias-varden/llama-explain-extension">https://github.com/tobias-varden/llama-explain-extension</a>
In the same area of privacy-aware AI there is Jolla Mind2. It's your own computer, so probably even more trustworthy than Orbit by Mozilla?<p>I have not studied either product in depth, so I am unable to comment on commonalities or differences.<p>Jolla has a mixed track record: They supported some phones over 10 years with decently working software (typing on one of those). They also failed at least once to deliver a crowd-sourced tablet to most of the backers. Not a risk-free choice, but at least someone trying to do the right thing.
Google pays money to Mozilla. Mozilla creates Firefox extension powered by Google AI(?). Mozilla pays Google for cloud services(?).<p>Is that the right flow? FAQ link is broken so I can't tell.
I wonder of this or something like it can be extended to "unclickbait" videos/article titles by spawning a background crawler that reads the article / watches the video and comes back with a resolution to the curiosity bait they used to get you to click. Would save countless hours and make the web less shitty IMO. Plenty of examples available just scrolling through youtube for training data.
such a random product. uses an old tiny model, they are paying the gcp bill, and it's to summarize content?<p>why not use their technical expertise to built an in-browser "<a href="https://big-agi.com/" rel="nofollow">https://big-agi.com/</a>" of sorts where users can paste in their API key and use bleeding edge models in combination with the browser's data which they could expose and manipulate as the creators of Firefox!<p>this product seems really random and quite frankly weird.
I wish they would copy this one <a href="https://github.com/webmachinelearning/prompt-api">https://github.com/webmachinelearning/prompt-api</a> and include a few options for small self-hosted LLMs using WebGPU or some built-in accelerated AI for Firefox.
I think Mozilla is doing a great job.<p>I really don't get the comments that they should not focus on anything else than the current browser.<p>If other browsers start adding llms, I bet those same people will start complaining that Firefox is outdated compared to those browsers in about a year.
Just keep it as optional extension and that will be perfectly fine. They should make it compatible with other browsers too. Considering llamafile project, maybe there will be an option for offline assistant, where user will be able to select their preferred model?
Hopefully Mozilla will eventually come up with a local-AI in the browser model, like the one currently being explored by Google Canary called Gemini Nano (which Google of course doesn't seem to want to make available in Chromium though).
It feels very resistant to doing anything other than summarizing. Even when you ask questions for details, the answer is always in the form of a simple summary.
From a product standpoint, I am curious if sidebar is the right way to integrate AI features in the browser? Did anyone see any better integrated solutions?
If you're interested in this, you might like <a href="https://nuggetize.com/" rel="nofollow">https://nuggetize.com/</a> which works without installing anything and with any browser.
I use Librewolf (based on Firefox), but about once a year I open Chrome for some shitty website that only works on Chrome. And I use Chrome for a few minutes.<p>It shocks me every time just how fast Chrome is. It is legitimately a superb piece of software. Going to Librewolf after feels like going back ten years in hardware.<p>Can we please start spending some money to make Firefox better? Instead of whatever Mozilla is currently doing?<p>Firefox Quantum was great. But why stop? Just keep doing that! It's the only thing you should be doing!
Why does Mozilla always spin their wheels releasing random side projects? Cloud-hosting an outdated tiny LLM that you can't swap out or run locally, to do basic summarization? This just doesn't feel like an area of strategic focus that makes sense for Mozilla. GPUs are expensive, talent to do inference well is expensive, and the actual product they're shipping seems pretty marginally useful at best.<p>If they shipped vertical tabs I'd probably switch back to Firefox. But that would require focusing on actual browser UX instead of random offshoots. What's their actual product differentiation from Chrome these days?
Just seeing the title as announcing some new mozilla service my first thought was "What personal data does this new mozilla feature send to cloudflare?" -- turns out the answer was emails and documents but to their own google cloud accounts rather than cloudflare.<p>Of course, no option to use a local model even though the one they're using is small enough that its perfectly reasonable to use locally. Even on a cell phone.
Given the broader Mozilla foundation's political biases, will these assistants be censored or heavily curated? I am reminded of when Mozilla chose to ban Dissenter, a free speech plugin powered by Gab, from its store. I've never used it but I found it distasteful that a company working on a basic utility program decided to become political. I don't understand why they cannot just focus on the basics and get those right. Still waiting for proper vertical tabs.