Andrei David

🚀 Watch how I connected Chrome's AI API to @ollama, enabling ANY web app built for Chrome's local Gemini to seamlessly work with open-source LLMs! A🧵on making AI in Chrome more open and flexible... 

Chrome 127+ introduced an experimental Prompt API for local AI experimentation using Gemini Nano. @googlechrome &amp; @ChromiumDev are pushing boundaries with built-in AI capabilities.

Why is this exciting?🔥The Prompt API lets developers send natural language instructions to locally running models, accelerating AI use-case discovery and informing future task-specific APIs.

Shoutout to @huggingface for their amazing tutorial on running Gemini Nano locally: https://huggingface.co/blog/Xenova/run-gemini-nano-in-your-browser 

But here's where it gets interesting...🎯What if we could use this same API with ANY model? That's where @ollama comes in!

I modified Chrome's AI implementation to talk to Ollama instead of Gemini Nano. The result? Any web app built for Chrome's AI API just works with Ollama's models! 🪄

Why is this important? 🤔
- Run any Ollama model locally
- Keep your data private
- Choose your own models
- Zero changes needed for web apps

Big thanks to @ollama for their amazing work making local LLMs accessible. Their API design made this integration surprisingly straightforward! 🙏

Speaking of local AI champions... @exolabs have been a huge inspirations in the "AI at home" movement. Their work on making AI accessible on personal devices is groundbreaking.

@alexocheema - quick question: Should we expand this POC to work with an Exo cluster? The possibilities for distributed local AI in Chrome seem exciting!🤔

How it started😄
https://x.com/AndreiDavid/status/1880742968912654548