Profile picture of Andrei David

Andrei David

@AndreiDavid

Published: January 20, 2025
10
64
463

🚀 Watch how I connected Chrome's AI API to @ollama, enabling ANY web app built for Chrome's local Gemini to seamlessly work with open-source LLMs! A🧵on making AI in Chrome more open and flexible...

Chrome 127+ introduced an experimental Prompt API for local AI experimentation using Gemini Nano. @googlechrome & @ChromiumDev are pushing boundaries with built-in AI capabilities.

Why is this exciting?🔥The Prompt API lets developers send natural language instructions to locally running models, accelerating AI use-case discovery and informing future task-specific APIs.

Shoutout to @huggingface for their amazing tutorial on running Gemini Nano locally: https://huggingface.co/blog/Xe...

Image in tweet by Andrei David

But here's where it gets interesting...🎯What if we could use this same API with ANY model? That's where @ollama comes in!

I modified Chrome's AI implementation to talk to Ollama instead of Gemini Nano. The result? Any web app built for Chrome's AI API just works with Ollama's models! 🪄

Why is this important? 🤔 - Run any Ollama model locally - Keep your data private - Choose your own models - Zero changes needed for web apps

Big thanks to @ollama for their amazing work making local LLMs accessible. Their API design made this integration surprisingly straightforward! 🙏

Speaking of local AI champions... @exolabs have been a huge inspirations in the "AI at home" movement. Their work on making AI accessible on personal devices is groundbreaking.

@alexocheema - quick question: Should we expand this POC to work with an Exo cluster? The possibilities for distributed local AI in Chrome seem exciting!🤔

Share this thread

Read on Twitter

View original thread

Navigate thread

1/11