Here's a great example of the kind of story I'm looking for

Another good one: fine-tuning a model to work better with a niche programming language Jane Street did this as well for OCaml: https://www.youtube.com/watch?...

Getting <500ms response times for a UI that updates as you type seems like a very strong justification for fine-tuning a small, fast custom model

v0 is running on some credible looking fine-tuned models, specializing in their Next.js stack

Here's a neat one: warehouse automation, using a vLLM (fine-tuned Gemini 2.5 Flash, big cost saving over 2.5 Pro) to check that containers on a conveyor belt are carrying the expected items

Shopify have deployed fine-tuned vision LLMs based on "LlaVA 1.5 7B, LLaMA 3.2 11B, and currently Qwen2VL 7B" to help process product photos at scale

@simonw It's useful to reduce errors in various tasks in fields with a lot of specialist terminology.

@xlr8harder Have you see any good case studies of that working in practice?

@simonw Embeddings count?

@rjurney Definitely interesting, see

@simonw Does this one count? Not yet commercial value but scientific value from fine-tuning Gemma 2 27B. No vocab change, just alter the next token probability distribution. https://x.com/sundarpichai/sta...

@oscarle_x Yes that is a great example!

@simonw At @ShopifyEng we have many use-cases. From product taxonomy here: https://shopify.engineering/le... To bespoke SQL generation for our agent. See ICML https://icml.cc/virtual/2025/4...

@acaste10 @ShopifyEng Neat example, thanks! Are you evaluating Queen 3 VL now to replace Qwen2VL?

@simonw One that’s very useful is a quick fine tune towards using specific tools. If you have a lot of tools and need very reliable results with a smaller model, fine tuning for your specific tools is a gold mine. It’s not straightforward, has complexity to it but the reward is very

@Matrix_Memories Have you see the work well yourself?

@simonw Hi Simon, I’m beginning to work on one now. My research says it won’t only produce value beyond current hosted models but is very much needed. Time to prove that. I’ll be happy to share the perf numbers vs hosted models once it’s done. Any other information you’re looking for?

@communicating A bunch of things! - How did you evaluate it was worth taking on? - How did you collect the training data for it? - How did you confirm it was a genuine improvement over the base model? - Did you explore alternative approaches (switch base model, prompt optimization) first?

@simonw Have you seen:

@agenticist I hadn't seen that one I'm particularly interested in case studies that aren't directly from vendors who sell fine tuning consulting services, but this one is still an interesting signal

@simonw Yes. Done it via RAG. Deployed a few private LLM's to enterprise customers as well.

@satvikmaker I don't count RAG as the same thing as fine tuning - by fine tuning here I mean training new model weights (eg with something like LoRA)

@simonw fine tunes of email replies to sound like/write like the author.

@i_am_brennan Have you seen that actually work?