Don't use structured output mode for reasoning tasks. We’re open sourcing Osmosis-Structure-0.6B: an extremely small model that can turn any unstructured data into any format (e.g. JSON schema). Use it with any model - download and blog below!
Ollama: https://ollama.com/Osmosis/Osm... Hugging Face: https://huggingface.co/osmosis... Full Write-Up: http://osmosis.ai/blog/structu...
Structured output mode hurts performance on reasoning tasks. This is because structured output mode enforces a schema and stops the model from thinking ‘freely’. So rather than having an LLM self-enforce an output schema, we trained Osmosis-Structure-0.6B to do it instead!
We trained the model on 500K+ tasks where the model needed to format data into a structured output, and a 4X jump in accuracy on benchmarks using Claude Sonnet 4.
Interestingly, o3 performed well even with structured output mode. We speculate that this may be due to a double pass - i.e. o3 generates an output, and then 4o-mini (or another small model) is used to validate / structure the output, similar to Osmosis-Structure-0.6B.
We came to this hypothesis since GPT-4.1's structured output time to completion is faster compared to its unstructured completions. In comparison, o3's time to completion for structured and unstructured calls was similar - sometimes even longer for structured outputs.
Try it out and let us know what you think!