Kasey Zhang

Don't use structured output mode for reasoning tasks.

We’re open sourcing Osmosis-Structure-0.6B: an extremely small model that can turn any unstructured data into any format (e.g. JSON schema).

Use it with any model - download and blog below! 

Ollama: https://ollama.com/Osmosis/Osmosis-Structure-0.6B
Hugging Face: https://huggingface.co/osmosis-ai/Osmosis-Structure-0.6B
Full Write-Up: http://osmosis.ai/blog/structured-outputs-comparison

Structured output mode hurts performance on reasoning tasks.

This is because structured output mode enforces a schema and stops the model from thinking ‘freely’.

So rather than having an LLM self-enforce an output schema, we trained Osmosis-Structure-0.6B to do it instead! 

We trained the model on 500K+ tasks where the model needed to format data into a structured output, and a 4X jump in accuracy on benchmarks using Claude Sonnet 4. 

Interestingly, o3 performed well even with structured output mode.

We speculate that this may be due to a double pass - i.e. o3 generates an output, and then 4o-mini (or another small model) is used to validate / structure the output, similar to Osmosis-Structure-0.6B.

We came to this hypothesis since GPT-4.1's structured output time to completion is faster compared to its unstructured completions.

In comparison, o3's time to completion for structured and unstructured calls was similar - sometimes even longer for structured outputs.

Try it out and let us know what you think!