TNG Technology Consulting GmbH

Today we release DeepSeek-R1T-Chimera, an open weights model adding R1 reasoning to @deepseek_ai  V3-0324 with a novel construction method.

In benchmarks, it appears to be as smart as R1 but much faster, using 40% fewer output tokens.

The Chimera is a child LLM, using V3s shared experts augmented with a custom merge of R1s and V3s routed experts. It is not a finetune or distillation, but constructed from neural network parts of both parent MoE models.

A bit surprisingly, we did not detect defects of the hybrid child model. Instead, its reasoning and thinking processes appear to be more compact and orderly than the sometimes very long and wandering thoughts of the R1 parent model.

Model weights are on @huggingface, just a little late for #ICLR2025. Kudos to @deepseek_ai for V3 and R1!

Link to model weights on @huggingface: https://huggingface.co/tngtech/DeepSeek-R1T-Chimera

For people without the infrastructure to run the 671B model, we can provide test access to our R1T cluster (DM us)