Benchmarks
Measured on our own servers.
Training and serving results, on the hardware your models would run on.
On 675 reserved tasks from task families it never trained on: 31% → 78%.
Public APIs for the same model run at about 200 tokens a second.
5 of 5
Training
Reinforcement learning on science tasks
A small open model after 100 steps of reinforcement learning.
28% → 95%
held-out tasks completed
Training
New kinds of task
The same model on kinds of task it never trained on.
31% → 78%
of 675 reserved tasks completed
Training
Reinforcement learning on production traffic
Gained by a small model trained on real production requests.
+9 points
against a frontier model
Serving
Single-stream speed
DeepSeek V4.1 Flash on our servers. Public APIs for the same model run at about 200.
1,073
tokens a second
Serving
Many customer models on one GPU
With 128 customer models loaded together on a single GPU.
14×
the stock engine's speed
OS3 measurements, October to November 2026, on NVIDIA B200 GPUs. Public API speeds: Artificial Analysis and OpenRouter listings, November 2026. Results on your tasks will differ.