Why Small Models Are Quietly Eating Large Ones

The capability gap between the largest frontier models and well-tuned 8B-class models has compressed faster than almost anyone predicted. For a long list of production tasks — classification, extraction, structured-output generation, routing — a fine-tuned small model now matches or beats a generalist frontier API at one-tenth the latency and one-fiftieth the cost.

The implication for product teams is uncomfortable: the default of calling the biggest available model is no longer the safe choice. It's the expensive choice. The work to identify which sub-tasks belong on which model is becoming a core engineering discipline.