DeepMind Shows Smaller Models Generate Better Synthetic Reasoning Data Than Larger Ones

A new DeepMind paper demonstrates that compute-matched sampling from smaller models produces higher-quality synthetic training data, with gains reaching 31.6% — a finding that could reshape how labs approach data generation.