SubQ Claims 12 Million Token Context with 52x Speed Gain Over FlashAttention
A new sparse-attention architecture called SubQ promises the first fully sub-quadratic frontier model, processing 12 million tokens at a fraction of the compute cost.
A model called SubQ is claiming a major architectural breakthrough: the first frontier-class LLM with fully sub-quadratic attention, enabling a 12-million-token context window. As @alex_whedon announced, SubQ uses a "Selective Sparse Attention" (SSA) architecture that focuses compute on relevant token relationships, achieving "nearly 1,000x less compute" than standard attention mechanisms and 52x faster inference than FlashAttention at one million tokens — all at less than 5% of Claude Opus's per-token cost.
Unlock the full briefing
Get every story in today's briefing, the full archive, and the daily AI intelligence brief.
All stories today
Full archive
Daily brief
Cancel anytime. Payments powered by Stripe.