Lighthouse: Sparse Training That Recovers Full Dense Performance

Dense attention's O(N²) complexity forces frontier models to scale GPU clusters just to handle million-token contexts—a computational ceiling that severely limits deployment. While prior sparse attention methods promise training speedups, they fail at the critical step: recovering as dense models at inference without major performance loss, forcing researchers to either retrain or accept degraded performance. Lighthouse resolves this fundamental trade-off through symmetric pyramid pooling and a two-stage recovery guarantee, achieving 1.4–1.7× pretraining speedup while outperforming dense-from-scratch baselines and introducing zero inference overhead.