Unified Expert Pools Enable Leaner, Safer Language Models

Recent breakthroughs in mixture-of-experts architectures tackle two critical challenges: parameter bloat and inference safety. UniPool demonstrates that a single shared expert pool can match conventional performance while cutting parameters by one-third to two-thirds, breaking the assumed coupling between model depth and expert count. MASCing meanwhile substantially improves safety by steering expert selection at inference time, raising adversarial defense rates from 52.5% to 83.9%.