Single Models, Multiple Perspectives: Distilling Multi-Agent Debate into Efficient LLMs

Multi-agent debate substantially improves LLM reasoning but incurs heavy computational costs through transcript generation. This work distills that debate intelligence into single models, matching or exceeding performance while reducing token usage by 93%. Beyond the efficiency breakthrough, researchers reveal the mechanistic basis—interpretable agent-specific subspaces in activation space—enabling practical control of internalized behaviors with implications for AI safety and alignment.