Why teaching AI agents to use tools keeps blowing up in training

Multi-step reinforcement learning for tool-using AI agents collapses mid-training not because the model loses its skills, but because the probabilities of a few structural control tokens spike and scr...