The Stability Paradox: Why Exploration Noise Improves DQN Evaluation

This finding challenges a foundational assumption in reinforcement learning: that evaluation requires deterministic policies with noise disabled. The developer discovered that disabling exploration noise created 'policy traps'—states where nearly identical Q-values force the agent into consistent failure loops. Re-enabling noise during evaluation resolved these collapse points, dramatically improving performance stability, suggesting ML practitioners may need to reconsider how they evaluate RL agents.