The Hidden NPU Fallback: Why Eval Tests Pass While Production Fails

ONNX Runtime's QNN execution provider silently offloads unsupported operations from Hexagon NPU to CPU—a fallback that cloud eval tests miss but production reveals through 3x latency spikes. Standard evaluation cannot observe execution routing, leaving the problem invisible until real traffic hits. The fix demands hardware-based CI assertions: actual device testing, median/variance latency gates, and ORT profiling analysis to verify NPU FLOP coverage.