GLM-5 대규모 서비스 중 발견한 레이스 컨디션 버그 수정기 — Coding Agent 추론 인프라의 Scaling Pain

This case study reveals critical infrastructure challenges that emerge when scaling AI inference to hundreds of millions of requests daily. By repurposing Speculative Decoding metrics for real-time quality monitoring and implementing LayerSplit for distributed KV Cache management, Z.ai demonstrates that modern large-scale AI systems require rigorous system engineering to guarantee both performance and correctness. The findings underscore how scaling laws push model capabilities and infrastructure to their limits, where output accuracy cannot be traded for throughput.