Cutting ASR Inference Cost with NVIDIA MPS on Amazon EC2

When an ASR pipeline is pushed to production, the interesting question is not only how fast it runs, but how much throughput you can extract from each GPU before latency starts to break. In the setup ...