We hit an at-least-once delivery trap. Here is how we fixed the race conditions.

If you are running an event-driven architecture with message queues like Kafka, you already know the drill, scalability is awesome, but network partitions, out-of-order delivery, and duplicate message...