My AI bot burned through my API budget overnight. So I built an open-source tool to make sure it never happens again.

The author's autonomous AI news bot exceeded its API budget due to an in-memory budget tracker resetting after a container restart, causing unchecked Gemini API calls. Finding that existing LLM monitoring tools like Langfuse and Helicone lacked quality assertion and regression detection, the author built toad-eye, an open-source observability toolkit built on OpenTelemetry that provides pre-configured Grafana dashboards, Jaeger tracing, and budget guard features.