I cost my client $60,000

Last year I made the biggest mistake of my career so far.

I was processing analytical data for a client. About 8 to 12 million records daily. The job required quering records repeatedly because there was no way to know the volume in advance. Routine stuff, or so I thought.

The one clue I had: I needed to spin up 70 machines at once, all calculating for several hours. If this had been running on our own database, fixed pricing, no problem. But this was Google Cloud. Resources scaled up automatically. Everything kept responding fine. No slowdown, no warning, no alert.

I found out something was wrong when a payment reminder came through — the company card got declined. I ignored it. Not my department.

Three days later, the actual invoice arrived. We normally pay about $70 a month. This one was $57,000.

Complete shock. I went cold.

The client happened to be on vacation by the sea. I called him, explained the situation, and said I'd file a dispute with Google. He took it better than I expected. Maybe the sun helped.

The dispute took 6 months. Google eventually compensated about $13,000. The rest was on us.

No one tells you about these moments. You hear about the wins, the launches, the milestones. Not the phone call where you tell a client you just burned through their cloud budget in a few days. Not the 6 months of waiting, checking email, hoping Google says yes.

It happened because cloud pricing is a black box. No usage cap, no notification threshold, nothing. The system just scales and bills. By the time you notice, the damage is done.

I set billing alerts on every cloud project since. Hard caps where possible. But honestly, the first spike alone was around $24,000 and Google's billing data is so delayed that no alert would have caught it in time. By the time the dashboard updates, the money is already gone.

But the real lesson isn't technical. It's that these things happen. Not because you're careless. Because you're human, operating in systems designed to be invisible until they explode.