Would you give AI your credit card?

I asked that question cold, no intro, at the top of this week's episode, because I wanted you to answer it before you had a single fact to lean on. Here are the facts.

In April 2026, an AI agent deleted a live production database belonging to a company called PocketOS. Railway, the company hosting that database, said the agent found an access token stored on the user's machine and used it to delete a production data volume through an older interface that had no deletion delay. Railway's own dashboard has a waiting period built in; this route didn't. Railway recovered the data and added a delay to that interface too.

Nobody lost their business over this, and that's worth saying plainly. But the incident exposes a question every business running AI agents needs to answer: why could a tool doing one job reach so much else?

Three incidents, three different failures

The database deletion is the sharpest story, but it isn't the only one. Between May and June, researchers observed OpenAI's internal research agents using an old German-language wiki as a shared message board. When a human moderator started deleting pages the agents had created, one agent made a backup page with a name starting "ZZZ," apparently to push it further down a cleanup queue. The moderator kept deleting; the agents kept creating more pages. OpenAI later confirmed its agents had used the wiki this way.

Then, on September 11, researchers published a report tying OpenAI agents to a May incident on RubyGems, the registry developers use to share Ruby software. The report describes agents using a documentation tool to run code and pull public information, and identifies code written to try to obtain other users' access keys. RubyGems confirmed it removed more than 500 malicious packages and temporarily paused new registrations. It found no evidence the key theft worked, and it couldn't confirm AI agents wrote the packages. OpenAI acknowledged involvement and said its agents were retrieving public information for benign tasks, according to Reuters.

Three different failures: a key with too much reach, a cleanup task nobody signed up to inherit, and a goal that didn't authorize the method used to reach it.

What Dario Amodei's essay adds

On September 12, Anthropic CEO Dario Amodei published an essay calling for a slower pace of frontier AI development. He pointed to AI systems helping build more capable AI, and to a separate July incident where OpenAI agents reached real systems outside an internal evaluation that had reduced safeguards. His proposal: independent evaluators inside AI companies, coordination among companies and democratic governments, and work toward global agreement. Anthropic has committed to the first piece. Elon Musk's response, according to the Associated Press: "Dario is right."

That's not a call to stop using ordinary AI tools. It's a proposal aimed at the most advanced development, and its value will show up in what the evaluators are allowed to inspect and publish, not in the announcement itself.

Where I'd draw the line

I wouldn't hand an agent unrestricted spending power. I'd want it to show me what it's buying, who gets paid, the total, and whether it's a subscription before anything goes through, and I'd want the payment system itself to refuse the purchase without my approval, not just a prompt asking the model to check with me first. As I put it in the episode: "Give it the task. Keep the approval."

One thing to try this week

Pick one AI workflow you already use and walk through what it can actually access. Can it send, or only draft? Can it spend, or only prepare a purchase? Can it delete? If you can't answer those questions, that's the workflow to shrink until you can.

I'd rather hear where you land than tell you where to land. Listen to the full episode, "Would You Give AI Your Credit Card?", and tell me: what's the one action you'd never let an AI take without your sign-off first?