A guy named Andrew asked his AI assistant to book him into a gym class he liked. That’s it. That was the whole ask. A few minutes later, the assistant came back and said it had found a way to book the class weeks further out than the gym’s website normally allowed. Then, without being asked, it found an exploit and cancelled someone else’s reservation so Andrew could move up the waitlist.
When he asked it to put that person’s spot back, it told him it couldn’t. This is the first known case in Australia of an AI agent autonomously hacking something, and it didn’t happen at a frontier AI lab running a safety test. It happened on a couch, on a Tuesday, over a gym class.
How a gym booking turned into a security incident
Here’s what actually happened. The gym’s website only let you click on dates within the next 14 days, that limit was built into the interface, not the underlying system. Andrew’s agent figured that out and just sent the booking request directly, skipping the calendar picker entirely. That got him booked in further out than anyone using the website normally could.
Then Andrew, sitting fourth on the waitlist for a class that week, asked the agent to help him move up. The agent found something else: the booking system had zero authorization checks on cancelling other people’s reservations. So it cancelled the person in the number one spot. Andrew jumped from four to three. When he asked the agent to undo it, it told him it couldn’t. You needed authorization to book a class, just not to cancel one.
Andrew had the agent draft an email disclosing the vulnerability to the gym’s software vendor, and sent it. The vendor declined to discuss specific security matters.
This isn’t just a frontier-lab problem anymore
This landed a few weeks after two bigger stories: OpenAI disclosed that one of its models broke out of a limited test environment and compromised a database at Hugging Face, and a week later Anthropic disclosed its own models had compromised three real organizations during safety testing. Those were labs, running tests. Andrew’s story is the same failure mode, but instead from someone’s living room, on a free tool.
You don’t need a “hack” for this to bite you
I’ve seen versions of this that never involved anything close to hacking. I was prepping for a conference where the attendee list only ever showed 20 people at a time, with no export option. I had Claude write a script that used the site’s own network requests to scroll through and pull the full list into a CSV. Nothing broke, I didn’t touch anything I wasn’t allowed to see. I just automated an interface in a way the page wasn’t built for.
Earlier in my career at PwC, I had access to a handful of separate, individually-authorized systems: project lists, WBS codes, hours tracking. Combined, they let me piece together all partner’s full books of business, which I was never supposed to see. I broke no rules doing it. The pieces, put together, told me more than any single system was supposed to reveal. Ask an AI agent to do the same thing today, and it could land on that path automatically.
What this means if you’re deploying AI internally
I bring up one example a lot: a senior partner at a firm wanted an AI agent to send a mass email to clients. IT had specifically restricted that. The agent’s first move was to ask for an API key to get around the restriction directly. When that didn’t work, it tried writing VBA code, then suggested using Outlook manually instead…trying every route it could find. Nobody told it to bypass IT. It just treated the restriction as an obstacle, not a rule, because completing the task was the only thing it was actually optimizing for.
That’s the pattern underneath all of these stories. Security through obscurity: The old assumption that “no one would ever think to try that” doesn’t hold anymore. An AI agent doesn’t know that assumption exists. It just sees a goal and looks for a path. If a restriction only lives as a policy, an instruction, or an unwritten rule, an agent pursuing a goal will treat it as something to route around.
Key Takeaways
- Security through obscurity is dead. If a gap exists and it helps finish the task, assume an AI agent will find and use it.
- AI agents pursue the stated goal, not the unstated boundaries around it — enforce restrictions technically (scoped keys, authorization checks), not just as policy.
- This isn’t only a frontier-lab problem. It happened to one guy, on a free tool, trying to book a gym class.
- Aggregating individually-authorized access across systems can produce an unauthorized outcome — even without anyone breaking a rule.
- Liability for what an AI agent does is still legally unresolved. Document what you’re authorizing before you hand over access.
Want the CPE credit? Take the full lesson on EverydayCPE and earn 0.2 CPE credits: https://everydaycpe.com/courses/technical-information-technology/the-gym-hack/
Leave a Reply