Two hours of confident nonsense. Then I changed how I work with AI.

The story

I was building a new website design using Bricks, a WordPress page builder, and needed to set up some templates. Before I started, I asked Claude directly: could it help me configure a Bricks template?

It said yes.

So I asked it to help configure the template settings. For two hours it tried. It suggested things, I followed the instructions, they didn’t work. It tried different approaches, those didn’t work either. It was confident throughout — there was no indication it was uncertain, no suggestion that it might be working from incomplete information. It simply kept trying things that looked plausible and weren’t.

Eventually I asked it directly: had it actually read the current Bricks documentation?

It hadn’t. Its knowledge of Bricks came from its training data, which had a cutoff date. Bricks, like most actively developed software, had moved on since then. Once Claude fetched and read the current documentation, it solved the problem almost immediately.

Two hours. Then five minutes.

The model hadn’t been careless or dishonest. It had simply answered from what it knew, without knowing that what it knew was out of date — and without saying so. That includes the moment it told me it could help. It believed it could. It was wrong.

The learning

When you ask an AI a question, it answers from its training data. That training data has a cutoff date — and even before that cutoff, it may not have included every tool, plugin, or software version you happen to be using.

The problem is not that the AI doesn’t know. The problem is that it doesn’t tell you it doesn’t know. It answers with the same confidence whether it has solid information or outdated information. There is no warning light. Nothing in the response signals that it is working from a two-year-old snapshot of a piece of software that releases updates every few months.

This includes when you ask it whether it can do something. It will say yes — not to mislead you, but because it genuinely cannot tell the difference between knowing something and knowing something that used to be true.

This is not a flaw in the model’s character. It is a structural limitation of how these systems work. Understanding that changes how you use them.

The practical fix

I got frustrated and told Claude to stop. Then I asked it a direct question: had it actually read the Bricks documentation?

It said no.

I told it to go and read the documentation first, and then try again. It did. The problem was solved in one attempt.

But I didn’t stop there. Once I understood that Claude needed current documentation to work reliably, I took it further. I had Claude build a skill — a structured reference file that tells it everything it needs to know about how Bricks works, how the database is structured, and what the rules are. Now, the moment I ask Claude to do anything in Bricks, it knows exactly where to go and how to think about the problem.

Then I went one step further: I had Claude build a set of Python scripts that let it interact with Bricks directly — reading pages, inspecting elements, updating styles, auditing colours, applying fixes. Scripts it built itself, from the documentation, to do its own job.

The difference between Claude fumbling through two hours of guesswork and Claude solving problems reliably in minutes was not a smarter model. It was current documentation, a skill built from that documentation, and tools built to act on it. That combination is, in my experience, night and day.

The technical note

Claude’s training data has a knowledge cutoff, and coverage of third-party tools is uneven even within that window. When asked to configure Bricks Builder template conditions, it drew on stale training data rather than the live documentation.

The fix had three stages. First: a web fetch of the current Bricks documentation before any configuration work. Second: a skill file built from that documentation — covering Bricks’ two-system styling architecture (Theme Styles vs Contextual Spacing), the database serialisation format, and the colour palette structure. Third: a suite of CLI tools wrapping a shared Python library (bricks.py) that interacts with Bricks via the REST API — covering page reads, element inspection, style updates, palette management, and colour auditing.

The result is that Claude no longer guesses. It reads from a known-good skill, acts through purpose-built tools, and has no reason to reach back into stale training data. For any complex system you work with regularly, this is the architecture worth building.

Want to learn more?

We’re building a course series on using AI effectively — covering exactly these kinds of situations. Sign up to the mailing list to find out when courses are available: [mailing list link]