--- title: "Why We Run AI in a Container, Not Behind an API Call" canonical: https://supportpages.io/blog/why-we-run-ai-in-a-container updated: 2026-07-24 --- The first version did the obvious thing. Take the diff, take a summary of the repository, put both in a prompt, ask for a help article. What came back was fluent, correctly structured, and told users to click a button called "Save changes" in a settings panel that did not have one. The button was called "Update". There was no settings panel. The flow it described was a reasonable guess at how the feature probably worked, assembled from a diff and priors about how Rails apps usually look. That is the failure mode you have to design against, and it is worse than being wrong. ## Plausible is harder to catch than wrong An obviously broken article gets deleted in review. A plausible one gets published, because a reviewer skimming for errors sees correct grammar, sensible step ordering and terminology that matches the product's general vibe. The specific claim - this button, that label, this order - is exactly the thing a reviewer cannot check without opening the app and walking the flow themselves. Which is the work you bought the tool to avoid. So the reviewer approves it. And then a customer follows step three, finds no Save button, and trusts your documentation slightly less than they did that morning. The root cause is not model quality. It is that a prompt-shaped architecture gives the model no way to check anything. It has a diff and a description. It cannot open `app/views/projects/_settings_panel.html.erb` and read the actual button label, so it does what any system asked to produce confident output from insufficient evidence does. It fills the gap. ## What changes when the model has a filesystem Give it a real clone and the epistemics change completely. Every generation run now happens inside a container with the repository checked out at the relevant commit: ```bash # inside the analyzer container, against a clone at /repo claude -p "/analyze-change pr 4821" ``` The agent can grep the routes file. It can open the view and read the literal string inside the button. It can follow a controller to the template it renders and confirm the step order matches the form. When the article says "click Update", that is because something read the word `Update` out of a file, not because it was a likely word to appear there. The wrapper around it is deliberately thin. A shell script writes context into `/input`, invokes one skill, and reads results back out of `/output`. Rails never talks to a model directly - it writes files, starts a container, and parses what comes back. Every job in the system works this way, which means adding a new capability is writing a new skill rather than a new integration. The isolation is doing real work too. Each run gets a clean filesystem and no access to anything except the clone and the input directory, which matters a great deal when the thing you are executing is a general-purpose agent pointed at a customer's private source code. ## What this architecture costs The honest accounting is that this architecture is expensive in ways the prompt version was not. Container startup is not free, and neither is cloning. A single generation run is measured in minutes rather than the second or two an API call takes. That is fine for a background job that produces an article a human will review tomorrow, and completely unacceptable for anything interactive - which is why the help centre's live question answering does not use this path at all. Two architectures, two very different latency budgets. We did not plan that from the start; we discovered it when the first interactive feature ran into the wall. The worker also needs the Docker socket to spawn sibling containers, which is a meaningful thing to hand a process. If I were starting again I would look harder at a dedicated build service rather than Docker-from-Docker, not because it has bitten us but because explaining it in a security review costs more time than the alternative would have. And image management turned into its own small project. Tags encode a skills version plus a digest of the build context, so jobs can lazily build whatever tag a project resolves to. That is a good design and it is also several hundred lines of infrastructure that exists purely because we chose containers. A prompt-based system would have needed none of it. ## The trade we made Slower, more moving parts, more infrastructure to explain. In exchange, the system can only describe things it has actually read. For documentation, that trade is not close. Speed was never the reason help centres are bad. --- ## About the Author **Simon Moxon** has been building SaaS companies since 2009. He's scaled teams to 20+ people, hit £2m ARR, and learned most of the lessons the hard way. Now he's building [SupportPages.io](https://supportpages.io) - help docs that write themselves when you ship code. He writes about what's changed in SaaS and what hasn't. [Twitter](https://twitter.com/simonmoxon) · [LinkedIn](https://www.linkedin.com/in/smoxon/)