# A Change Is Not Done When the Command Succeeds *A write can succeed without being read. A deploy can succeed without changing what's live. Here's the check we run to catch the difference, and the real bug it just found in our own product.* addAI.dev · Field Notes By [Jay](mailto:jay@addAI.dev) · September 2026 `#TechDeepDive` `#AITools` The write completed. The deploy passed. The import returned success. None of that tells you whether the thing downstream actually got what you meant to send it. That gap keeps showing up in real work. A file can be written and never read. A deployment can succeed while the live site still serves the old version. A DNS zone can import cleanly while the public internet keeps resolving something else entirely. So we built a habit around a simple question: can we check the same change from both ends, independently, instead of trusting the command that made it? We call this habit **Round Trip Validation**, and it just caught a real bug in our own free scan tool before it reached more than a handful of visitors. ## What Round Trip Validation actually checks Every check runs two routes that don't talk to each other until the very end: 1. **Forward.** Make the change, then follow it to whatever actually depends on it, and record what really happened there, not just whether the command said it worked. 2. **Blind reverse.** Separately, starting only from that same real-world result and with no memory of what the forward check found, work backward to figure out where that result actually came from. 3. **Compare.** Agreement is real evidence the change worked. Disagreement isn't a false alarm to explain away, it's the finding. *[Diagram: Round Trip Validation follows a change forward through its actual consumer and records the result. Separately, a fresh reviewer starts from the same live outcome, traces backward through what the system actually used, and records a reconstructed finding without seeing the forward finding. The records meet only at a final comparison.]* *Two independent records, compared only at the end, not two checks agreeing to agree.* > The forward route asks what changed. The reverse route starts at the outcome and asks what actually caused it. ## A real bug this just caught, in our own product addAI.dev runs a free scanner (the same one at [addai.dev/scan](/scan.html)) that checks how ready a website is for AI crawlers and agents. We recently added a few basic domain and hosting facts to it, including who a site's registrar is. The forward route looked fine on its own: the tool asked a public domain registry for that information, got back a response, and reported what it found. When the registry had nothing on file, the tool reported "not identified." That's a normal, expected result: some domains genuinely don't expose that information publicly. Except every single scan came back "not identified," including for domains with completely public, easy-to-find registrar records: Cloudflare's own domain, WordPress.org's, ours. The forward route never noticed anything was wrong, because from inside that one request, a real "nothing on file" and "the request itself failed" look identical. The tool got a response. That response didn't have an answer in it, so it reported the same generic result either way. The reverse route started somewhere else entirely: not from our code, but from the outcome. Pick a domain everyone already knows has a public registrar, ask the same public registry directly, with no assumptions carried over from our own tool, and see what comes back. It came back immediately: a real registrar name, sitting right there in the response. Two independent checks, same domain, opposite answers. That disagreement is the whole point of running both routes. It told us this wasn't missing data. It was our own request getting turned away, and our code reading a rejection the same way it read "nothing found." The actual cause was small: our request was missing one piece of information the registry required before it would return real data, so it was quietly refusing us instead of telling us there was nothing to find. We added it, checked the fix against real domains, and shipped it the same day. > A successful action tells us what the tool accepted. The round trip checks what the system actually used. ## This isn't a new idea We didn't invent independent forward-and-backward checking. NASA's own software engineering guidance already tracks whether a requirement can be followed forward into what got built, and backward again, useful for finding coverage gaps, though not on its own a blind check. A 2008 quality-control method called Reverse Semantic Traceability has an independent reviewer reconstruct an earlier artifact from a later one and compare the two meanings, the closest precedent we found. A newer testing technique called retromorphic testing runs a result back through a reverse operation and checks what comes out against the original input, related to the older idea of a round-trip test, where the same process converts a value and converts it back to check nothing changed. Ours differs in one deliberate way: the check-back is a separate, blind pass, not the same process checking its own work. And a recent, still-early research idea called Cross-Context Review keeps a review session blind to the production conversation that created whatever it's reviewing. Round Trip Validation borrows a piece from each of these: two directions, an independent reconstruction, an explicit reverse step and comparison, and a reviewer kept blind to the forward answer, applied to what a real system actually consumed, not just to a paper trail. ## Where this fits in how we work We already describe how we work as Plan, Deploy, Review. Round Trip Validation is what makes that last step mean something more than a glance at our own output: not checking that our work looks right, but going and finding out, independently, whether it actually is. That's the standard we hold our own product to before we call anything done, and it's the same standard we bring to a client's AI agent, their DNS, or their site's AI-readiness files. > Not "did the command succeed," but "did the change actually reach whoever depends on it." [Get in touch](/contact.html) ## Sources - [Zhereb, Pavlov, Doroshenko, and Sergienko, "Using Reverse Semantic Traceability for Quality Control in Agile MSF-based Projects"](https://www.slideserve.com/janfowler/using-reverse-semantic-traceability-for-quality-control-in-agile-msf-based-projects-powerpoint-ppt-presentation) (SEC(R) 2008 conference presentation). Closest method precedent: independently restore requirements from a design artifact, compare, and decide whether to rework. - [NASA Software Engineering Handbook, SWE-052: Bidirectional Traceability](https://swehb.nasa.gov/spaces/SWEHBVD/pages/102695427/SWE-052%2B-%2BBidirectional%2BTraceability). Reference for following requirement relationships both forward and backward. - [Yu, Mang, Guo, and He, "Retromorphic Testing: A New Approach to the Test Oracle Problem"](https://arxiv.org/html/2310.06433). Reference for forward and backward transformations and comparing the returned result with its source. - [Wikimedia, "testreduce"](https://github.com/wikimedia/mediawiki-services-parsoid-testreduce). The codebase behind "mass roundtrip (wikitext -> html -> wikitext) testing," the established software-testing sense of "round trip": the same pipeline converts a value and converts it back, then checks the result against the original. Our reverse route is a separate, blind pass, not the same pipeline checking its own work. - [Song, "Cross-Context Review: Improving LLM Output Quality by Separating Production and Review Sessions"](https://arxiv.org/html/2603.12123). Early preprint. Reference for keeping production history out of the reviewer's context, not evidence that any review method guarantees correctness.