"Done" Has to Mean Done: How an AI Website Editor Decides an Edit Really Happened

By Zahi Raifer

When an AI says "I've updated your page", someone has to make sure that's actually true. In a chat demo, a wrong claim of success is a bit embarrassing. On a live business website, it is a broken promise. The owner thinks the phone number changed, and customers keep calling the old one.

At Orivox, which edits customer websites from a chat, a lot of the design comes down to one question: who gets to say an edit is done, and what has to be true before they can say it? This post walks through how we answer that, from one turn in the in-house editor to the external AI connector.

The model doesn't get to decide

The core rule is simple. A language model plans small, named changes to a page, and plain Python carries them out. The model never writes files. It writes a list of operations, and Python applies them.

That split also decides who owns the word "done". The planner can propose changes, but only the server decides whether a turn is saved. That is one of two principles underneath all of our save-time checks. The other is that every message the system sends back to the planner tells it what to do next, so it can correct course on its own.

So "done" is not something the model says. It is a state the server reaches, or doesn't.

What a finished turn actually contains

One edit turn has a fixed shape:

1. The owner types a request in Hebrew or English. The web app checks the account, queues the turn and takes the project lock.
2. The web app sends the editing service the latest message. The editing service keeps the full conversation.
3. The editing service assembles the page in memory, partials included, and asks the planner what to do.
4. The planner answers in words or calls a tool. The main tool takes a list of operations.
5. Python applies the operations with BeautifulSoup, runs its checks, saves the page, commits the change to the project's Git repository and records the turn.
6. The browser reloads the preview with the change in place.

Step 5 is where "done" gets defined. A turn that really changed something leaves evidence behind: a saved page, a Git commit and a recorded turn. Step 6 then shows the owner the result instead of just describing it. A description alone doesn't count.

No credit for an edit that didn't happen

One of the deterministic save-time checks is aimed squarely at this problem:

A claim of success needs a real change. The assistant never reports a change that did not happen.

It sits alongside the other save-time guards, all written in Python. They don't depend on the model behaving well, and each one is pinned by a replay test so a later change can't quietly drop it.

The success check matters because language models are fluent. A model can write "I've made the FAQ cards dark" just as easily whether or not its operations matched anything. Tying the claim to an actual change on the page closes that gap.

Which tool calls can finish a turn

The planner has 11 tools, and they don't all play the same role. An edit call completes the turn. The others (reading another page, looking at a screenshot of the rendered page, reading a page the owner links to with their consent, taking a palette from an image, and so on) feed their result back so the planner can keep going.

That makes "done" a property of the call type too. Looking at a screenshot is progress, not completion. A turn ends with an edit the server can check and commit, not with a remark that things look fine.

Another save-time guard supports this from a different direction: a page is read before it is written. Edits to another page are based on that page as it is now, so "done" refers to the real current page and not to the model's memory of it.

Done also means you can undo it

A change you can't undo is a gamble. On Orivox, every edit turn is its own Git commit, and the first edit commits the original site as a baseline. Undo is a restore, not a reconstruction.

That also means each "done" has an exact address in history. When the owner asks to go back, the system isn't guessing what the page used to look like. It returns to a specific commit.

There is also one writer per project. A single lock covers the whole turn across every process, so two changes can never interleave on the same site.

The other side: never say "not done" after it's done

The opposite mistake matters just as much, and it shows up most clearly through the second way into the engine.

Owners can also build and edit an Orivox site from their own Claude, with their own subscription, through a remote MCP server. Their AI does the reasoning and our side carries it out. The MCP edit tool takes the same 14 operations the in-house planner emits, and the editing service applies them with the same checks, the same Git commit and the same transcript.

An outside AI agent will retry anything that looks like it failed. So the connector follows a rule that mirrors the success check:

A completed write is always reported as done. If usage metering fails after the files are written, the tool still returns success, so the AI never retries and creates a duplicate.

Put the two rules side by side and you get a consistent contract. The system never claims a change it didn't make, and it never denies a change it did make. Bookkeeping steps that come after the write don't get to change the answer.

What this adds up to

None of this is a clever algorithm. It's a handful of rules that keep the word "done" honest:

• The model proposes, and the server decides whether a turn is saved.
• A success message requires a real change on the page.
• Only an edit call ends a turn. Looking and reading are steps along the way.
• Every finished turn is a Git commit, so every "done" can be undone.
• A finished write is reported as finished, even if bookkeeping afterwards fails.

If you're building anything where an AI acts on real systems for real people, it's worth writing down what "done" means in your product, and then making code, not the model, responsible for it.

---

About the author: Zahi Raifer is the founder of Orivox and has spent 23 years in software, working across AI, Python, .NET and databases, with a focus on product development.

AI disclosure: This article was drafted with the help of AI tools, based on Orivox's own technical documentation, and reviewed by the author.