This describes work done as a machine learning engineer at a startup. Model names, prompts and product internals are left out. The numbers are from our own evaluation at the time.

The brief sounds simple. A user uploads a picture of a thing, a chair or a creature or a character, and asks for the four canonical views so a reconstruction pipeline can build a mesh from them. Any capable image model will produce four attractive images. The problem is that "attractive" is not the requirement. The requirement is that all four are the same object: same proportions, same colours, same decorations, the left view showing the mirror of the right view, the back view keeping the identity the front view established. Four independent generations almost never satisfy that. You get four cousins.

Why the evaluator is a separate agent

The system is a ReAct loop over eight specialised agents: requirement understanding, per-view prompt optimisation, generation, single-image evaluation, cross-view evaluation, targeted editing, external retrieval, and a controller that picks the next action. The split that mattered most was separating the generator from the judge.

A model asked to generate and then rate its own output is generous to itself. The generator's job is to try hard; the evaluator's job is to find problems, and it does that better when it has never seen the prompt that produced the images, only the original requirement and the four results side by side at full resolution. It scores each view on prompt adherence, visual quality, correct orientation, and whether the subject is complete and uncropped. Then it scores the set on geometry and identity consistency, colour and decoration consistency, scale alignment, and whether every user-requested modification actually shows up in every view. The output is structured: an overall score, per-view issues with severity, a list of views to regenerate, and edit suggestions.

One rule I added after watching the loop misbehave: if the evaluator's output cannot be parsed, that is a failure, not a pass. The first version defaulted to "fine" on a parse error and quietly waved bad sets through.

Regenerate one view, not four

The naive retry policy regenerates the whole set whenever the evaluator is unhappy. That is expensive and, worse, it throws away views that were already correct and replaces them with new ones that might not be. The loop instead repairs locally:

This is the same instinct as clamping a slightly over-budget card instead of rejecting it: fix the smallest thing that is wrong. The dependency structure makes it work. Front is generated first and becomes the anchor. Left and right are generated against it with an explicit instruction that they show opposite sides, and for characters and animals that limb positions mirror. Back inherits from front. When the evaluator flags the left view, the controller knows exactly which references to hand the generator for the redo.

When to stop

A loop that can always try again will always try again. Three caps run at once, and the first one hit ends the run: a quality threshold, a maximum iteration count (four rounds), and the user's subscription budget, which is reserved up front and settled at the end so a runaway loop cannot overspend. Hitting a cap does not mean failure. Every candidate set is versioned with its score, so the controller returns the best version it has seen, and can roll back to it if a later round made things worse.

The quality threshold is dynamic, and this was the subtle part. A request with two constraints ("a red chair") and a request with nine ("a red velvet armchair with brass studs, a torn left cushion and a cat asleep on it") should not be held to the same bar for "fully satisfies every condition." The more constraints the requirement understanding agent extracts, the more the threshold relaxes, within limits. Without that, complex requests spun through all four rounds and returned a near-miss anyway; with it, they stopped early on the near-miss and saved the cost of the last two rounds.

The result. Over 90% of requests converged to an output the evaluator accepted within two iterations. Most of the rest hit the iteration cap on genuinely hard subjects, such as heavily asymmetric objects where "mirror the left side" has no clean answer, and shipped their best version.

What the loop costs

Splitting into agents is not free. There are more tokens, more latency, more state to hand between steps, and when something goes wrong it is harder to say which agent is to blame. The price was worth paying for two reasons. Local regeneration cut the cost of a retry to roughly a quarter of a full set. And a replayable trace of every thought, action, observation and version made failures debuggable: when a user reported a bad result, I could replay the run and see the exact evaluator call that let it through.

The pattern I took away, and later reused on a very different problem, is the shape of the loop rather than any of the agents in it. Generate, judge with something that did not generate, repair the smallest thing that is wrong, version everything, and stop for a reason you can name. The generator is the least important piece.