The Blind Critic: Why Autonomous Systems Must Evaluate Their Own Output Without Knowing Who Made It

Every autonomous system that produces output eventually needs to evaluate it. The output must meet a quality bar before it reaches the user. The obvious approach is self-check. The dangerous approach is self-check without structural blindness.

The evaluator knows what the creator intended. It knows which approach was taken, which alternatives were rejected, and which failure modes were already handled. That knowledge is a contaminant. The evaluator gives benefit of the doubt to familiar patterns. It treats "tried hard" as a proxy for "succeeded." It grades against intention instead of requirement. The result is a self-approval loop that looks rigorous but is actually a rubber stamp.

The blind critic pattern solves this by stripping provenance before evaluation. The critic receives the output without knowing who made it, what they intended, which path they took, or what they considered and rejected. The critic evaluates the artifact against the specification, against objective criteria, and against nothing else.

Why Self-Evaluation Drifts

Three patterns corrupt self-evaluation in production systems.

First, intention contamination. The evaluator knows what the creator was trying to achieve and unconsciously grades the output against that intent rather than against the spec. A chapter that was "trying to build tension" gets a pass on structural incoherence because the evaluator can see the goal. A trading strategy that was "trying to capture momentum" gets a pass on drawdown depth because the evaluator knows the design philosophy. The output is graded on effort and intent rather than on results.

Second, path bias. The evaluator knows which alternatives the creator rejected. It treats the chosen path as the right one because it can see the other options that were considered and dismissed. The critic cannot independently assess whether the rejected alternatives were actually better because it has already absorbed the creator's conclusion.

Third, relationship calibration. When the same creator and evaluator interact repeatedly, the evaluator learns the creator's patterns and begins to predict rather than inspect. It sees the opening paragraph and pre-approves the conclusion because "this creator usually sticks the landing." It sees the first function and skims the rest because "this module has been reliable before." Evaluation degrades into pattern matching, and the system ships output that looks like past successes without actually being one.

The Blind Critic Architecture

Systems that evaluate honestly use four mechanisms to enforce blindness.

Provenance Stripping

The pipeline removes identifying information before the artifact reaches the critic. No creator identity, no approach history, no rejected alternatives, no design notes. The critic receives the raw output and the specification. Everything else is stripped at the architectural level, not by policy. The critic does not have to try to be impartial. It simply does not have access to the information that would bias it.

This is different from asking the same agent to evaluate its own output "objectively." A single agent with full context cannot unknow what it knows. Blindness requires a separate evaluator that genuinely does not have the creator's context, or a pipeline that strips it before the evaluation step.

Specification-Bound Criteria

The critic evaluates against the specification, not against the creator's intent. The spec is the only shared reference. For a code module, the spec is the interface contract, the performance budget, and the security constraints. For a book chapter, the spec is the outline, the character consistency rules, and the tension arc. For a video, the spec is the platform format, the duration target, and the engagement hooks.

When criteria are bound to the spec, the critic does not need to infer intent. The spec says what the output must do. The critic checks whether it does that. This makes evaluation deterministic and auditable. The critic's judgment can be verified against the spec after the fact.

Critic Rotation

The system rotates which critic evaluates which output. A single critic that always evaluates the same creator develops the relationship calibration described above. Rotation prevents the critic from learning the creator's patterns. Each evaluation is a fresh assessment against the spec, not a continuation of an ongoing relationship.

Rotation also makes bias detectable. When two critics evaluate the same output differently, the disagreement is a signal. It means the criteria are ambiguous, the spec is incomplete, or one critic has drifted. The system tracks inter-critic agreement rates and flags divergences for review.

Adversarial Calibration

The critic is periodically tested with known-good and known-bad outputs mixed into the evaluation queue. If the critic approves a known-bad output, its criteria are too loose. If it rejects a known-good output, its criteria are too strict. The calibration set is invisible to the critic and refreshed regularly to prevent gaming.

This is the same principle as exam proctoring with seeded questions. The critic does not know which evaluations are calibration checks, so it must apply the same standard to everything. The calibration results feed back into the critic's scoring thresholds.

How Darcron Uses the Blind Critic

Darcron, the autonomous AI software factory, runs a gauntlet loop where feature proposals are built by one agent and evaluated by another. The builder proposes a feature: a new API endpoint, a UI component, a database migration. The proposal includes the implementation, the test suite, and the architectural rationale.

Before the proposal reaches the critic, the pipeline strips the builder's identity and design history. The critic receives the implementation and the specification. It does not know which builder produced the proposal, which alternatives were considered, or how many iterations the builder went through.

The critic evaluates against three criteria: test coverage must exceed 80 percent, the implementation must match the architectural pattern specified in the design doc, and the performance budget must not be exceeded. These criteria are bound to the spec, not to the builder's intent. A proposal that "tried to be elegant" but skipped tests fails. A proposal that "had to cut corners" but meets all three criteria passes.

When the critic rejects a proposal, the rejection includes the specific criteria that were not met. The builder receives the rejection without knowing which critic evaluated the work. The builder fixes the gaps and resubmits. The loop continues until the proposal passes or the round limit is reached.

Darcron rotates critics across proposals so each evaluation is independent. When two critics disagree, the system routes to a third for tie-breaking. The disagreement data feeds back into the spec clarification loop, making criteria more precise over time.

Key Takeaways for Agentic Evaluation

• T-BC1: Strip provenance before evaluation. Remove creator identity, design history, and rejected alternatives before the artifact reaches the critic. Blindness must be architectural, not procedural.
• T-BC2: Evaluate against the spec, not against intent. The specification is the only shared reference between creator and critic. Criteria bound to the spec are deterministic and auditable.
• T-BC3: Rotate critics to prevent calibration drift. A critic that always evaluates the same creator learns to predict rather than inspect. Rotation keeps evaluation fresh.
• T-BC4: Detect bias through inter-critic disagreement. When two critics evaluate the same output differently, the disagreement is a diagnostic signal. Track agreement rates and flag divergences.
• T-BC5: Calibrate critics with seeded known-good and known-bad outputs. Invisible calibration checks prevent criteria from drifting too loose or too tight. Refresh the calibration set regularly.

Self-evaluation without blindness is not evaluation. It is self-confirmation. The blind critic pattern turns self-review into actual review by removing the information that corrupts judgment. The system does not have to trust itself to be honest. It trusts the architecture that makes dishonesty impossible.