Documentation

Core concepts

StructTrace keeps structural, semantic, operational, and inferential questions separate so one green metric cannot hide another failure.

Baseline and candidate

Both variants are evaluated on the same joined case identities, against the same external schema, expected values, evaluators, primary outcome, and evidence-unit policy.

Structural versus semantic correctness

Strict parsing asks whether the whole output is valid JSON. Schema validity asks whether it satisfies the external contract. Semantic correctness asks whether explicit deterministic rules agree with the expected task outcome. A schema-valid output that fails the primary outcome is valid but wrong.

Independent evidence units

Rows are execution observations, not automatically independent evidence. StructTrace groups repeated observations by a declared identity or a safe default fingerprint. Exact duplicates do not multiply inference. Conflicting repeated trials force insufficient evidence instead of selecting one row.

Paired transitions

Release authorization

A positive average is not authorization. StructTrace evaluates quality failures, evidence insufficiency, and runtime errors separately. Advisory never authorizes. Regression covers configured relative change only. Release requires a complete safe profile and is the sole mode that can authorize deployment.

Fully evaluated semantic pairs

Operational failure is not relabeled as an incorrect answer. The complete-denominator deployment view keeps it as failure; the semantic-only view reports only pairs with explicit binary semantic outcomes for both variants and states how many were excluded.