Documentation
Start with the decision
StructTrace compares a baseline and candidate on the same cases. Start with recorded outputs, then add live adapters only when you need them.
Install locallySource and release pathsFirst comparisonInvoice demo to real filesCore conceptsValidity, correctness, and authority
What StructTrace does
StructTrace evaluates paired structured outputs against one fixed external JSON Schema and explicit deterministic correctness rules. It preserves operational and evaluation failures, calculates paired transitions over independent evidence units, generates a local report, applies a multi-state gate, and retains enough evidence for deterministic replay.
The recommended path
- Run the bundled invoice demo and inspect why its gate refuses release authority.
- Import a golden dataset plus baseline and candidate output files.
- Define what correct means field by field.
- Run one paired comparison and inspect every discordant case.
- Replay the evidence, pin recurring regressions, and only then add the gate to CI.
structtrace doctor
structtrace demo invoice --open
structtrace init my-comparison --template recorded
structtrace --project-root my-comparison run
structtrace --project-root my-comparison report latest --open
structtrace --project-root my-comparison replay latestSupport levels
| Workflow | Status | Meaning |
|---|---|---|
| Recorded JSONL | Stable candidate | No process, Python, or provider dependency. |
| Command adapter | Beta | Bounded versioned JSONL protocol, no shell by default. |
| Python callable | Beta | Persistent bounded worker with failures retained per case. |
| OpenAI-compatible | Experimental | Explicit endpoint; broader provider and scheduling evidence remains open. |
Long-form reference
The repository contains the complete versioned documentation for inputs and configuration, evaluation, release gates, replay, and troubleshooting.