Why LLMs Shouldn't Be the Final Authority in Engineering Verification

Engineering verification is a claim about the physical world. That makes the last step different from ordinary knowledge work: someone must be able to show what was checked, against which sources, under which rule and with what authority.

Investigation and authority are different jobs

Language models are good at discovering candidate evidence. They can search a drawing set, interpret a requirement, connect terminology across documents and assemble a useful investigation trail. Those capabilities reduce the cost of getting to the relevant material.

They are not, by themselves, a reliable authority for a contractual or safety-critical determination. A plausible sentence is not a proof, and confidence is not a governed acceptance criterion.

Make the boundary explicit

A robust workflow separates the stages:

1. Discover: retrieve candidate sources and model elements.
2. Interpret: turn prose into typed statements and measurable fields.
3. Investigate: assemble provenance, context and counterexamples.
4. Verify: evaluate a fixed predicate against the assembled population.
5. Decide: let an accountable reviewer accept, reject or request more evidence.

The model can assist the first three stages. The final determination should remain visible as a rule, a result and a reviewable record.

A counterexample is more useful than a score

For a requirement such as “all exterior luminaires shall be mounted above 10 ft,” the system should expose the population, the measured values and the element that breaks the predicate. “NOT MET — fixture L-114 at 8.5 ft” gives a reviewer something concrete to inspect. A model score of 0.87 does not.

AI investigates. Governed logic decides.

Design for the handoff

The product surface should preserve the chain from requirement to source, element, finding and determination. When that chain is easy to open, AI becomes leverage for engineering judgment instead of a substitute for it.

This is the pattern demonstrated in the Evidence Graph and the Verification Playground: synthetic data, explicit provenance and a visible counterexample.