Most audit products say "we have dozens of checks." We publish how each one is tested — and what that testing cannot prove. Every run goes into a public report, including the bad ones and the gaps.
A regression corpus of 1110 fixtures pins every tool's behaviour, but we write those fixtures ourselves, so it is a build gate — not evidence, and not published as such. The only measurement that can tell you anything about us is the one we do not control.
This is the only leg that can support a claim like "as good as Lighthouse." Tools with industry counterparts (Lighthouse, axe-core, Mozilla Observatory, SSL Labs, Google Rich Results Test, IAB CMP Validator, dig, Google's robots.txt parser) run against the same sites in parallel with the reference, and every disagreement lands in the public report with the date it was measured — including the rows where we come off worse.
What this does not prove. It covers 22 of our 74 tools — the rest (compliance, answer-engine readiness, regulation tracking) have no industry counterpart, so for those "as good as the leader" is not a claim anyone can make, us included. The cohort is 10 sites and does not rotate, which means a score is evidence about those 10 and not about the web. And the reference can be wrong: the first real disagreement we investigated turned out to be a bug in our reference, not in the tool — the read-out said RED until we checked it by hand.
None of these run today — they are design notes, listed so the gap is visible rather than implied. The one thing below that is real is the confidence label, and it ships on every finding.
Saying "this is compliant, this isn't" isn't enough. How the finding was produced is always visible in the report. The user can answer "how much should I trust this?" from the report itself.
Every finding's JSON output also carries mandatory method (how it was measured), limitations (caveats), and evidence.guidelines (regulator reference) fields. When an auditor asks "how did you produce this finding?" the answer comes straight from the report.
When a tool disagrees with its reference, the public report shows the gap. We fix the code and re-run, repeat until the curve goes up. We don't delete the old number.
Each daily report stays at /validation-report/archive/<date>.html. "It was 78% in February 2026, 96% in May 2026" — improvement is provable.
The customer contract carries this line: "Evidalux publishes precision/recall/agreement metrics for every plugin daily in a public report." This is a legal commitment.
Every run has its own snapshot under /validation-report/archive/ — 177 of them so far. Nothing is overwritten, so a number can be checked against the run that produced it.
Every run published — including the bad ones, and what the tests can't prove. Proof before pitch.