EvidaLux Tools Validation Report

Every run of our 64 audit tools — behaviour lock, agreement with external references, and real-site oracles. Methodology over marketing — bad numbers publish too.
Generated: 2026-05-07T11:54:56.463114+00:00
This is an archived snapshot. For the latest, see here.

Section 1 — Cross-Tool Agreement
Diff against industry-reference tools on the same 10-site cohort. cohort: 2026-05-cohort1
median agreement 0.00
Day-1 baseline. Reference tool integrations are stubbed. Real PSI/axe/Observatory/SSL Labs wiring happens next sprint. Honest baseline numbers will appear once integrations land.
PluginReference toolBandAgreementAgreed / ComparedComparison metric
axe_wcagaxe-core CLI (Deque)RED0.000 / 0
10 of 10 sites unmeasured
—
broken_linkslinkchecker (W3C)RED0.000 / 0
10 of 10 sites unmeasured
—
exposed_filesnuclei exposed-panels templatesRED0.000 / 0
10 of 10 sites unmeasured
—
hreflang_validatorhreflang.orgRED0.000 / 0
10 of 10 sites unmeasured
—
iab_tcfIAB CMP ValidatorRED0.000 / 0
10 of 10 sites unmeasured
—
lighthouse_perfGoogle PageSpeed Insights APIRED0.000 / 0
10 of 10 sites unmeasured
—
lighthouse_seoGoogle PageSpeed Insights APIRED0.000 / 0
10 of 10 sites unmeasured
—
robots_txt_auditGoogle robotstxt parserRED0.000 / 0
10 of 10 sites unmeasured
—
security_headersMozilla Observatory APIRED0.000 / 0
10 of 10 sites unmeasured
—
structured_dataGoogle Rich Results Test APIRED0.000 / 0
10 of 10 sites unmeasured
—
tls_deepSSL Labs APIRED0.000 / 0
10 of 10 sites unmeasured
—

Section 2 — Real-World Correctness

Plugin findings on random real sites (US/Europe/Turkey/Arab/South America) scored against a real-browser + HTTP ground truth. Unreachable sites are INCONCLUSIVE (never gate); a gate failure is a false positive/negative on a reachable site.
RegionWhenVerdictReachableInconclusiveFPFN
arab, europe, south_america, turkey, us2026-10-21 02:59 UTCFAIL851505
arab, europe, south_america, turkey, us2026-10-20 03:03 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-19 02:57 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-18 03:36 UTCFAIL861424
arab, europe, south_america, turkey, us2026-10-18 02:57 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-17 02:54 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-16 03:05 UTCFAIL861425
arab, europe, south_america, turkey, us2026-10-15 03:02 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-14 02:54 UTCFAIL851525
arab, europe, south_america, turkey, us2026-10-13 03:04 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-12 03:02 UTCFAIL851525
arab, europe, south_america, turkey, us2026-10-11 03:00 UTCFAIL841624
arab, europe, south_america, turkey, us2026-10-10 03:02 UTCFAIL851525
arab, europe, south_america, turkey, us2026-10-09 03:01 UTCFAIL841624
arab, europe, south_america, turkey, us2026-10-08 02:54 UTCFAIL861425
arab, europe, south_america, turkey, us2026-10-07 03:01 UTCFAIL841625
arab, europe, south_america, turkey, us2026-10-06 02:56 UTCFAIL841625
arab, europe, south_america, turkey, us2026-10-05 02:58 UTCFAIL851525
arab, europe, south_america, turkey, us2026-10-04 03:38 UTCFAIL861425
arab, europe, south_america, turkey, us2026-10-04 03:01 UTCFAIL851534
arab, europe, south_america, turkey, us2026-10-03 02:58 UTCFAIL851524
arab, europe, south_america, turkey, us2026-10-02 02:53 UTCFAIL851525
arab, europe, south_america, turkey, us2026-10-01 02:54 UTCFAIL871325
arab, europe, south_america, turkey, us2026-09-30 02:48 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-29 03:00 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-28 02:49 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-27 03:36 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-27 02:52 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-26 02:42 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-25 03:00 UTCFAIL861425

Latest run — 2026-10-21 02:59 UTC

PluginConfirmedFPFNInconclusive
accessibility.eaa_mapping000138
compliance.accessibility_statement580094
compliance.age_verification00085
compliance.ai_disclosure00085
compliance.child_consent00085
compliance.cookie_consent000207
compliance.cross_border_transfer000152
compliance.dark_pattern6700152
compliance.data_subject_request160069
compliance.dpo_contact00085
compliance.eu_representative00085
compliance.geo_consistency000170
compliance.iab_tcf00085
compliance.iab_tcf_verified50080
compliance.legal_disclosure7200147
compliance.odr_link670018
compliance.pay_or_consent_wall00085
compliance.pricing_indication000170
compliance.privacy_policy_content1400151
compliance.purchase_disclosure000152
compliance.required_pages15800128
quality.api_test000216
security.exposed_files1560012
security.headers58000324
security.tls_deep2390093
seo.broken_links190022
seo.canonical_audit46003
seo.duplicate_content00085
seo.freshness360025
seo.hreflang_validator71003
seo.meta_tags90050
seo.robots_txt_audit16002
seo.sitemap74008
seo.structured_data83009
tech.dns_health34000170
tech.stack_detection00085

Wrong findings — latest run (gate)

  • false_negative seo.meta_tags/meta.title.missing — https://www.imdb.com/ — oracle has no title; plugin didn't flag
  • false_negative seo.meta_tags/meta.title.missing — https://www.booking.com/ — oracle has no title; plugin didn't flag
  • false_negative seo.meta_tags/meta.title.missing — https://www.dw.com/ — oracle has no title; plugin didn't flag
  • false_negative seo.meta_tags/meta.title.missing — https://www.amazon.ae/ — oracle has no title; plugin didn't flag
  • false_negative seo.meta_tags/meta.title.missing — https://www.petra.gov.jo/ — oracle has no title; plugin didn't flag

Past reports

Legend — column meanings

Cross-Tool Agreement
Plugin
The audit tool we are validating (e.g. seo.lighthouse_seo).
Reference tool
The industry-standard tool whose findings we compare against on the same site (e.g. PageSpeed Insights, Pa11y, W3C linkchecker, Mozilla Observatory).
Band
Bucketed from the agreement score:
GREEN agreement ≥ 0.90.
YELLOW agreement ≥ 0.70.
RED below 0.70.
Agreement
How much our plugin and the reference tool produce the same verdict, on the same site, on the same 10-site cohort (0.00–1.00). High = strong industry consensus that we are seeing the same thing.
Agreed / Compared
On each of the 10 cohort sites, did our plugin and the reference tool give the same verdict? The left number is how many sites we agreed on; the right is how many sites we could actually compare (if either tool fails on a site — timeout, blocked, 5xx — that site does not count).
Comparison metric
Short description of how the agreement number is computed for this pair (e.g. "in-domain Jaccard", "score delta ≤ 5"). The metric is plugin-specific because each reference tool exposes its findings differently.