EvidaLux Tools Validation Report

Every run of our 64 audit tools — behaviour lock, agreement with external references, and real-site oracles. Methodology over marketing — bad numbers publish too.
Generated: 2026-05-08T12:27:35.528364+00:00
This is an archived snapshot. For the latest, see here.

Section 1 — Cross-Tool Agreement
Diff against industry-reference tools on the same 10-site cohort. cohort: 2026-05-cohort1
1 of 14 plugins measured · median agreement 0.40
PEND rows mean the reference-tool integration is queued. Each integration may use a different per-site match metric (see plugin.metric). Median agreement is taken across implemented rows only.
PluginReference toolBandAgreementAgreed / ComparedComparison metric
accessibility.axeaxe-core CLI (Deque)PEND——Integration queued
compliance.iab_tcfIAB CMP ValidatorPEND——Integration queued
compliance.iab_tcf_verifiedIAB CMP ValidatorPEND——Integration queued
quality.lighthouse_perfGoogle PageSpeed Insights APIPEND——Integration queued
quality.owasp_zap_scanOWASP ZAP REST APIPEND——Integration queued
quality.vulnerability_nucleinuclei templates (live)PEND——Integration queued
search.lighthouse_seoGoogle PageSpeed Insights APIPEND——Integration queued
security.exposed_filesnuclei exposed-panels templatesPEND——Integration queued
security.headersMozilla Observatory v2 (MDN)RED0.404 / 10overall score within ±15 of Mozilla Observatory v2 score
security.tls_deepSSL Labs APIPEND——Integration queued
seo.broken_linkslinkchecker (W3C)PEND——Integration queued
seo.hreflang_validatorhreflang.orgPEND——Integration queued
seo.robots_txt_auditGoogle robotstxt parserPEND——Integration queued
seo.structured_dataGoogle Rich Results Test APIPEND——Integration queued

Section 2 — Real-World Correctness

Plugin findings on random real sites (US/Europe/Turkey/Arab/South America) scored against a real-browser + HTTP ground truth. Unreachable sites are INCONCLUSIVE (never gate); a gate failure is a false positive/negative on a reachable site.
RegionWhenVerdictReachableInconclusiveFPFN
arab, europe, south_america, turkey, us2026-10-21 02:59 UTCFAIL851505
arab, europe, south_america, turkey, us2026-10-20 03:03 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-19 02:57 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-18 03:36 UTCFAIL861424
arab, europe, south_america, turkey, us2026-10-18 02:57 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-17 02:54 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-16 03:05 UTCFAIL861425
arab, europe, south_america, turkey, us2026-10-15 03:02 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-14 02:54 UTCFAIL851525
arab, europe, south_america, turkey, us2026-10-13 03:04 UTCFAIL871325
arab, europe, south_america, turkey, us2026-10-12 03:02 UTCFAIL851525
arab, europe, south_america, turkey, us2026-10-11 03:00 UTCFAIL841624
arab, europe, south_america, turkey, us2026-10-10 03:02 UTCFAIL851525
arab, europe, south_america, turkey, us2026-10-09 03:01 UTCFAIL841624
arab, europe, south_america, turkey, us2026-10-08 02:54 UTCFAIL861425
arab, europe, south_america, turkey, us2026-10-07 03:01 UTCFAIL841625
arab, europe, south_america, turkey, us2026-10-06 02:56 UTCFAIL841625
arab, europe, south_america, turkey, us2026-10-05 02:58 UTCFAIL851525
arab, europe, south_america, turkey, us2026-10-04 03:38 UTCFAIL861425
arab, europe, south_america, turkey, us2026-10-04 03:01 UTCFAIL851534
arab, europe, south_america, turkey, us2026-10-03 02:58 UTCFAIL851524
arab, europe, south_america, turkey, us2026-10-02 02:53 UTCFAIL851525
arab, europe, south_america, turkey, us2026-10-01 02:54 UTCFAIL871325
arab, europe, south_america, turkey, us2026-09-30 02:48 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-29 03:00 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-28 02:49 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-27 03:36 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-27 02:52 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-26 02:42 UTCFAIL861425
arab, europe, south_america, turkey, us2026-09-25 03:00 UTCFAIL861425

Latest run — 2026-10-21 02:59 UTC

PluginConfirmedFPFNInconclusive
accessibility.eaa_mapping000138
compliance.accessibility_statement580094
compliance.age_verification00085
compliance.ai_disclosure00085
compliance.child_consent00085
compliance.cookie_consent000207
compliance.cross_border_transfer000152
compliance.dark_pattern6700152
compliance.data_subject_request160069
compliance.dpo_contact00085
compliance.eu_representative00085
compliance.geo_consistency000170
compliance.iab_tcf00085
compliance.iab_tcf_verified50080
compliance.legal_disclosure7200147
compliance.odr_link670018
compliance.pay_or_consent_wall00085
compliance.pricing_indication000170
compliance.privacy_policy_content1400151
compliance.purchase_disclosure000152
compliance.required_pages15800128
quality.api_test000216
security.exposed_files1560012
security.headers58000324
security.tls_deep2390093
seo.broken_links190022
seo.canonical_audit46003
seo.duplicate_content00085
seo.freshness360025
seo.hreflang_validator71003
seo.meta_tags90050
seo.robots_txt_audit16002
seo.sitemap74008
seo.structured_data83009
tech.dns_health34000170
tech.stack_detection00085

Wrong findings — latest run (gate)

  • false_negative seo.meta_tags/meta.title.missing — https://www.imdb.com/ — oracle has no title; plugin didn't flag
  • false_negative seo.meta_tags/meta.title.missing — https://www.booking.com/ — oracle has no title; plugin didn't flag
  • false_negative seo.meta_tags/meta.title.missing — https://www.dw.com/ — oracle has no title; plugin didn't flag
  • false_negative seo.meta_tags/meta.title.missing — https://www.amazon.ae/ — oracle has no title; plugin didn't flag
  • false_negative seo.meta_tags/meta.title.missing — https://www.petra.gov.jo/ — oracle has no title; plugin didn't flag

Past reports

Legend — column meanings

Cross-Tool Agreement
Plugin
The audit tool we are validating (e.g. seo.lighthouse_seo).
Reference tool
The industry-standard tool whose findings we compare against on the same site (e.g. PageSpeed Insights, Pa11y, W3C linkchecker, Mozilla Observatory).
Band
Bucketed from the agreement score:
GREEN agreement ≥ 0.90.
YELLOW agreement ≥ 0.70.
RED below 0.70.
Agreement
How much our plugin and the reference tool produce the same verdict, on the same site, on the same 10-site cohort (0.00–1.00). High = strong industry consensus that we are seeing the same thing.
Agreed / Compared
On each of the 10 cohort sites, did our plugin and the reference tool give the same verdict? The left number is how many sites we agreed on; the right is how many sites we could actually compare (if either tool fails on a site — timeout, blocked, 5xx — that site does not count).
Comparison metric
Short description of how the agreement number is computed for this pair (e.g. "in-domain Jaccard", "score delta ≤ 5"). The metric is plugin-specific because each reference tool exposes its findings differently.