EvidaLux Tools Validation Report

Every run of our 64 audit tools — behaviour lock, agreement with external references, and real-site oracles. Methodology over marketing — bad numbers publish too.
Generated: 2026-05-09T20:11:48.048126+00:00
This is an archived snapshot. For the latest, see here.

Section 1 — Cross-Tool Agreement
Diff against industry-reference tools on the same 10-site cohort. cohort: 2026-05-cohort1
2 of 14 plugins measured · median agreement 0.65
PEND rows mean the reference-tool integration is queued. Each integration may use a different per-site match metric (see plugin.metric). Median agreement is taken across implemented rows only.
PluginReference toolBandAgreementAgreed / ComparedComparison metric
accessibility.axePa11y (htmlcs runner)ERR——https://www.evidalux.com/: FileNotFoundError: [Errno 2] No such file or directory: 'pa11y'; https://www.example.com/: Fi
compliance.iab_tcfIAB CMP ValidatorPEND——Integration queued
compliance.iab_tcf_verifiedIAB CMP ValidatorPEND——Integration queued
quality.lighthouse_perfGoogle PageSpeed Insights APIERR——https://www.evidalux.com/: RuntimeError: lighthouse runner failed: lighthouse binary not on PATH; https://www.example.co
quality.owasp_zap_scanOWASP ZAP REST APIPEND——Integration queued
quality.vulnerability_nucleinuclei templates (live)PEND——Integration queued
search.lighthouse_seoGoogle PageSpeed Insights APIERR——https://www.evidalux.com/: RuntimeError: lighthouse runner failed: lighthouse binary not on PATH; https://www.example.co
security.exposed_filesnuclei exposed-panels templatesPEND——Integration queued
security.headersMozilla Observatory v2 (MDN)RED0.404 / 10overall score within ±15 of Mozilla Observatory v2 score
security.tls_deepSSL Labs APIPEND——Integration queued
seo.broken_linkslinkchecker (W3C)ERR——https://www.evidalux.com/: FileNotFoundError: [Errno 2] No such file or directory: 'linkchecker'; https://www.example.co
seo.hreflang_validatorhreflang.orgPEND——Integration queued
seo.robots_txt_auditGoogle robotstxt parserPEND——Integration queued
seo.structured_datavalidator.schema.orgGREEN0.909 / 10schema.org type-set Jaccard ≥0.50 (validator.schema.org vs plugin JSON-LD/Microdata/RDFa; Google Rich Results API retired 2024)

Section 2 — Real-World Correctness

Plugin findings on random real sites (US/Europe/Turkey/Arab/South America) scored against a real-browser + HTTP ground truth. Unreachable sites are INCONCLUSIVE (never gate); a gate failure is a false positive/negative on a reachable site.
RegionWhenVerdictReachableInconclusiveFPFN
arab, europe, south_america, turkey, us2026-07-20 02:41 UTCFAIL861410
arab, europe, south_america, turkey, us2026-07-19 02:37 UTCFAIL881210
arab, europe, south_america, turkey, us2026-07-18 20:31 UTCPASS851500
arab, europe, south_america, turkey, us2026-07-18 19:22 UTCFAIL841650
arab, europe, south_america, turkey, us2026-07-18 18:18 UTCPASS18700
arab, europe, south_america, turkey, us2026-07-18 18:00 UTCFAIL20530
arab, europe, south_america, turkey, us2026-07-18 16:09 UTCFAIL205441
arab, europe, south_america, turkey, us2026-07-04 06:08 UTCFAIL20511
arab, europe, south_america, turkey, us2026-07-03 06:20 UTCFAIL19611

Latest run — 2026-07-20 02:41 UTC

PluginConfirmedFPFNInconclusive
accessibility.eaa_mapping000155
compliance.accessibility_statement740098
compliance.age_verification00086
compliance.ai_disclosure00086
compliance.child_consent00086
compliance.cookie_consent000210
compliance.cross_border_transfer000172
compliance.dark_pattern8600172
compliance.data_subject_request80078
compliance.dpo_contact00086
compliance.eu_representative00086
compliance.geo_consistency000172
compliance.iab_tcf00086
compliance.iab_tcf_verified60080
compliance.legal_disclosure8200176
compliance.odr_link86000
compliance.pay_or_consent_wall00086
compliance.pricing_indication000172
compliance.privacy_policy_content3600116
compliance.purchase_disclosure000172
compliance.required_pages2740070
quality.api_test000222
security.exposed_files1570013
security.headers60800303
security.tls_deep2461093
seo.broken_links380028
seo.canonical_audit51004
seo.duplicate_content00085
seo.freshness340028
seo.hreflang_validator74005
seo.meta_tags158000
seo.robots_txt_audit17001
seo.sitemap84003
seo.structured_data80009
tech.dns_health34400172
tech.stack_detection00086

Wrong findings — latest run (gate)

  • false_positive security.tls_deep/tls.no_https — https://www.isbank.com.tr/ — TLS handshake succeeded independently

Past reports

Legend — column meanings

Cross-Tool Agreement
Plugin
The audit tool we are validating (e.g. seo.lighthouse_seo).
Reference tool
The industry-standard tool whose findings we compare against on the same site (e.g. PageSpeed Insights, Pa11y, W3C linkchecker, Mozilla Observatory).
Band
Bucketed from the agreement score:
GREEN agreement ≥ 0.90.
YELLOW agreement ≥ 0.70.
RED below 0.70.
Agreement
How much our plugin and the reference tool produce the same verdict, on the same site, on the same 10-site cohort (0.00–1.00). High = strong industry consensus that we are seeing the same thing.
Agreed / Compared
On each of the 10 cohort sites, did our plugin and the reference tool give the same verdict? The left number is how many sites we agreed on; the right is how many sites we could actually compare (if either tool fails on a site — timeout, blocked, 5xx — that site does not count).
Comparison metric
Short description of how the agreement number is computed for this pair (e.g. "in-domain Jaccard", "score delta ≤ 5"). The metric is plugin-specific because each reference tool exposes its findings differently.