EvidaLux Tools Validation Report

Before

quality.owasp_zap_scan: P=0.00, R=0.00, F1=0.00 · None

After

quality.owasp_zap_scan: P=0.00, R=0.00, F1=0.00 · GREEN

Before

security.tls_deep: P=0.00, R=0.00, F1=0.00 · None

After

security.tls_deep: P=0.00, R=0.00, F1=0.00 · GREEN

Before

quality.vulnerability_nuclei: P=0.00, R=0.00, F1=0.00 · None

After

quality.vulnerability_nuclei: P=0.00, R=0.00, F1=0.00 · GREEN

Before

security.exposed_files: P=0.00, R=0.00, F1=0.00 · None

After

security.exposed_files: P=0.00, R=0.00, F1=0.00 · GREEN

Before

{"scope": "repo", "kvkk_refs_in_app_plugins_tests_frontend_locale": 45, "supported_langs": 35, "compliance_plugins": "27 (incl. verbis_registration)", "note_en": "Repo carried TR-specific routing, KVKK article tuples, Turkish keyword lists, and a TR validation report.", "note_tr": "Repo TR-spesifik routing, KVKK madde tuple'ları, Türkçe keyword listeleri ve TR validation raporu taşıyordu."}

After

{"scope": "repo", "kvkk_refs_in_app_plugins_tests_frontend_locale": 0, "supported_langs": 34, "compliance_plugins": "26", "note_en": "grep -ri 'kvkk|verbis' app/ plugins/ tests/ frontend/src/ locale/ returns 0 hits; 937 unit tests pass; frontend typecheck clean.", "note_tr": "grep -ri 'kvkk|verbis' app/ plugins/ tests/ frontend/src/ locale/ sıfır hit; 937 unit test geçiyor; frontend typecheck temiz."}

Before

compliance.iab_tcf_verified: P=0.00, R=0.00, F1=0.00 · None

After

compliance.iab_tcf_verified: P=0.00, R=0.00, F1=0.00 · GREEN

Before

compliance.iab_tcf: P=0.00, R=0.00, F1=0.00 · None

After

compliance.iab_tcf: P=0.00, R=0.00, F1=0.00 · GREEN

Before

seo.hreflang_validator: P=0.00, R=0.00, F1=0.00 · None

After

seo.hreflang_validator: P=0.00, R=0.00, F1=0.00 · RED

Before

compliance.iab_tcf_verified §2 row: status=implemented but every cohort site reported `fetched=False, fetched=False → MATCH (empty/empty)` — no real cmp_id agreement signal.

After

Playwright live cohort scoring works. compliance.iab_tcf_verified: GREEN 0.90 (9/10 — 1 timing artefact on notion.so). See three follow-up plugin-specific entries below for the live numbers each integration produced on its first real daily run.

Before

§2 row `compliance.iab_tcf_verified` status=pending (no implementation).

After

§2 row `compliance.iab_tcf_verified` status=implemented. 4/4 mock match scenarios pass (both-empty, cmpId-match, cmpId-divergence with misconfig verdict, both-no-playwright graceful). Live cohort baseline blocked on adding Playwright to validation env (follow-up).

Before

§2 row `compliance.iab_tcf` status=pending (no implementation).

After

§2 row `compliance.iab_tcf` status=implemented. 4/4 mock match scenarios pass; live ref on evidalux.com (tcf=False, expected) and bbc.co.uk (tcf=False, JS-deferred CMP — both vantages miss, consistent). Live cohort baseline next daily run.

Before

§2 row `seo.hreflang_validator` status=pending (no implementation).

After

§2 row `seo.hreflang_validator` status=implemented. Live cohort baseline next daily run; sanity tests pass on 5 mock scenarios + live evidalux.com ref call (3 declared codes, 0 invalid). Follow-up entry will append the band/agreement once measured.

Before

FN legend entry: 'missed (real-world: regulator catches what we didn't)' — authority unspecified.

After

FN legend entry: 'missed (real-world: a regulator — KVKK Kurul, EDPB, FTC, EAA enforcement, etc. — catches what we didn't)' — concrete authorities listed.

Before

{"scope": "cross_tool", "plugin": "seo.robots_txt_audit", "agreement_or_status": "PEND", "note_en": "Plan §9 line 566 pending; no reference implementation chosen.", "note_tr": "Plan §9 line 566 pending; referans implementasyon seçilmemişti."}

After

{"scope": "cross_tool", "plugin": "seo.robots_txt_audit", "agreement": 1.0, "band": "GREEN", "sites_compared": 10, "sites_agreed": 10, "run_id": "2026-05-14T05-55-01Z", "delta_en": "10/10 MATCH on the first CI run — third GREEN integration after search.lighthouse_seo and seo.structured_data. 8 sites fetched on both sides (bbc, gov.uk, iyzico, trendyol, hepsiburada, zalando, notion, koltukyataktemizleme) all return homepage-allowed for User-agent `*` and both sides agree. 2 sites empty/empty (evidalux + example.com — neither tool got a readable robots.txt; the boolean match metric treats this as agreement). Notably hepsiburada agreed here even though it 403'd validator.schema.org and Mozilla Observatory — robots.txt fetch is a different request fingerprint than the validator probes, so the anti-bot UA-block doesn't bite. Boolean homepage indexability is a coarse metric — disallow path-level set comparison would surface more potential delta. Plan §9 already flags path-set diff as a next-sprint follow-up; not a band concern.", "delta_tr": "İlk CI run'da 10/10 MATCH — search.lighthouse_seo ve seo.structured_data'dan sonra üçüncü GREEN entegrasyon. 8 site her iki tarafta da fetched (bbc, gov.uk, iyzico, trendyol, hepsiburada, zalando, notion, koltukyataktemizleme), hepsi User-agent `*` için homepage-allowed döndü ve iki taraf hemfikir. 2 site empty/empty (evidalux + example.com — ikisinin de okunabilir robots.txt'i yok; boolean match metrici bunu agreement sayıyor). Dikkat çekici: hepsiburada validator.schema.org ve Mozilla Observatory'de 403 verirken burada anlaştı — robots.txt fetch'i validator probe'larından farklı bir request fingerprint, anti-bot UA-block ısırmıyor. Boolean homepage indexability kaba bir metric — disallow path-level set karşılaştırması daha fazla potansiyel delta yüzeye çıkarır. Plan §9 path-set diff'i bir sonraki sprint follow-up'ı olarak işaretlemiş; band endişesi değil.", "follow_up_en": "(1) Disallow path-level Jaccard agreement — sample a handful of disallowed paths from both sides and compare set overlap; surfaces parser disagreement on wildcards, `$` anchors, comment handling. (2) Sitemap directive cross-check — Protego exposes `sitemaps()`; plugin currently only flags presence. Pairing both would add a second sub-metric. (3) Multi-UA matrix — homepage indexability for `Googlebot`, `GPTBot`, `CCBot` (relevant to aeo.llm_crawler_audit) so the robots.txt agreement folds into LLM crawler policy auditing.", "follow_up_tr": "(1) Disallow path-level Jaccard agreement — her iki taraftan bir avuç disallowed path örnekle ve set kesişimini karşılaştır; wildcard, `$` anchor, comment handling üzerinde parser uyuşmazlıklarını yüzeye çıkarır. (2) Sitemap directive cross-check — Protego `sitemaps()` expose ediyor; plugin şu an sadece varlığı flag'liyor. İkisini eşlemek ikinci bir alt-metric ekler. (3) Multi-UA matrix — `Googlebot`, `GPTBot`, `CCBot` için homepage indexability (aeo.llm_crawler_audit için relevant) — böylece robots.txt agreement LLM crawler policy auditing'ine de katlanır."}

Before

accessibility.axe: P=0.00, R=0.00, F1=0.00 · —

After

accessibility.axe: P=0.00, R=0.00, F1=0.00 · RED

Before

{"scope": "cross_tool", "note_en": "4 plugins in `error` status: search.lighthouse_seo, quality.lighthouse_perf, accessibility.axe, seo.broken_links. Public §2 showed four ERR pills.", "note_tr": "4 plugin `error` durumunda: search.lighthouse_seo, quality.lighthouse_perf, accessibility.axe, seo.broken_links. Public §2'de dört ERR pill görünüyordu."}

After

{"scope": "cross_tool", "note_en": "Partial recovery (1 of 4): `seo.broken_links` ↔ linkchecker is back to YELLOW 0.75 (close to pre-regression baseline 0.78). The other three still fail for different reasons surfaced by the fix: (a) `accessibility.axe`: Pa11y now starts but chromium executablePath hardcoded to `/usr/local/bin/chromium` (Docker path) — apt installs to `/usr/bin/chromium-browser` (issue #16). (b) `search.lighthouse_seo` + `quality.lighthouse_perf`: now report missing `GOOGLE_PSI_API_KEY` env (the underlying integration runs only when the secret is provided — plan §12 has a deadline of 2026-05-13 for this exact key).", "note_tr": "Kısmi recovery (4'ten 1'i): `seo.broken_links` ↔ linkchecker YELLOW 0.75'e döndü (regression öncesi baseline 0.78'e çok yakın). Diğer üçü düzeltmenin ortaya çıkardığı farklı nedenlerle hâlâ fail ediyor: (a) `accessibility.axe`: Pa11y artık başlıyor ama chromium executablePath `/usr/local/bin/chromium`'a hardcode (Docker path) — apt `/usr/bin/chromium-browser`'a kuruyor (issue #16). (b) `search.lighthouse_seo` + `quality.lighthouse_perf`: artık `GOOGLE_PSI_API_KEY` env eksikliğini raporluyor (alttaki entegrasyon sadece secret sağlandığında çalışır — plan §12'de tam bu key için 2026-05-13 deadline'ı var)."}

Before

{"scope": "cross_tool", "seo.structured_data": "PEND", "median_implemented_agreement": 0.5, "implemented_count": 5}

After

{"scope": "cross_tool", "seo.structured_data": {"agreement": 0.9, "band": "GREEN", "sites_compared": 10, "sites_agreed": 9, "delta_en": "9/10 MATCH on first try — the second GREEN integration after search.lighthouse_seo. bbc.co.uk: 4∩4 (NewsMediaOrganization, ItemList, CollectionPage, ImageObject). trendyol jac=0.87 (13∩15; plugin missed Country + EntryPoint). koltuk jac=0.88 (15∩17; plugin missed Country + Thing). 6 sites empty/empty (no structured data, both tools agree). 1 MISS — hepsiburada: ours=15 types (Answer/ContactPoint/FAQPage/ImageObject/MemberProgram/...) but ref=0. The validator received 0 triples — almost certainly anti-bot 403 (the same UA-block pattern Mozilla Observatory hit on the security.headers integration). Marking as cross-tool fingerprint divergence rather than plugin defect.", "delta_tr": "İlk denemede 9/10 MATCH — search.lighthouse_seo'dan sonra ikinci GREEN entegrasyon. bbc.co.uk: 4∩4 (NewsMediaOrganization, ItemList, CollectionPage, ImageObject). trendyol jac=0.87 (13∩15; plugin Country + EntryPoint kaçırdı). koltuk jac=0.88 (15∩17; plugin Country + Thing kaçırdı). 6 site empty/empty (no structured data, her iki araç hemfikir). 1 MISS — hepsiburada: ours=15 type (Answer/ContactPoint/FAQPage/ImageObject/MemberProgram/...) ama ref=0. Validator 0 triple aldı — neredeyse kesin anti-bot 403 (Mozilla Observatory'nin security.headers entegrasyonunda vurduğu aynı UA-block pattern). Plugin defect değil, cross-tool fingerprint divergence olarak işaretlendi."}, "median_implemented_agreement": 0.67, "implemented_count": 6, "follow_up_en": "(1) hepsiburada anti-bot 403 across multiple reference tools — pattern. Plan §9 already has a UA-block follow-up for security.headers; expand its scope to be cross-tool instead of per-plugin. (2) trendyol/koltuk plugin missed `Country` + `Thing` + `EntryPoint` — these are deeply nested types in JSON-LD graphs. Plugin's _collect_jsonld_types may be flattening only top-level @type. Check whether nested @type traversal is complete.", "follow_up_tr": "(1) hepsiburada birden fazla referans tool'da anti-bot 403 — pattern. Plan §9'da security.headers için UA-block follow-up'ı var; scope'u per-plugin yerine cross-tool olarak genişlet. (2) trendyol/koltuk plugin'i `Country` + `Thing` + `EntryPoint` kaçırdı — JSON-LD graph'larda derin nested type'lar. Plugin'in _collect_jsonld_types sadece top-level @type'ı flat'liyor olabilir. Nested @type traversal'ının tam olup olmadığını kontrol et."}

Before

{"scope": "cross_tool", "accessibility.axe": "PEND", "median_implemented_agreement": 0.59, "implemented_count": 4}

After

{"scope": "cross_tool", "accessibility.axe": {"agreement": 0.44, "band": "RED", "sites_compared": 9, "sites_agreed": 4, "errors": 1, "delta_en": "MATCH 4 (evidalux jac=0.5, example empty/empty, iyzico jac=0.6, koltuk jac=1.0). MISS 5 (bbc, gov.uk, hepsiburada, trendyol, zalando — Pa11y reports 1.1.1/1.3.1/2.4.1/4.1.1/4.1.3 SCs the plugin's axe-core doesn't, and the plugin emits 1.4.1/2.4.4/2.5.8/4.1.2/2.4.2 SCs Pa11y doesn't). 1 ERROR (notion: Pa11y navigation timeout 60s on heavy SPA). The disagreements aren't plugin defects — they're real coverage differences between axe-core 4.x rule pack and HTML_CodeSniffer's WCAG2AA criteria. axe-core 4.5+ deprecated 4.1.1 (modern HTML5 parsers handle the historical issue) — htmlcs still flags it. Conversely, axe has 2.5.8 (target size, WCAG 2.2) and 1.4.1 (use of color) checks htmlcs lacks.", "delta_tr": "MATCH 4 (evidalux jac=0.5, example empty/empty, iyzico jac=0.6, koltuk jac=1.0). MISS 5 (bbc, gov.uk, hepsiburada, trendyol, zalando — Pa11y plugin'in axe-core'unda olmayan 1.1.1/1.3.1/2.4.1/4.1.1/4.1.3 SC'lerini raporluyor, plugin Pa11y'de olmayan 1.4.1/2.4.4/2.5.8/4.1.2/2.4.2 SC'lerini emit ediyor). 1 ERROR (notion: Pa11y heavy SPA üzerinde 60s navigation timeout). Uyumsuzluklar plugin kusuru değil — axe-core 4.x rule pack ile HTML_CodeSniffer'ın WCAG2AA kriterleri arasında gerçek kapsam farkı. axe-core 4.5+ 4.1.1'i deprecated etti (modern HTML5 parser'lar tarihsel sorunu hallediyor) — htmlcs hâlâ flag ediyor. Tersine, axe'da 2.5.8 (target size, WCAG 2.2) ve 1.4.1 (use of color) kontrolleri htmlcs'de yok."}, "median_implemented_agreement": 0.5, "implemented_count": 5, "follow_up_en": "(1) Try Pa11y `-e axe,htmlcs` (combined runner) so the reference set is a superset rather than a complementary set — agreement should rise. (2) Pa11y timeout config 60→90 s for SPA-heavy sites like notion. (3) Document axe vs htmlcs WCAG coverage delta in customer-facing docs so reading 0.44 doesn't get misread as 'half the time we're wrong' — it's 'we agree on half the SCs, the rest are coverage-set mismatch'.", "follow_up_tr": "(1) Pa11y `-e axe,htmlcs` (combined runner) dene — referans set complementary değil superset olur, agreement yükselmeli. (2) Pa11y timeout config 60→90 sn SPA-ağır siteler için (notion). (3) axe vs htmlcs WCAG kapsam farkını müşteri-yüzü dökümana yaz; 0.44 okuyan biri 'yarı zaman yanlışız' diye yorumlamasın — '%50 SC'lerde anlaşıyoruz, kalanı kapsam-set uyumsuzluğu'."}

Before

seo.broken_links: P=0.00, R=0.00, F1=0.00 · RED

After

seo.broken_links: P=0.00, R=0.00, F1=0.00 · YELLOW

Before

seo.broken_links: P=0.00, R=0.00, F1=0.00 · RED

After

seo.broken_links: P=0.00, R=0.00, F1=0.00 · RED

Before

seo.broken_links: P=0.00, R=0.00, F1=0.00 · RED

After

seo.broken_links: P=0.00, R=0.00, F1=0.00 · RED

Before

{"scope": "cross_tool", "seo.broken_links": "PEND", "median_implemented_agreement": 0.5, "implemented_count": 3}

After

{"scope": "cross_tool", "seo.broken_links": {"agreement": 0.4, "band": "RED", "sites_compared": 5, "sites_agreed": 2, "errors": 5, "note_en": "5 sites timed out at 180 s (bbc, gov.uk, iyzico, hepsiburada, zalando — large sites; linkchecker recursion-2 burns through hundreds of links). 2 MATCH (example, koltuk — both empty/empty). 3 MISS expose two real bugs: (a) URL canonicalization — evidalux.com jaccard 0.0 even though both tools found the same 4 broken /legal/ URLs because plugin follows redirects and emits the www-stripped form while linkchecker keeps the surfaced form; same URL, different string. (b) Third-party-CDN treatment — trendyol+notion plugin reports CDN URLs (gtm, gstatic, ctfassets) as broken via HEAD probe 403, linkchecker treats them as valid. The plugin's confidence VERIFIED on those is wrong — anti-bot 403 isn't a reachability fact.", "note_tr": "5 site 180 sn'de timeout (bbc, gov.uk, iyzico, hepsiburada, zalando — büyük siteler; linkchecker recursion-2 yüzlerce link tarıyor). 2 MATCH (example, koltuk — her ikisi de empty/empty). 3 MISS iki gerçek bug'ı ortaya çıkardı: (a) URL canonicalization — evidalux.com jaccard 0.0 olmasına rağmen her iki araç da aynı 4 /legal/ URL'i broken bulmuş; plugin redirect takip edip www-stripped form yayıyor, linkchecker yüzeydeki formu koruyor; aynı URL, farklı string. (b) Third-party CDN muamelesi — trendyol+notion plugin'i CDN URL'lerini (gtm, gstatic, ctfassets) HEAD probe 403 üzerinden broken raporluyor, linkchecker geçerli sayıyor. Plugin'in bu bulgulardaki confidence VERIFIED yanlış — anti-bot 403 erişilebilirlik gerçeği değil."}, "median_implemented_agreement": 0.5, "implemented_count": 4, "follow_up_en": "Two real plugin defects + one harness-side timeout policy. (1) Plugin URL canonicalization fix: emit URLs in their as-linked form rather than the redirect-final form, OR have the cross_tool harness canonicalize both sides before set comparison (strip www, normalize trailing slash, lowercase host). (2) Plugin third-party probe semantics: HEAD-probe 403 from a CDN should not be a 'links.broken' finding under VERIFIED confidence — either downgrade to DETECTED with a 'likely-anti-bot' reason or skip 3xx-403 from known CDN domain list. (3) linkchecker timeout: 180 s cuts off enterprise sites mid-crawl. Options: bump to 600 s (slow but honest), narrow to recursion-1 (matches plugin's external-only HEAD probes more closely; less recall but more comparable scope), or run cohort sites in parallel via xargs.", "follow_up_tr": "İki gerçek plugin kusuru + bir harness timeout politikası. (1) Plugin URL canonicalization fix: URL'leri redirect-final form yerine link-edildiği orijinal formda yay, VEYA cross_tool harness'ı set karşılaştırması öncesi iki tarafı canonicalize etsin (www strip, trailing slash normalize, host lowercase). (2) Plugin third-party probe semantiği: CDN'den HEAD-probe 403 VERIFIED confidence ile 'links.broken' finding olmamalı — ya 'likely-anti-bot' sebebiyle DETECTED'e indir ya da bilinen CDN domain list'inden 3xx-403'leri atla. (3) linkchecker timeout: 180 sn enterprise siteleri tarama ortasında kesiyor. Seçenekler: 600 sn'ye çıkar (yavaş ama dürüst), recursion-1'e daralt (plugin'in external-only HEAD probe'larıyla daha eşleşir; daha az recall ama daha karşılaştırılabilir scope), veya cohort siteleri xargs ile paralel koş."}

Before

{"scope": "cross_tool", "search.lighthouse_seo": "PEND", "quality.lighthouse_perf": "PEND", "median_implemented_agreement": 0.4, "implemented_count": 1}

After

{"scope": "cross_tool", "search.lighthouse_seo": {"agreement": 1.0, "band": "GREEN", "sites_compared": 9, "sites_agreed": 9, "note_en": "1 site lost to PSI 500 (bbc.co.uk desktop) — sample drop, not disagreement", "note_tr": "1 site PSI 500 hatasıyla düştü (bbc.co.uk desktop) — sample kaybı, uyumsuzluk değil"}, "quality.lighthouse_perf": {"agreement": 0.5, "band": "RED", "sites_compared": 10, "sites_agreed": 5, "deltas_en": "performance category drives all 5 misses: bbc 29pt, iyzico 36, trendyol 57, hepsiburada 34, notion 33 — accessibility + best-practices stay within ±12. Pattern: container-bound Chromium hits CPU throttling that PSI's hosted cluster doesn't, so our 'performance' score systematically lower. Not a plugin defect — environment-driven measurement variance.", "deltas_tr": "5 MISS'in tamamı performance kategorisinden geliyor: bbc 29 puan, iyzico 36, trendyol 57, hepsiburada 34, notion 33 — accessibility + best-practices ±12 içinde kalıyor. Örüntü: container-bound Chromium PSI'nin hosted cluster'ında olmayan CPU throttling'e takılıyor, bu yüzden 'performance' skorumuz sistematik daha düşük. Plugin kusuru değil — environment kaynaklı ölçüm varyansı."}, "median_implemented_agreement": 0.5, "implemented_count": 3, "follow_up_en": "Three Faz 2 sub-items added: (1) parallelize Lighthouse subprocess across cohort sites — 10 sequential runs took 17 min, parallel-2 cuts to ~9 min; (2) document worker-browser perf-score gap publicly so customers reading our quality.lighthouse_perf report don't conflate environment variance with site regression; (3) consider pinning Lighthouse throttling to PSI's exact slow-4G + 4× CPU profile so deltas converge.", "follow_up_tr": "Üç Faz 2 alt-iş eklendi: (1) cohort siteleri arasında Lighthouse subprocess paralelleştir — 10 sequential run 17 dk sürdü, parallel-2 ile ~9 dk; (2) worker-browser perf-skor farkını public olarak belgele ki müşteriler quality.lighthouse_perf raporumuzu okurken environment varyansını site gerilemesiyle karıştırmasın; (3) Lighthouse throttling'i PSI'nin tam slow-4G + 4× CPU profiline sabitlemeyi düşün — deltalar yakınsasın."}

Before

{"scope": "cross_tool", "median_agreement": 0.0, "implemented_count": 0, "pending_count": 11, "_note": "every pair returned fake 0% RED via empty-set stubs"}

After

{"scope": "cross_tool", "median_agreement": 0.4, "implemented_count": 1, "pending_count": 13, "_note": "first real integration; honest baseline reveals scoring scale + UA-block issues"}

Before

median P=1.00, R=1.00 · 57/0/0

After

median P=1.00, R=1.00 · 59/0/0

Before

median P=1.00, R=1.00 · 53/0/0

After

median P=1.00, R=1.00 · 57/0/0

Before

median P=1.00, R=1.00 · 52/0/0

After

median P=1.00, R=1.00 · 53/0/0

Before

median P=1.00, R=1.00 · 51/0/0

After

median P=1.00, R=1.00 · 52/0/0

Before

median P=1.00, R=1.00 · 51/0/0

After

median P=1.00, R=1.00 · 51/0/0

Before

median P=1.00, R=1.00 · 48/0/0

After

median P=1.00, R=1.00 · 51/0/0

Before

median P=1.00, R=1.00 · 47/0/0

After

median P=1.00, R=1.00 · 48/0/0

Before

median P=1.00, R=1.00 · 41/0/0

After

median P=1.00, R=1.00 · 47/0/0

Before

accessibility.eaa_mapping: P=0.12, R=0.10, F1=0.11 · RED

After

accessibility.eaa_mapping: P=1.00, R=1.00, F1=1.00 · GREEN

Before

median P=1.00, R=1.00 · 38/0/0

After

median P=1.00, R=1.00 · 40/0/0

Before

compliance.privacy_policy_content: P=0.86, R=1.00, F1=0.92 · YELLOW

After

compliance.privacy_policy_content: P=1.00, R=1.00, F1=1.00 · GREEN

Before

compliance.verbis_registration: P=0.50, R=1.00, F1=0.67 · RED

After

compliance.verbis_registration: P=1.00, R=1.00, F1=1.00 · GREEN

Before

compliance.iab_tcf: P=0.00, R=0.00, F1=0.00 · RED

After

compliance.iab_tcf: P=1.00, R=1.00, F1=1.00 · GREEN

Before

median P=1.00, R=1.00 · 23/0/0

After

median P=1.00, R=1.00 · 38/0/0

Before

seo.broken_links: P=0.00, R=0.00, F1=0.00 · —

After

seo.broken_links: P=1.00, R=1.00, F1=1.00 · GREEN

Before

accessibility.axe: P=0.00, R=0.00, F1=0.00 · —

After

accessibility.axe: P=1.00, R=1.00, F1=1.00 · GREEN

Before

quality.lighthouse_perf: P=0.00, R=0.00, F1=0.00 · —

After

quality.lighthouse_perf: P=1.00, R=1.00, F1=1.00 · GREEN

Before

compliance.cookie_consent: P=0.00, R=0.00, F1=0.00 · —

After

compliance.cookie_consent: P=1.00, R=1.00, F1=1.00 · GREEN

Before

seo.freshness: P=0.80, R=1.00, F1=0.89 · RED

After

seo.freshness: P=1.00, R=1.00, F1=1.00 · GREEN

Before

security.tls_deep: P=0.00, R=0.00, F1=0.00 · —

After

security.tls_deep: P=1.00, R=1.00, F1=1.00 · GREEN

Before

Tier B plugins (Lighthouse, axe, dns_health, tls_deep, cookie_consent) couldn't be fixture-tested at all — real side-effects bypassed the harness.

After

median P=1.00, R=1.00 · 18/0/0

Before

4/7 archive entries 404'd; users saw dead links in a public transparency report.

After

All archive entries serve 200 directly from host filesystem; renderer changes auto-propagate without docker cp.

Before

median P=0.83, R=1.00 · 6/4/6

After

median P=1.00, R=1.00 · 9/3/4

Before

seo.duplicate_content: P=0.78, R=1.00, F1=0.88 · RED

After

seo.duplicate_content: P=1.00, R=1.00, F1=1.00 · GREEN

Before

median P=0.50, R=0.93 · 0/0/10

After

median P=0.83, R=1.00 · 6/4/6

Before

compliance.dark_pattern: P=0.86, R=1.00, F1=0.92 · YELLOW

After

compliance.dark_pattern: P=1.00, R=1.00, F1=1.00 · GREEN

Before

7 fixtures had expectations the plugin code didn't fulfil; FN/FP cycled with each calibration attempt.

After

Each gap documented in-fixture; calibration log carries the rationale; no silent test deletions.

Before

Day-2 baseline lost when Day-3 ran on same date.

After

Every run preserved as a permanent archive entry.

Before

security.headers: P=0.00, R=0.00, F1=0.00 · RED

After

security.headers: P=0.80, R=1.00, F1=0.89 · RED

Before

median P=0.50, R=0.93 · 0/0/10

After

median P=0.83, R=0.93 · 2/2/6