Issue: The Reality of Modern Cookie Compliance
Over the years cookie compliance has become a deeply technical problem. Modern websites no longer serve a static set of cookies that can be easily identified. Instead, cookies are deployed dynamically based on a combination of user location, regulatory jurisdiction, consent state, JavaScript execution, and third-party tag orchestration.
In practice, this means the cookie experience of a user in the UK may differ materially from one in the EU, the US, or elsewhere. Consent banners themselves introduce further complexity: many sites suppress non-essential cookies until consent is granted, while others partially initialise tags using frameworks such as Google Consent Mode or IAB TCF. As a result, the true cookie footprint of a site only becomes visible after interacting with the consent layer.
Traditional cookie scanners struggle in this environment. Most rely on a single crawl, from a single location, without meaningful interaction. They capture what is present at first load, but miss cookies that can be injected later via JavaScript, fired conditionally after consent, or served only to users in specific regions. For teams responsible for GDPR and global privacy compliance, this creates a compliance blind spot and without the right tools decisions are being made on incomplete or misleading data.
The challenge, then, is not simply identifying cookies — it is understanding when, where, and why they are set.
Solution: A Consent-Aware, Multi-Location Scanning Architecture
To address these limitations, we engineered a cookie scanning tool designed around real user behaviour, modern web architecture, and regulatory nuance.
Playwright at the Core
At the heart of the scanner is Playwright. We chose Playwright over traditional crawling or headless browser tools due to its ability to reliably interact with modern, JavaScript-heavy websites. Playwright allows us to:
-
Accurately load single page applications (SPAs) and client-rendered applications
-
Wait for network idle states rather than arbitrary timeouts
-
Execute scripted interactions such as scrolling, navigation, and consent actions
-
Observe browser-level cookie state changes in real time
This ensures cookies loaded asynchronously, or triggered by user interaction, are captured consistently.
Multi-Location Scanning by Design
A key design principle was geographic fidelity. Websites frequently vary their behaviour based on user location whether its which consent banner is shown, to which third-party services are pre-enabled.
To reflect this, the scanner provisions fresh browser environments routed through region-specific network paths. Each scan can be executed from multiple global locations, allowing us to observe geo-specific cookie behaviour and banner logic as it would appear to real users in those regions.
This approach makes it possible to detect discrepancies that static, single-location scanners cannot, such as cookies that are only served to users in certain jurisdictions or consent frameworks that behave differently across regions.
Consent-Aware Analysis: Before and After Consent
One of the most significant differentiators is the scanner’s two-phase consent model.
For each scan, the system performs:
-
Pre-consent pass, recording all cookies set before any consent interaction.
-
Post-consent pass, where the scanner attempts to interact with the consent banner (for example, “Accept All”) and then re-scans the same pages.
This allows us to explicitly map:
-
Cookies set without consent
-
Cookies introduced only after consent
-
Cookies that persist across both states
Rather than guessing intent, the scanner observes actual browser behaviour, producing a clear and auditable view of the cookie lifecycle.
Data Normalisation and Reporting
Raw scan data is processed through a normalisation pipeline that consolidates cookies across pages, stages, and consent states. The final PDF report presents cookies grouped by category and consent timing, identifying those that only appear after consent and those that may warrant closer review.
The aim is clarity: enabling technical, legal, and compliance teams to understand what is happening without needing to interpret raw browser logs or network traces.
Technical Differentiators
Several aspects of the platform distinguish it from conventional cookie scanning tools:
- True dynamic analysis: Cookies triggered by JavaScript, delayed execution, or user interaction are captured.
- Consent-aware scanning: Explicit separation of pre- and post-consent behaviour.
- Multi-location execution: Geo-specific cookie sets and consent flows are observable.
- Non-intrusive operation: Scans are isolated and do not impact live users or site performance.
- Scalable architecture: Designed to run multiple concurrent scans without sacrificing depth or accuracy.
Together, these features enable a level of insight that traditional crawlers cannot provide.
Outcome
To validate the approach, we ran the scanner against a range of real websites, comparing results against typical single-pass scans. Although further development is planned, these initial results clearly illustrate the impact of consent-aware, multi-location analysis.
In one case involving a UK-based agency website, a traditional scan identified 10 cookies. Running the same site through the consent-aware scanner revealed 29 cookies once post-consent behaviour was observed – nearly a threefold increase.
A large London-based media and marketing agency site showed an even starker contrast. An initial scan detected just 3 cookies, while the consent-aware approach identified 48 cookies after consent interaction and full page execution.
Even in more controlled environments, differences were still evident. An international drinks brand website increased from 2 cookies identified in a static scan to 5 once post-consent behaviour was analysed.
These results underscore a consistent pattern: without interacting with consent banners and allowing modern sites to fully execute, a significant portion of a site’s cookie footprint remains invisible.
Conclusion: Built for Continuous, Real-World Compliance
This project was not designed to produce a checkbox report. It was engineered to reflect how modern websites actually behave, and how regulators increasingly expect organisations to understand their data practices.
By combining dynamic browser automation, consent-aware analysis, and multi-location execution, the scanner provides a more accurate representation of cookie behaviour across jurisdictions and consent states. As privacy regulations continue to evolve, tools that rely on static assumptions will struggle to keep pace.
Our approach treats cookie compliance as an ongoing, technical discipline that benefits from robust engineering, real-world simulation, and transparent reporting.
See the technical detail for yourself. Request a cookie scan of your website.
