Software: OpenWPM

The core of WebTAP is OpenWPM, our open-source software for conducting automated Web Privacy Measurements on a scale of thousands to millions of websites. It is useful to researchers, journalists, regulators, privacy advocates, and anyone with an interest in online privacy.

OpenWPM is built on top of Firefox, with automation provided by Selenium, and includes several hooks for data collection, including simulating users’ browsing histories, using a proxy and enabling Firefox extensions, and recording observations (e.g., response metadata, cookies, behavior of scripts). Check out the Github repository for more information on the instrumentation and data collection.

In April 2019, OpenWPM transitioned away from Princeton WebTAP and is now maintained by Mozilla.

Data

Since 2015, we have conducted a web census to study third-party online tracking. Each month, we visit the web’s 1 million most popular sites using OpenWPM and record data pertaining to user privacy, including cookies, fingerprinting scripts, the effect of browser privacy tools, and the exchange of tracking data between different sites (“cookie syncing”).

We have released the entire Princeton Web Census data — about 15 terabytes — containing privacy measurements of 1 million sites conducted each month from December 2015 to June 2018.

Studies using OpenWPM

As of October 2020 OpenWPM has been used in 76 studies.

Year	Venue	Study Name
2014	ACM CCS	The Web Never Forgets: Persistent Tracking Mechanisms in the Wild
2014	ACM CoSN	Cognitive disconnect: Understanding Facebook Connect login permissions
2015	WWW	Cookies that give you away: The surveillance implications of web tracking
2015	NDSS	Upgrading HTTPS in midair: HSTS and key pinning in practice
2015	Tech Science	Web privacy census
2015	W2SP	Variations in tracking in relation to geographic location
2016	IFIP AICT	Evaluating Websites and Their Adherence to Data Protection Principles
2016	ACM CCS	Online Tracking: A 1-million-site Measurement and Analysis
2016	WWW	No honor among thieves: A large-scale analysis of malicious web shells
2017	NDSS	Dial One for Scam: A Large-Scale Analysis of Technical Support Scams
2017	PETS	Cross-Device Tracking: Measurement and Disclosures
2017	CODASPY	Identifying HTTPS-Protected Netflix Videos in Real-Time
2017	WWW	De-anonymizing Web Browsing Data with Social Networks*
2017	IWPE	Battery Status Not Included: Assessing Privacy in Web Standards
2017	Annual Privacy Forum	PrivacyScore: Improving Privacy and Security via Crowd-Sourced Benchmarks of Websites
2017	arXiv	Horcrux: A Password Manager for Paranoids
2017	USENIX Security	Measuring the Insecurity of Mobile Deep Links of Android
2017	Applied Economics Letters	Online advertising networks and consumer perceptions of privacy
2018	PETS	When the cookie meets the blockchain: Privacy risks of web payments via cryptocurrencies
2018	PETS	I never signed up for this! Privacy implications of email tracking
2018	ACM TOIT	Measuring third party tracker power across web and mobile
2018	CALIcon	Third Party Trackers on Law School Library Websites
2018	Master Thesis, Delft University of Technology	Tracking Cookies in the European Union, an Empirical Analysis of the Current Situation
2018	ACM CCS	The Web’s Sixth Sense: A Study of Scripts Accessing Smartphone Sensors
2018	ACSAC	Raising the Bar: Evaluating Origin-wide Security Manifests
2018	arXiv	The Unwanted Sharing Economy: An Analysis of Cookie Syncing and User Transparency under GDPR
2018	PhD thesis, Princeton University	Automated discovery of privacy violations on the web
2018	AINTEC’18	Understanding abusive web resources: characteristics and counter-measures of malicious web resources and cryptocurrency mining
2018	ACSAC	Raising the Bar: Evaluating Origin-wide Security Manifests
2018	SSRN	Acquisitions in the Third Party Tracking Industry: Competition and Data Protection Aspects
2019	Communications in Computer and Information Science	Transparency in Keyword Faceted Search: An Investigation on Google Shopping
2019	arXiv	The Price of Free Illegal Live Streaming Services
2019	Advances in Intelligent Systems and Computing	Usage of HTTPS by Municipal Websites in Portugal
2019	ConPro	The Impact of User Location on Cookie Notices (Inside and Outside of the European Union)
2019	WWW	Before and After GDPR: The Changes in Third Party Presence at Public and Private European Websites
2019	IEEE EuroS&P	TraffickStop: Detecting and Measuring Illicit Traffic Monetization Through Large-Scale DNS Analysis
2019	SSRN	The Market for Data Privacy
2019	ACM CSCW	Dark Patterns at Scale: Findings from a Crawl of 11K Shopping Websites
2019	Computer Communications	A comparison of web privacy protection techniques
2019	DPM	On Privacy Risks of Public WiFi Captive Portals
2019	Computers & Security	Towards a global perspective on web tracking
2019	APF	Towards Transparency in Email Tracking
2019	RAID	Talon: An Automated Framework for Cross-Device Tracking Detection
2019	ACM CCS	Watching You Watch: The Tracking Ecosystem of Over-the-Top TV Streaming Devices
2019	ACM IMC	Tales from the Porn: A Comprehensive Privacy Analysis of the Web Porn Ecosystem
2019	IEEE EuroS&P	TraffickStop: Detecting and Measuring Illicit Traffic Monetization Through Large-scale DNS Analysis
2019	The New York Times	I Visited 47 Sites. Hundreds of Trackers Followed Me.
2019	The Washington Post	Think you’re anonymous online? A third of popular websites are ‘fingerprinting’ you.
2019	ESORICS	Fingerprint surface-based detection of web bot detectors
2019	DPM	A Study on Subject Data Access in Online Advertising after the GDPR
2019	IEEE SPW	After GDPR, Still Tracking or Not? Understanding Opt-Out States for Online Behavioral Advertising
2020	PETS	Missed by Filter Lists: Detecting Unknown Third-Party Trackers with Invisible Pixels
2020	PETS	Inferring Tracker-Advertiser Relationships in the Online Advertising Ecosystem using Header Bidding
2020	PETS	A Comparative Measurement Study of Web Tracking on Mobile and Desktop Environments
2020	PETS	No boundaries: data exfiltration by third parties embedded on web pages
2020	PETS	In-Depth Evaluation of Redirect Tracking and Link Usage
2020	The Web Conference	The Representativeness of Automated Web Crawls as a Surrogate for Human Browsing✟
2020	The Web Conference	Apophanies or Epiphanies? How Crawlers Impact Our Understanding of the Web✟
2020	The Web Conference	Stop Tracking me Bro! Differential Tracking of User Demographics on Hyper-partisan Websites
2020	The Web Conference	Beyond the Front Page: Measuring Third Party Dynamics in the Field
2020	ACM ASIACCS	Measuring the Impact of the GDPR on Data Sharing in Ad Networks
2020	arXiv	Actions speak louder than words: Semi-supervised learning for browser fingerprinting detection
2020	PAM	Extortion or Expansion? An investigation into the costs and consequences of ICANN’s gTLD experiments
2020	Bachelor Thesis, Radboud University	Design and implementation of a stealthy OpenWPM web scraper
2020	IWPE	On Compliance of Cookie Purposes with the Purpose Specification Principle
2020	FTC PrivacyCon	Unaccounted Privacy Violation: A Comparative Analysis of Persistent Identification of Users Across Social Contexts
2020	IEEE EuroS&P	Multi-country Study of Third Party Trackers from Real Browser Histories
2020	TMA	Characterizing CNAME Cloaking-Based Tracking on the Web
2020	TMA	Clash of the Trackers: Measuring the Evolution of the Online Tracking Ecosystem
2020	WEIS	The Impact of the GDPR on Content Providers
2020	PhD Thesis, University of Michigan	Enhancing System Transparency, Trust, and Privacy with Internet Measurement
2020	Masters Thesis, Concordia University	A Large-Scale Evaluation of Privacy Practices of Public WiFi Captive Portals
2020	IEEE Globecom	A machine learning approach for detecting CNAME cloaking-based tracking on the Web
2021	NDSS	Reining in the Web’s Inconsistencies with Site Policy
2021	PETS	Unveiling Web Fingerprinting in the Wild Via Code Mining and Machine Learning
2021	IEEE S&P	Fingerprinting the Fingerprinters: Learning to Detect Browser Fingerprinting Behaviors

* Uses data released by us.
✟ Studies OpenWPM’s behavior.