Shadows in the Interface: A Comprehensive Study on Dark

Patterns
LIMING NIE, Shenzhen Technology University, China
YANGYANG ZHAO∗ , Zhejiang Sci-Tech University, China
CHENGLIN LI∗ , Zhejiang Sci-Tech University, China
XUQIONG LUO, Changsha University of Science and Technology, China
YANG LIU, Nanyang Technological University, Singapore
As digital interfaces become increasingly prevalent, a series of ethical issues have surfaced, with dark patterns
emerging as a key research focus. These manipulative design strategies are widely employed in User Interfaces
(UI) with the primary aim of steering user behavior in favor of service providers, often at the expense of the users
themselves. This paper aims to address three main challenges in the study of dark patterns: inconsistencies and
incompleteness in classification, limitations of detection tools, and inadequacies in data comprehensiveness.
In this paper, we introduce a comprehensive framework, called the Dark Pattern Analysis Framework (DPAF).
Using this framework, we construct a comprehensive taxonomy of dark patterns, encompassing 64 types, each
labeled with its impact on users and the likely scenarios in which it appears, validated through an industry
survey. When assessing the capabilities of the detection tools and the completeness of the dataset, we find
that of all dark patterns, the five detection tools can only identify 32, yielding a coverage rate of merely 50%.
Although the four existing datasets collectively contain 5,566 instances, they cover only 32 of all types of dark
patterns, also resulting in a total coverage rate of 50%. The results discussed above suggest that there is still
significant room for advancement in the field of dark pattern detection. Through this research, we not only
deepen our understanding of dark pattern classification and detection tools, but also offer valuable insights
for future research and practice in this field.
CCS Concepts: • Human-centered computing → HCI theory, concepts and models; Empirical studies in HCI .
Additional Key Words and Phrases: Dark Pattern, Taxonomy, Detection tools, GUI
ACM Reference Format:
Liming Nie, Yangyang Zhao, Chenglin Li, Xuqiong Luo, and Yang Liu. 2024. Shadows in the Interface: A
Comprehensive Study on Dark Patterns. Proc. ACM Softw. Eng. 1, FSE, Article 10 (July 2024), 22 pages.
https://doi.org/10.1145/3643736

1 INTRODUCTION
The emergence of digital interfaces has fundamentally transformed the way we interact with
technology, raising a series of ethical issues. Among these, "dark patterns" stand out as a significant
concern [18, 31, 34]. These manipulative design strategies pervade User Interfaces (UI), with the
aim of steering user behavior towards advantageous outcomes for service providers, often at the
expense of users [42, 49, 57]. Given their widespread presence and impact on user choices and
∗ Corresponding authors.

Authors’ addresses: Liming Nie, Shenzhen Technology University, Shenzhen, China, nieliming@sztu.edu.cn; Yangyang Zhao,
Zhejiang Sci-Tech University, Hangzhou, China, yangyangzhao@zstu.edu.cn; Chenglin Li, Zhejiang Sci-Tech University,
Hangzhou, China, lichenglin2110@163.com; Xuqiong Luo, Changsha University of Science and Technology, Changsha,
China, luoxuqiong@csust.edu.cn; Yang Liu, Nanyang Technological University, Singapore, Singapore, yangliu@ntu.edu.sg.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and
the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses,
contact the owner/author(s).
© 2024 Copyright held by the owner/author(s).
ACM 2994-970X/2024/7-ART10
https://doi.org/10.1145/3643736

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:2 Nie et al.

privacy, dark patterns have attracted substantial regulatory attention [14, 25, 43, 51]. However,
attempts to address this issue are hindered by the absence of a comprehensive framework for
understanding and classifying them, underlining the importance of establishing a robust taxonomy
and analysis framework.
Dark patterns have increasingly become the focus of academic research due to their increas-
ing ethical and practical implications across various digital platforms [32]. The seminal work of
Harry Brignull in 2010 laid the foundations for this field [17], offering an initial classification
system. Current research primarily focuses on two areas: classification and detection. The field
of classification has evolved significantly since its inception, moving from simplistic taxonomies
to more nuanced categories that consider various digital platforms and cognitive biases exploited
[29, 41]. Despite this, there is a continual effort to develop a universally accepted taxonomy that
encompasses newly identified dark patterns [23, 31, 50]. Similarly, detection methods have evolved
[20, 38, 48]. The initial approaches relied heavily on manual scrutiny, which, although accurate, is
impractical due to its time-consuming nature [37]. Subsequent semi-automated methods employed
clustering techniques to mimic user behavior [50]. Recent work has explored machine learning
approaches for specific contexts such as cookie banners but requires labeled training data [48].
However, current research faces several limitations: The first is that the prior taxonomies are
inconsistent and incomplete. Most studies are confined to specific types or contexts and lack a
comprehensive taxonomy. Meanwhile, existing taxonomies often overlook user impact and likely
scenarios where dark patterns appear. The second is the limitations of the detection tools. Existing
tools often suffer from incomplete coverage or low accuracy. No study has yet analyzed the full
range of the capabilities of these tools in a complete taxonomy. The third is the completeness of
the data. Previous research often relies on limited datasets, which may not be representative, and
there is a notable absence of holistic analysis on their coverage and utility.
In this paper, we introduce the Dark Pattern Analysis Framework (DPAF), a novel two-stage
approach aimed at addressing existing gaps in the field. Initially, the framework employs a systematic
literature review to assess the current state of dark pattern taxonomies. Subsequently, it embarks on
taxonomy construction and labeling. Specifically, we conduct a relatively comprehensive taxonomy
of dark patterns that not only integrates existing taxonomies, but also adds the types not included
in previous taxonomies. In addition, each type is labeled with its impact on users and the likely
scenarios in which it appears. Following this, we evaluate the capabilities of existing dark pattern
detection tools and examine the extent to which existing datasets encompass the types of dark
patterns identified by the newly constructed taxonomy. Through this approach, our aim is to answer
three research questions (RQ):

• RQ1: Are current dark pattern taxonomies comprehensive? Our systematic review highlights
the incompleteness of existing dark pattern taxonomies. To fill this gap, we have created
the most exhaustive dark pattern taxonomy to date, comprising 64 unique types, validated
through an industry survey. We have also labeled each type based on its impact on users and
the contexts in which it is likely to appear, enhancing the taxonomy’s applicability.
• RQ2: What are the capabilities and limitations of current Dark Pattern detection tools? Of
the 64 types of dark patterns, the five selected tools could identify only 32, with a coverage
rate of only 50%. Furthermore, these tools do not identify the remaining 32 patterns, which
constitute 50% of the total.
• RQ3: What is the current status of the available data supporting automated Dark Pattern
recognition? The four available datasets contain 5,566 instances, covering only 32 of the
64 types of dark patterns, with a coverage rate of 50%. The absence of instances for the
remaining 32 types limits the capabilities of current detection tools.

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
Shadows in the Interface: A Comprehensive Study on Dark Patterns 10:3

By answering these questions, this paper deepens our understanding of dark pattern classification
and detection tools, offering valuable insights for the field’s future research and practice. For further
details, please consult our online website [7]. Our contributions are as follows:
• Conducted the first systematic literature review focused solely on dark patterns.
• Established a comprehensive dark pattern taxonomy, fostering standardized communication
among various stakeholders.
• Identified current research limitations and suggested directions for future studies.

2 BACKGROUND AND MOTIVATION

2.1 Dark Patterns
In contemporary society, User Interface (UI) and User Experience (UX) design have become deeply
embedded in our daily lives. Whether shopping, socializing, or working, we constantly interact with
various applications and online platforms. The design of these platforms directly influences our
decisions, emotions, and behaviors. An adeptly designed interface can streamline tasks, enhance
user experience, and even nudge consumers to make wiser choices. However, with technological
advancements and intensifying business competition, a design strategy called "dark patterns" has
gradually emerged.
Dark patterns, a term introduced in the scholarly literature [13, 19], refer to manipulative or
deceptive elements in UX design. These are deliberately constructed designs with the intent of
inducing users to take actions they might not willingly choose if they were fully informed or
aware of alternatives [50]. This strategy capitalizes on individuals’ cognitive biases and habitual
behaviors to achieve specific commercial or other objectives, such as enticing users to mistakenly
click on advertisements, unknowingly agree to subscriptions, or share personal data without a
complete understanding. Dark patterns show themselves in multiple forms, such as misleading
buttons, hidden fees at checkout, complex unsubscribe procedures, and ads disguised as authentic
content. Such design practices may erode trust in products or services and, at times, might even
incur legal liabilities, posing potential adverse impacts on users, businesses, and the broader
online ecosystem [26, 61]. Fundamentally, dark patterns are considered unethical design tactics
because they manipulate and deceive users, coercing them into choices that benefit on-line services,
frequently to the detriment of the users’ best interests.

2.2 Motivation
The increasing prevalence of dark patterns in user interfaces is raising significant concerns. These
deceptive designs can undermine user trust, nudging them towards decisions that are not in their
best interest, potentially diminishing user engagement and retention. Beyond mere manipulation,
dark patterns can coerce users into unintentional purchases or the sharing of sensitive information,
posing legal challenges to companies and violating user rights. Furthermore, by instilling a sense of
urgency, these patterns increase psychological stress, which could evolve into wider mental health
problems. From the standpoint of data security, dark patterns increase the risk of data breaches
when users are duped into providing too much information. Although dark patterns might improve
short-term metrics, the extensive and adverse long-term effects on both individuals and society
warrant serious consideration.
The widespread use of dark patterns and their potential negative effects highlight an urgent
need to better understand and identify these deceptive practices. This is crucial to preventing
their development and improving the overall quality of software. However, tackling dark patterns
presents numerous challenges in technical, conceptual, and practical aspects. Key obstacles include:

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:4 Nie et al.

• Inconsistencies and Incompleteness in Classification: Although significant research effort has

been aimed at creating classification frameworks for dark patterns, much of this work is
limited to certain types or contexts, without a unified and standardized categorization. An
all-encompassing taxonomy that includes dark patterns from various sources has yet to be
established. This lack of standardization hampers efforts to investigate, identify, and address
these manipulative design practices.
• Limitations of Detection Tools: Given the potential hazards posed by dark patterns, the creation
and implementation of automated detection tools are crucial. These tools can serve as a
deterrent, encouraging designers and developers to follow ethical and user-focused design
principles. However, the detection effectiveness of current tools varies, and many struggle
to achieve thorough coverage or maintain accuracy. Critical assessment of the capabilities
and limitations of these tools is vital for fostering further investigation and improvement in
this area. Despite the clear need for such evaluations, research dedicated to examining these
aspects has been conspicuously absent to date.
• Inadequacies in Data Comprehensiveness: The effectiveness of dark pattern detection tools is
significantly influenced by the quality and size of the datasets they rely on. Inadequacies or
biases in these datasets can diminish the effectiveness of the tools. Numerous studies have
used limited datasets that may not fully represent the broader context, with a notable absence
of comprehensive analysis on their inclusion and applicability. Therefore, it is essential to
evaluate current datasets, focusing on their diversity and thoroughness.
In light of these existing challenges, a comprehensive exploration of dark patterns becomes
imperative. Motivated by this urgency,our research seeks to bridge the existing knowledge gaps,
offering a robust unified classification standard and a detailed examination of the current dark
pattern detection tools and available data. Through our endeavors, we strive to advance both
academic discourse and practical interventions related to dark patterns.

3 METHODOLOGY
In this section, we first address three research questions and the intentions behind them. Subse-
quently, the overall framework of the corresponding empirical study is introduced. Following that,
we will outline each segment and step within the proposed framework in detail, especially focusing
on the construction of the dark pattern taxonomy, annotation of tool capabilities and limitations,
and the process of marking data availability.

3.1 Research Questions(RQs)

• RQ1: Are current dark pattern taxonomies comprehensive? To effectively understand, detect,
and intervene with dark patterns, a comprehensive and widely accepted taxonomy is needed.
A clear and comprehensive taxonomy can help researchers and relevant practitioners better
grasp the diversity and complexity of dark patterns. It can strongly support the formulation
of related policies and regulations and can also help improve the capabilities of related
detection tools. Therefore, building the most comprehensive and industry-accepted dark
pattern taxonomy is of paramount importance.
• RQ2: What are the capabilities and limitations of current Dark Pattern detection tools?
Identification of dark patterns is time-consuming and requires specialized expert experience.
Although some automated detection tools have been proposed, the capabilities and limitations
of these detection tools have not been adequately researched or understood. Therefore, a
thorough investigation of these aspects is crucial for further optimizing these tools, enhancing
their accuracy and reliability, and expanding their scope of application.

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
Shadows in the Interface: A Comprehensive Study on Dark Patterns 10:5

1. Systematic Literature Review 2. Taxonomy Construction and Marking

Construction of Taxonomy 2
Scientific Literature Review 1

Experts evaluation
Prior Taxonomies Update

Preliminary Update RQ1.

Candidate papers Deduplicate Papers
Augmented Optimized Taxonomy
759 374 Taxonomy
Taxonomy
Regulatory Reports
Merge Select

Making of Tools Capability and Limitation 3 Tool capability making

Libraries Relevant papers
57
Manual
Selected papers RQ2.
Detection marking Tool Capabilities and
76 Tools Capability
Tools Limitations Table
Search Snowballing

Added papers Making of Existing Datasets Availability 4

Search Query
19 Data availability making
Manual
marking RQ3.
Dataset
Dataset Available Benchmark Datasets Data availability
Dataset

Fig. 1. The framework of DPAF

• RQ3: What is the current status of the available data supporting automated Dark Pattern
recognition? A significant obstacle hindering research and solutions related to dark patterns is
the lack of comprehensive and reliable datasets. However, the current status of data availability
has not been adequately studied. Thus, a deep analysis of the availability of existing data and
its actual coverage in the comprehensive dark pattern taxonomy is necessary and urgent.

3.2 Framework
To address the three research questions mentioned above, we propose a framework, named Dark
Pattern Analysis Framework (DPAF), based on the taxonomy construction method used by Ladisa et
al.[45]. Our framework entails constructing a taxonomy for dark patterns, annotating the capabilities
and limitations of existing detection tools within this taxonomy, marking the availability of data,
and an update mechanism. As shown in Figure 1, DPAF comprises two stages: Systematic Literature
Review and Taxonomy Construction and Marking. We will discuss each stage and the specific steps
involved.

3.3 Scientific Literature Review

This section describes a systematic review of academic literature related to dark pattern research 1 .
To achieve this, we followed a structured methodology to perform a literature review, encompassing
various stages[39, 54].
3.3.1 Design of Search Query. Our systematic review of dark pattern research began with the
design of an effective search query. To ensure a robust query, we initiated an exploratory search
on Google Scholar using the keyword "dark pattern." This preliminary step revealed four relevant
studies [18, 40, 48, 50] that provided initial insight into the topic and helped us identify related
keywords. Subsequently, we analyzed these keywords to construct a search query string that would
yield the most relevant papers on dark patterns. The search query is as follows:
(Dark patterns OR dark nudging OR Digital sludging OR Manipulation techniques OR
Deception techniques OR deceptive button OR dark pattern recognition OR dark
pattern classification AND GUI OR UI OR user interface OR graphical user interface)

3.3.2 Literature Search. With our well-crafted search query in hand, we proceeded to conduct an
extensive literature search across five reputable databases: IEEE Xplore, Scopus, Google Scholar,

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:6 Nie et al.

ACM Digital Library, and arXiv. Specifically, we incorporated selected arXiv papers to ensure a
comprehensive view of current research. We limited our search scope to titles, abstracts, and author
keywords and focused exclusively on publications in the English language, including journals,
conference proceedings, and book chapters.

3.3.3 Merge Candidate Literature. Following our systematic literature search, we recorded the
number of candidate literature retrieved from each database. The breakdown was as follows: Google
Scholar (461), IEEE Xplore (107), Scopus (83), ACM Digital Library (91), and arXiv (17). In total, we
initially identified 759 candidate papers.

3.3.4 Deduplication. To ensure the integrity of our review, we meticulously removed duplicate
entries from the pool of candidate literature. This process resulted in a refined set of 374 papers
that were subsequently subjected to the next stage of paper selection based on inclusion criteria.

3.3.5 Paper Selection. The paper selection process was carried out in two phases. During the initial
phase, we conducted a preselection by scrutinizing the meta-information of each paper, specifically
focusing on publication type, title, and abstract. Papers that were clearly irrelevant to dark patterns
in the UIs were excluded during this phase. This initial screening reduced the candidate literature
to 143 articles, which advanced to the second phase of selection. In the second phase, we acquired
the full text of all remaining papers and conducted a comprehensive examination of their content.
Specifically, we dived into the introduction, methods, and results sections to confirm whether each
paper presented a study related to dark patterns in UIs. Furthermore, all sections of the study had
to be written in English to be considered. Papers that did not meet these criteria were rejected.
To improve the precision and consistency of our review, a parallel review strategy was adopted.
Each piece of literature was independently reviewed by at least two authors. Inclusion in our final
review required a consensus on the relevance of the article among the authors. At the end of this
rigorous selection process, we identified 57 relevant papers.

3.3.6 Snowballing. To further enhance the completeness of our selection, we adopted a bidirectional
snowballing strategy, following references both forward and backward from the 57 papers initially
deemed relevant. This approach applied the established selection criteria consistently across all
rounds of snowballing. We included only those documents that were directly cited or were cited
by our chosen literature, capping the inclusion at a maximum of two levels of citation depth to
maintain focus and manageability.This iterative process led to the inclusion of 19 additional new
papers. Consequently, we arrive at a final selection of 76 relevant papers that formed the foundation
of our comprehensive study of dark patterns.
Of the 76 papers on dark patterns, 7 originate from arXiv. Subsequent checks revealed that 4 are
formally published, leaving 3 exclusive to arXiv. Refer to our website[7] for more information. Note
that these 3 papers did not reveal new dark pattern types. Figure 2 shows the publishing trends per
year according to the 76 selected papers. These publications were meticulously reviewed to extract
details related to prevalent dark patterns and the recognition and classification of such patterns.
These extracted data serve as the basis for answering our RQs.

3.4 Construction of Taxonomy

This subsection delineates the methodology for constructing an optimized taxonomy 2 , as shown
in Figure 1, which serves as an augmented version accepted by the industry. To achieve this aim, the
process is divided into three sequential stages: First, we construct a preliminary enhanced taxonomy
that serves as an enhanced version of existing taxonomies. Second, we conduct a survey to industry
experts to assess the rationality, completeness, and usefulness of this preliminary taxonomy. Finally,

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
Shadows in the Interface: A Comprehensive Study on Dark Patterns 10:7

Fig. 2. Annual Publication Trend of Dark Patterns-Related Research Papers

based on the feedback received from these experts, the taxonomy is refined to produce the final
optimized taxonomy.
3.4.1 Building an Preliminary Augmented Taxonomy. The main goal of this step is to build an
augmented version of the taxonomy based on previous work on dark patterns. Previous studies
mainly included prior taxonomies [15, 19, 31, 32, 46, 50], reports [10, 11, 21, 47, 55], and some dark
patterns that appeared in the articles but were not included in prior taxonomies [20, 24, 30, 35, 44,
48, 61]. In addition to this, we annotate each type of dark pattern contained within our Preliminary
Augmented Taxonomy in terms of its potential user impact and application scenarios. The objective
is to gain a more comprehensive understanding of the potential effects of dark patterns on users,
as well as their scope of application across various contexts. To achieve the objectives stated above,
we follow a multistep approach divided into four principal phases.
A. Aggregation of Existing Taxonomies. First, from 76 related articles, we identified 15 key
taxonomies that provide a substantive basis for our work. We manually extracted existing dark
pattern types along with their classification information from the prior taxonomies.
Second, using the latest taxonomy proposed by Gray et al.[32] as a baseline, we consolidated
similar dark patterns to avoid redundancy. This consolidation was classified into two categories:
Textual Similarity and Conceptual Similarity. On the one hand, textual similarity refers to dark
patterns that either have the exact same definition, point to the same dark pattern instance, or use
synonyms. For these types of patterns, we only need to record their paper sources, descriptions, and
the type attribution of these patterns in the original taxonomy version. For those whose categories
are inconsistent, three of the authors will discuss and determine the final category based on existing
classifications and the principle of majority submission. On the other hand, conceptual similarity
means that patterns are different in terms but similar in nature. Because we do not know in advance
which dark patterns are conceptually similar, we use manual confirmation to check all patterns,
except textual similarity. Specifically, two of the authors will read each pattern separately. The
text descriptions, related examples, and attribution information corresponding to the remaining
unconfirmed patterns give you your own conclusion, that is, they either belong to the same dark
pattern or should belong to a separate pattern. After the two authors completed all the annotations,
find patterns with differences in the annotations. For these annotations, a third author was asked
to make independent judgments and finally negotiated. The outcome of this phase was an initial
version of the most comprehensive taxonomy that covers all known dark patterns.
Conceptual similarity is for more complex similar dark pattern judgments. It mainly includes
the following steps: a). similarity judgment of description: for the definition of dark patterns,
although they may be described by different words, they may express the same or similar meanings.
These kinds of dark patterns can be merged into the same class of dark patterns. For all detection
tools, the corpus is an important basis for detecting dark patterns. Therefore, this step is a way of

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:8 Nie et al.

categorization in line with the detection basis. b). Judging by the similarity of the presentation of
instances: most of the existing detection tools take visual cue as an important basis for detecting
dark patterns. Therefore, we care whether the typical instances of the two types of dark pattern
have similarity in their expressions. For example, the graphical interfaces when they appear and the
descriptions. c). Judgment of Potential Impact on Users: In order to be consistent with the real-world
discussion of dark patterns, we include the different types of impact that dark patterns may have
on users as an additional judgmental step. The authors carefully reviewed five rules proposed by
regulatory agencies or organizations. These articles made generalizations about dark patterns, and
some of their comments were adopted by us in constructing the dark pattern taxonomy.
In addition, in previous dark pattern classification methods, almost all existing detection tools
use the smallest unit of dark pattern type as a detection unit. Because they are widely discussed,
they are retained as representative types during the construction of the taxonomy. This means that
they are typically not merged into other dark pattern types. Given the importance of classification
for detection tools, our website offers specific examples of merging to illustrate the process of
combining dark pattern types more effectively.
B. Add Missing Dark Patterns. In building upon the initial taxonomy of dark patterns, which
aggregated findings from prior literature, our objective is to construct the most comprehensive
taxonomy to date. Despite the extensive categorization already provided in existing taxonomies,
such as the seminal work by Gray et al. [32], we identified coverage gaps that may have resulted
from divergent focal points in previous research. To rectify this, we undertook a deep reading
of all relevant literature, comprising 76 papers, in order to broaden the scope of our taxonomy.
Specifically. we have supplemented the latest taxonomy from [32] by incorporating additional 10
dark patterns, thereby establishing a preliminary taxonomy for the upcoming survey phase.
C. Assessment of User Impact. Although much of the prior research has primarily focused on the
identification and description of dark patterns, there has been a comparatively limited exploration
of the end impact these patterns impose on consumers. In an attempt to offer a comprehensive
understanding of the multi-dimensional ramifications that all variants of dark patterns could
potentially exert on users, we engage in a systematic annotation of each identified type of dark
pattern with its associated harms. For the purposes of this study, we employ the foundational
framework proposed by the Organization for Economic Cooperation and Development (OECD) to
categorize the harms of dark patterns [55]. This framework identifies dark pattern harms from six
unique vantage points, which, for ease of reference and clarity, we herein label as H1 to H6:

• Harm to User Autonomy (H1): This category suggests that certain dark patterns compromise
user autonomy by forcing consumers to make choices they would not otherwise make,
limiting the available options, and obfuscating the decision-making process.
• Personal User Detriment: This domain encompasses three separate, yet interconnected, harms:
Financial Loss (H2), Privacy Harms (H3), and Psychological Detriment and Time Loss (H4). H2
addresses dark patterns that manipulate consumers into making unnecessary purchases or
overspending. H3 emphasizes how dark patterns could induce users into unintentionally
disseminating excessive personal information, thereby elevating their risk exposure. H4
reflects on dark patterns that place emotional and cognitive strain on users, exploiting their
vulnerabilities and potentially leading to time wastage.
• Structural User Detriment: This facet includes two distinct, yet interrelated, harms: Weaker or
Distorted Competition (H5) and Reduced Consumer Trust and Engagement (H6). H5 means that
dark patterns can distort market competition by preventing or dissuading consumers from
shopping and comparing offers. H6 posits that dark patterns may erode consumer trust and

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
Shadows in the Interface: A Comprehensive Study on Dark Patterns 10:9

engagement by deceiving users into oversharing information or overspending, particularly if

such practices are subsequently uncovered.

In alignment with the aforesaid definitions, we performed a meticulous assessment of each

identified dark pattern to gauge its potential impact on users. This process consisted of two pivotal
phases: First, we unambiguously included those dark patterns that were explicitly cited by the
OECD as emblematic instances of particular kinds of harm. These were directly incorporated into
our analytical framework. Subsequently, for any unaccounted for dark patterns that were not
explicitly delineated, our interdisciplinary team of three authors undertook rigorous collective
discussions. These discussions aimed to systematically assign potential harms to users, guided by
the definitions and corresponding examples established by the OECD [55].
D. Common Scenarios. To gain a more comprehensive understanding of the possible impacts of
dark patterns on users, particularly in different application scenarios, it is imperative to annotate
the scenarios in which these patterns manifest. Such annotation serves a dual purpose: it not only
identifies the broad applications of dark patterns but also delves deeper into their potential impacts
within diverse user groups and environments. Towards this aim, we have deployed a two-fold
methodological approach for systematically annotating various dark pattern scenarios.
Initially, we amalgamated dark pattern scenarios discussed in previous research [37, 43] and
categorized them into four distinct classes: Social Platforms (S), E-commerce (E), Entertainment
Software (ES), and General or Universal (All). These classifications serve as a robust foundation for
the subsequent annotation of dark patterns. In the second step, we embark on an intricate and
detailed annotation process. Initially, we extract pertinent data regarding the potential impact of
dark patterns across various scenarios from existing scholarly publications. To elaborate, suppose a
particular paper concentrates on categorizing types of dark patterns prevalent on social platforms
[59]. During our data acquisition phase, we would not only incorporate all types of dark patterns
detailed in that paper into our taxonomy but also label them under the dimension of Social Platform
to denote the scope of their impact. This approach might result in a specific type of dark pattern
being discussed in multiple articles centered around different application scenarios, thus acquiring
multiple tags. This is, in fact, an expected outcome. Our taxonomy specifically identifies three key
application scenarios: Social Platforms, E-commerce, and Entertainment Software. Therefore, should
a dark pattern type be prevalent across these three scenarios, we assign it the designation All.
At this point, we can complete the annotation of all dark pattern type impact scenarios, thereby
furnishing a nuanced and comprehensive framework that enhances our understanding of how
these patterns operate across various contexts.
Upon the successful completion of these key steps, we have established a taxonomy of existing
dark patterns that is both preliminary and comprehensive. This taxonomy not only encompasses all
dark patterns cited in current academic research and regulatory reports, but also increases the two
dimensions of severity and impact scenarios to understand the potential impact of dark patterns on
users and the different application scenarios they affect.

3.4.2 Conducting the Expert Survey. The primary aim of this online survey is to gather feedback
from industry professionals, with a particular focus on designers and UI-related developers. The
survey addresses multiple focal points: a).Assess the rationality, completeness, understandability,
and usefulness of the dark pattern taxonomy conducted by our work. b). Assess the effectiveness
of the annotations in terms of the degree of impact on users and common scenarios. c).Assess the
efficacy of existing tools that detect dark patterns. d).Assess the availability of existing dark pattern
datasets. e).Collect information from practitioners on their awareness and strategies for identifying
and mitigating dark patterns.

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:10 Nie et al.

The entire survey construction process consists of the following three parts: The first is the design
of the questionnaire. The questionnaire comprises 16 questions. Various aspects of focal points
are evaluated using a Likert scale [9], ranging from 1 (low) to 5 (high). A minority of questions
use binary options, such as whether participants have actually read the taxonomy document and
background materials provided by us. Another binary question assesses whether participants
feel adequately informed to provide valid feedback. These are critical factors for the quality of
the survey data. Additionally, the questionnaire also includes demographic questions that cover
participants’ age, tenure in their respective industries, and background information. The second is
the distribution and scoring method. Snowball sampling techniques are utilized to reach potential
participants [28]. Initial recruitment is targeted at designers and UI-related developers with more
than 5 years of industry experience, which is later expanded to encompass general practitioners in
related fields. The primary channels of recruitment include social media platforms and industry
forums. Participants are required to first read the taxonomy document and background materials
provided by us and subsequently rate each addressed question. The third is privacy safeguards. To
ensure the confidentiality and anonymity of participants, the survey does not collect personally
identifiable information. The data collection process adheres to the principle of data minimization.
Responses are securely stored, and only aggregated data is reported. The survey is developed
using Wjx.cn, a comprehensive platform for survey design, data collection, and analysis. For the
actual setting of the questionnaire question, please refer to our website[7]. The survey started
on September 1, 2023, and ended on September 25, 2023. The survey collected feedback from 176
respondents.

3.4.3 Updating the Taxonomy. Upon receiving feedback from participants, we will engage in a
systematic process to update our existing taxonomy. Initially, we will analyze all the responses
collected, meticulously reviewing any contentious points or suggested modifications. Then more
interviews with industry experts will be conducted to determine whether the proposed changes
should be implemented. Second, any alterations to the taxonomy will be rigorously documented,
with annotations explaining the rationale behind each modification. This framework ensures that
the taxonomy remains dynamic and evolves in accordance with expert opinions and industry needs.
Furthermore, to account for the dynamic nature of dark patterns, especially as user interfaces
continue to evolve and new types of dark patterns emerge, it is imperative to incorporate a robust
maintenance mechanism within DPAF. This mechanism would involve periodic reviews and updates
to the taxonomy, facilitated by automated scraping algorithms and expert consultations. In doing
so, DPAF remains an adaptive tool that not only captures the current state of dark patterns, but
also anticipates and adapts to future developments.
3.5 Analysis of Detection Tools
We conducted an in-depth analysis of the 76 articles we collated (refer to Section 3.3) with the aim
of unearthing all automated tools adept at identifying dark patterns 3 . To streamline our selection
process and ensure a focused analysis, we devised the following criteria. Only those tools that met
all the criteria were chosen as the subjects of our investigation.

• Criterion #1 (Dark Pattern Recognition): It is imperative for the tools to have a direct
bearing on dark pattern identification tasks. Their descriptions should encompass verbiage
that resonates with keywords such as "identification," "detection," and "classification."
• Criterion #2 (Automation): Recognizing that manual identification has inherent drawbacks
like increased time consumption and reduced efficiency, our analysis zeroes in exclusively on
tools offering automated capabilities.

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
Shadows in the Interface: A Comprehensive Study on Dark Patterns 10:11

Table 1. Final Optimized Taxonomy of Dark Patterns

Common AidUI UIGuard DY AGM ADD ContextDP-D Rico’-D AGM-D LLE-D
Categories Sub-Categories Dark Pattern Types Sources Severity
Scenarios§ [48] [20] [52] [50] [22] (301) [5] (1660) [1] (1818) [2] (1787) [4]
Nagging† - - [20, 31, 46] H3,H4,H5 All ✓ ✓ ✓(57) ✓(188) ✓
Obstruction Roach Motel Immortal Accounts† [15, 19, 20, 31, 46, 50] H1,H2,H3,H5 All ✓ ✓ ✓ ✓(31) ✓
Dead End† [11] H1,H2,H4 S ✓ ✓ ✓ ✓
Forced Grace Period* [51] H1,H4 All
Creating Barriers Price Comparison [19, 20, 31, 46] H1,H2,H5,H6 E ✓ ✓
Prevention†
Intermediate Currency† [20, 31, 46] H1,H2,H5 S/ES ✓ ✓
Adding Steps Privacy Maze† [11] H1,H3 S
Labyrinthine Naviga- [51] H1,H4 S
tion*
Sneaking Bait And Switch Disguised Ad [19, 20, 31, 46] H4,H6 All ✓ ✓ ✓ ✓ ✓(21) ✓(46) ✓
Hiding Information Sneak Into Basket† [19, 31, 46, 50] H1,H2,H5 E ✓ ✓ ✓(21) ✓
Drip Pricing, Hidden [19, 20, 31, 46, 50] H2,H5,H6 E ✓ ✓ ✓(5)
Costs, Or Partitioned
Pricing†
Reference Pricing† [10, 55] H2,H5,H6 E
(De)Contextualizing Conflicting Information [11, 30] H4,H6 S
Cues
Information Without [11] H4 S
Context
Interface In- Manipulating Visual False Hierarchy† [20, 31, 46] H2 All ✓ ✓ ✓(273) ✓(25) ✓
terference Choice Architecture
Visual Prominence [11] H1,H2 All
Bundling [10] H1 E
Pressured Selling† [21, 50] H2,H4.H6 E ✓ ✓(67)
Persuasive Language*† [51] H1,H2,H4 E/S
Hard To Close * Small or Moving Close [20] H4 All ✓ ✓(684)
Button*
Bad Defaults/Pre- - [15, 20, 31, 46] H1,H2,H3,H6 S ✓ ✓ ✓(111) ✓(232) ✓
Selection†
Emotional Or Sen- Cuteness† [44, 50] H2,H5 E
sory Manipulation
Positive Or Negative [20, 50] H2,H3,H4 All ✓ ✓
Framing
Fear Of Missing Out*† [51] E/S
Trick Questions† - [19, 20, 31, 46, 50] H2,H4,H6 E ✓ ✓ ✓ ✓(9) ✓
Choice Overload - [10, 11] H4 S
Hidden Information† - [31, 46] H3,H5,H6 All ✓
Language Inaccessi- Wrong Language† [11] H3,H4 All
bility
Complex Language [10] H4 All
Feedforward Ambi- - [11, 30] H4 All
guity
Plain Evil* - [51] H1,H4 S
Engaging Strategy* Addictive Design* [51] H1,H4 S
Infnite Scrolling* [51] H1,H4 S
Pull To Refresh* [51] H1,H4 S
Reduced Friction* [51] H4 S
Customisation* [51] H4 S
Forced Ac- Forced Continuity† - [19, 20, 31, 46, 50] H1,H2,H5 E ✓ ✓ ✓(1)
tion
Forced Registration [15, 46, 50] H1,H5 E/S ✓ ✓(6) ✓
Forced Communica- Privacy Zuckering† [19, 20, 31, 46] H1,H3 E/S ✓ ✓ ✓(117) ✓
tion Or Disclosure
Friend Spam† [19, 46] H1,H3 E/S ✓ ✓
Address Book Leeching† [15, 46] H1,H3 E/S
Social Pyramid† [20, 31, 46] H1,H3 All ✓ ✓(16) ✓
Granting and Interac- [51] H1 S
tion*
Gamification Pay-To-Play† [21, 31, 46] H1,H2 ES ✓ ✓ ✓(11)
Grinding [21, 31, 46, 61] H1,H4 ES ✓ ✓
Playing By Appoint- [61] H1 ES
ment*
Forced Advertise- Countdown On Ads* [20] H4 All ✓ ✓(4) ✓
ment*
Watch Ads To Unlock [20] H4 All ✓ ✓(2) ✓
Features Or Get Re-
wards*
Pay To Avoid* [20] H1,H4 ES ✓ ✓(97) ✓
Automatic Execu- Automating The User [30] H1,H4 All
tion* Away*
Automatic Accept Third [51] H1 S
Party Term*
Attention Capture Auto-Play [21] H1,H5 All ✓ ✓ ✓(13)
Social Engi- Scarcity And Popular- High Demand† [46, 50] H2,H6 E ✓ ✓ ✓(5) ✓(47)
neering ity Claims
Social Proof Low Stock† [46, 50] H2,H6 E ✓ ✓ ✓(19) ✓(632)
Endorsement And Testi- [46, 50] H2,H6 E ✓ ✓(12)
monials
Parasocial Pressure† [21, 46, 50] H2,H6 E
Urgency Activity Messages† [46, 50] H2,H6 E ✓ ✓ ✓(10) ✓(313)
Countdown Timer [46, 50] H2,H6 E ✓ ✓ ✓(28) ✓(393)
Shaming Limited Time Message† [46, 50] H2,H6 E ✓ ✓ ✓(26) ✓(88)
Personalization Confirmshaming† [19, 46, 50] H2 E ✓ ✓ ✓(169)
Shades Of Grey* Encouraging Anti-Social [61] H4 ES
Behavior*
Psychological Tricks* [61] H4,H6 ES
Games For Other Pur- [61] H4,H6 ES
poses*
Pre-Delivered - [61] H2 ES
Content*†
§:"Common scenarios" refers to the application scenarios in which the dark mode is present. It encompasses four distinct
categories: Social Platforms (S), E-commerce (E), Entertainment Software (ES), and General or Universal (All). Dark patterns
identified with a ’†’ symbol signify actions or features that result in financial losses and privacy harms for users, highlighting
the necessity to prioritize their detection in the realm of dark pattern recognition. Dark patterns labeled with a ’*’ denote
newly introduced dark patterns.

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:12 Nie et al.

Table 2. Tools List for Dark Patterns Detection

Tools Technology Platform Automaticity Availability

AidUI [48] Visual Cue Detection based on deep learning; Spatial Analysis; Text Analysis; Color Analysis. Web & Mobile Automatic Yes [6]
UIguard [20] UI extraction based on Faster-RCNN; Text Analysis; Icon Semantic Understanding; UI Color Mobile Automatic No
Extraction; Knowledge-Driven checker
DY [52] Hierarchical and k-means cluster analysis algorithms Web Automatic No
AGM [50] Corpus creation; Crawler; Hierarchical Clustering using HDBSCAN; Manual Examination Web Semi-Automatic Yes [3]
ADD [22] Web crawling and web scraping techniques; Natural language processing Web Automatic No
SSS [58] Text extraction and analysis Web Automatic No

• Criterion #3 (Functional Description): Clarity is crucial. As such, each tool must be

supported by comprehensive and clear documentation, to ensure a seamless evaluation of its
features and functionalities.
• Criterion #4 (Types of Dark Patterns Detected): The documentation of each tool should
elucidate the specific dark patterns that it can detect.
In light of criteria #1 through #3, six tools emerged as potent candidates for dark pattern
recognition. Table 2 provides a detailed overview of these tools. The first column lists the tool’s
name and source. In instances where tools lacked a designated name, we adopted an abbreviation
strategy, leveraging the initial letters of the first three authors of the corresponding paper. The next
column outlines the primary technologies employed by each tool. This is followed by a column
specifying the tool’s target platform, either web or mobile. Lastly, columns indicate the tool’s
automation capability and its public availability. Furthermore, RQ2 underscores the importance
of dissecting the prowess of various tools in the realm of dark pattern identification. However,
as Table 2 reveals, most of these tools are not open source. Therefore, for such tools, it is crucial
that their documentation transparently enumerates the dark patterns that they are proficient in
detecting. Adherence to criterion #4 resulted in the exclusion of SSS [58] from our list. Consequently,
our finalized list for in-depth study comprises five tools: AidUI[48], UIGuard[20], DY[52], AGM [50],
and ADD[22].
In our evaluation of selected dark pattern detection tools, we employed a manual marking strategy
to categorize their claimed or actual detection capabilities within the taxonomy we developed.
Initially, we conducted a comprehensive review of the academic literature, documentation, and
code of each tool, open source or commercial, to meticulously compile a list of dark pattern types
that each tool purportedly or effectively recognizes. Following this process, we systematically
annotated these capabilities within our constructed dark pattern taxonomy. During the practical
evaluation phase, out of the six tools evaluated, only AidUI[48] was executable, with the others
either lacking accessible source or executable code or having non-functional source code. For
AidUI, we conducted an in-depth analysis of its detection capabilities after execution. For tools that
could not be executed, we relied on an extensive review of their source code and documentation to
evaluate their claimed detection abilities.
It is worth noting that if a certain dark pattern a tool claims to identify has been merged with
other similar dark patterns in our taxonomy, the newly amalgamated pattern within our taxonomy
is annotated as recognizable.

3.6 Analysis of Datasets

To explore whether current public datasets suffice for studies related to dark patterns, we embarked
on an evaluation of the comprehensiveness of these datasets in terms of dark patterns 4 . To achieve
this, we established precise criteria for the selection of the dataset, ensuring alignment with the
intricacies of dark pattern detection.

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
Shadows in the Interface: A Comprehensive Study on Dark Patterns 10:13

Table 3. Datasets List for Dark Patterns

Dataset C/J Publish Year Platform # Instances #Patterns coverage Availability

ContextDP-D [48] ICSE 2023 Web & Mobile 301 10 Yes [5]
Rico’-D [20] HCI 2023 Mobile 1,660 12 Yes [1]
AGM-D [50] HCI 2019 Web 1,818 15 Yes [2]
LLE-D [24] CHI 2020 Mobile 1787 16 Permission required [4]

Initially, we delved into our repository of 76 dark pattern literature, aiming to pinpoint all papers
that describe or allude to datasets containing dark pattern instances. Our dataset acquisition was
steered by two fundamental criteria:
• Criterion #1 (Presence of Dark Pattern Instances): The dataset should explicitly con-
tain screenshots or videos of web pages or mobile apps with dark patterns. This provides
researchers with a solid foundation from which to extract and comprehend the features of
dark patterns.
• Criterion #2 (Open Accessibility): The dataset must be publicly disclosed by its authors
and readily available without constraints.
Employing these criteria, our literature review yielded 4 articles that touched on datasets cap-
turing dark pattern instances. Among these, 3 were primarily oriented toward the exploration
of dark pattern detection tools, while one dedicated its focus to charting the prevalence of dark
patterns. Table 3 provides a detailed overview of the primary information for these four datasets.
The first column describes the name and source of the datasets. Notably, in instances where the
datasets lacked a distinct designation by their authors, we instituted a uniform naming convention:
using the initials of the first three authors, complemented by the "-D" suffix. This is followed by the
conference or journal of the associated paper and its publication year.The fourth column specifies
the target platform, either web or mobile. The fifth and sixth columns depict the scale of the dataset
(number of instances it encompasses) and the count of dark pattern types it covers, respectively.
The last column highlights the availability, complemented by a direct link if applicable. As indicated
in the table, except for LLE-D [24], all other datasets are directly accessible.
For the four datasets selected, we enumerated the types of dark patterns present within each
and documented their respective instance counts. Within our dark pattern taxonomy, we denoted
the spectrum of dark patterns encompassed by each dataset. Consistent with our prior approach to
marking tool capability, if a covered dark pattern has been merged with another dark pattern, we
would denote the merged dark pattern as present in this dataset.

4 RESULTS AND INSIGHTS

This section introduces the dark pattern taxonomy constructed based on our proposed DPAF
framework, as well as the annotations regarding tool capabilities and data availability made on the
taxonomy, to address the three research questions.

4.1 Taxonomy
For RQ1, after following the steps outlined in Section 3.4, from the 76 scientific literature, we have
constructed the most comprehensive and standardized taxonomy of dark patterns to date, as shown
in Table 1.
In general, this taxonomy encompasses 64 Dark Patterns Types of dark patterns(in the third
column). These types belong to 6 categories (in the first column) and 31 sub-categories (in the
second column). Compared to the most recent taxonomy [32], our taxonomy has added 7 new sub-
categories and 20 new Dark Patterns Types. These newly added patterns are annotated with asterisks

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:14 Nie et al.

Table 4. Feedback Results of Survey

# Feedback # Valid feedback Rationality Completeness Understandability Usefulness Severity Annotation Scenarios Annotation
176 173 100% 92% 87% 92% 100% 100%

in Table 1. The ’Severity’ column indicates the types of impact each dark pattern has on users.
Among the identified dark patterns, 49 of 64 types, which account for 77%, exhibit multifaceted
impacts on users, as evidenced by their association with multiple labels. Among all the H1-H6
impact labels, we find that H2 and H3 are the most severe, as they cause financial loss and privacy
leakage, respectively. In the table, 33 dark patterns are labeled with these two severe types. If
priority ranking is needed for dark pattern identification tasks, those labeled H2 and H3 should
be identified first. The ’Common Scenarios’ column specifies the potential settings where each
type of dark pattern may appear. Fifteen types of dark patterns are prevalent in all three business
scenarios: Social Platforms (S), E-Commerce (E), and Entertainment Software (ES), demonstrating
their ubiquity. Furthermore, 17 types are unique to Social Platforms, 17 to E-Commerce, and 9 to
Enterprise Software, with 6 types appearing in both E and S settings. Analyzing the likely scenarios
for each dark pattern type helps to provide information support to stakeholders such as researchers,
designers, and policymakers.
These findings have been widely supported by industry experts. Specifically, as shown in Table 4,
an analysis of the feedback from all 173 valid participants reveals that more than 87% believe our dark
patterns taxonomy is essentially rationality, completeness, understandability, and usefulness, rating
it 4 or 5 points. The general consensus is that our annotations on Severity and Common Scenarios
are reasonable. Furthermore, we received suggestions from 10 participants who recommended
including screenshots for each dark pattern type for a better understanding. This was the main
reason for the slightly lower approval rate with respect to comprehensibility. Due to the considerable
manual annotation work involved, we plan to introduce an interactive website in future research
to facilitate a more intuitive understanding of dark patterns[7]. Furthermore, through the survey,
we found that 61% of the participants were aware of the existence of dark patterns in the apps
they developed, 23% were not aware, and 15% were uncertain. In terms of avoiding dark patterns,
30% reported that their organizations had adopted strategies to evade them, 23% said that no such
measures were in place, and the remaining 46% were unaware. This indicates a general lack of
awareness and proactive measures against dark patterns in the industry. For more information,
please refer to our website.

Answer to RQ1: Through a review of 76 scientific papers, we identified and generalized

64 unique types of dark patterns. We further organized and annotated these patterns
according to their categories, descriptions, impacts on users, and common scenarios. This
taxonomy was validated by industry participants, with more than 87% of responses deeming
it rationality, completeness, understandability and usefulness.

4.2 Capability of Detection Tools

For RQ2, our objective is to evaluate the capabilities of various dark pattern detection tools in
alignment with our comprehensive taxonomy. Following the procedures introduced in Section 3.5,
we selected a total of five different dark pattern detection tools, including AidUI[48], UIGuard[20],
DY[52], AGM[50], and ADD[22]. As illustrated in Table 2, among the five tools, three are designed
to detect dark patterns in web pages, one is designed for mobile apps, and the other is for both
platforms. These tools use advanced techniques to identify dark patterns, such as machine learning
and natural language processing. Except for AGM, which automates most of its processes but still

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
Shadows in the Interface: A Comprehensive Study on Dark Patterns 10:15

Fig. 3. Comprehensive Dark Pattern Instances Count Across Three Datasets

requires manual intervention, all other tools claim to be fully automated. In addition, among these
tools, AidUI and AGM are open source, while other tools are currently not available.
To further investigate the efficacy of these tools, Table 1 delves into the detection capabilities
of these tools in relation to our comprehensive dark pattern taxonomy. The 7 to 11 columns of
Table 1 correspond to each tool, marked with ✓if the tool can identify the respective dark pattern.
As illustrated in the table, these five tools collectively detect 32 distinct dark patterns, achieving
a coverage rate of 50% (=32/64). Most detectable dark patterns can be identified using only one
or two tools. Specifically, 11 of these dark patterns are detected solely by one tool, while 20 are
identified by 2 or 3 tools. Remarkably, only one dark pattern (i.e., "Disguised Ad") can be recognized
by four of the tools. Intriguingly, none of these 32 dark patterns is universally detected across
all tools. Considering the capabilities of individual tools, they can identify between 5 to 15 types
of dark patterns. Specifically, UIGuard and AGM can detect the most, identifying up to 15 dark
patterns, while ADD detects the fewest, pinpointing only 5 dark patterns. It suggests that these tools,
employing varied technical approaches and focusing on different research aspects, demonstrate
significant discrepancies in the types of dark patterns that they can identify. Moreover, in relation to
the 64 dark patterns we have cataloged, the current tools demonstrate a relatively low recognition
rate. The highest recognition rate is only 23. 4%, that is, 15 of 64). This underscores the inadequacy
of the detection capabilities of existing tools.
Additionally, there are 32 dark patterns, comprising 50% of the total, that remain undetected by
any of the tools. The inability to detect these patterns could be attributed to various reasons: certain
patterns might be inherently intricate, heightening detection challenges; some might manifest
in diverse manners, complicating the formulation of clear detection rules; or certain patterns
may seldom surface in real-world scenarios, leading to an insufficiency of representative samples,
which in turn hampers a comprehensive understanding and identification. These observations
highlight that the current dark pattern detection tools have substantial limitations, reflecting a lack
of comprehensiveness and robustness.
The results demonstrate a significant gap between the breadth of dark patterns identified in our
taxonomy and those currently detectable by existing tools. This underscores the need for more
advanced and comprehensive tools to protect users from a wider array of dark patterns.

Answer to RQ2: From the five tools selected, they collectively identify 32 out of the 64
dark patterns from our taxonomy, achieving a 50% coverage. However, 32 dark patterns
(50% of the total) remain undetected by all the tools.

4.3 Availability of Datasets

For RQ3, through an in-depth analysis of the selected datasets, our objective is to examine their cov-
erage in relation to the taxonomy we have established, allowing us to assess the comprehensiveness
and integrity of these datasets.

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:16 Nie et al.

As described in section 3.6, we selected a total of four datasets containing dark pattern in-
stances,including ContextDP-D [48], Rico’-D [20], AGM-D [50] and LLE-D [24]. Table 3 reveals a
collective count of 5,566 instances in these datasets. More specifically, Rico’-D and LLE-D contain
1,660 and 1,787 dark pattern instances from mobile apps, respectively. AGM-D encompasses 1,818
instances exclusively from web pages. And ContextDP-D has a smaller count of 301 instances from
both platforms.
To delve deeper into the current data coverage of dark patterns, Table 1 elucidates the types
of dark patterns encompassed by each dataset. The final four columns correspond to each of the
datasets. When a particular dark pattern instance is present within a dataset, we mark the respective
cell as "covered" with a checkmark and denote the specific instance count of that dark pattern.
After merging these datasets, Figure 3 provides a statistical count of the instances for each covered
dark pattern, represented in the form of a bar chart. It is worth noting that while the LLE-D dataset
is claimed publicly available, it requires permission, hindering direct access and retrieval of the
data. Consequently, in our evaluation of the comprehensiveness of this dataset, we primarily rely
on descriptions provided in the associated literature. Only when it explicitly mentions the inclusion
of a specific dark pattern instance within its dataset do we mark it as covered in our table. Given
our inability to access this dataset, we remain uninformed about the precise count of certain dark
pattern instances. Hence, in subsequent result descriptions, statistics involving instance counts
will exclude this particular dataset.
From Table 1, it is discernible that these four datasets collectively cover only 32 types of dark
patterns, resulting in an overall coverage rate of 50% (that is, 32/64). When considering each dataset
individually, the range of covered dark patterns narrows further, capturing merely 10 to 16 distinct
types. Furthermore, as shown in Figure 3, there is a significant disparity in the number of instances
for each type of dark pattern within the datasets. In particular, the instance counts for "Low Stock"
and "Small or Moving Close Button" stand out, both exceeding 600 instances. This suggests that
these dark patterns might be more prevalent in everyday applications, making their instances more
readily collectible. In contrast, 6 dark patterns exhibit substantially fewer instances, with types
like "Tick questions", "Forced Continuity" and "Countdown on Ads" having counts of less than
ten. This pronounced data imbalance could potentially jeopardize the performance of machine
learning-based dark pattern prediction and classification, introducing model biases and thereby
compromising model accuracy and robustness.
Furthermore, the absence of 32 types of dark patterns (accounting for 50%) in the datasets implies
that machine learning models may fail to acquire the features of these patterns, rendering them
ineffective in recognizing and classifying such instances. The possible reasons for the absence of
these dark patterns in the datasets can be delineated as follows: (1)Ambiguity in Definitions and
Absence of Precise Criteria: The ambiguity surrounding the definitions of some dark patterns often
impedes researchers from conclusively identifying them on screencasts of mobile applications or
web pages. For instance, while "complex language" is described as "Decision-relevant information
can be deliberately or unintentionally made difficult to understand by using complex language" [10,
56], there is no concrete benchmark that establishes what qualifies as complex language. This lack
of clarity might lead researchers astray when trying to pinpoint such patterns. (2) Dependence
on Contextual and Historical User Interaction: Some dark patterns are not confined to a singular
interface and manifest themselves based on user-interaction history or a broader context. As such,
they elude immediate detection on standalone interfaces, making their data collection challenging.
An illustrative example is "Address Book Leeching", where services conduct database searches
upon users importing their contact lists. Although these services may store these contact details
for ulterior reasons, not initially disclosed to users, malicious intent may surface only at a later
interaction juncture [15]. The current method of collecting dark pattern instance data mostly

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
Shadows in the Interface: A Comprehensive Study on Dark Patterns 10:17

relies on manual identification and labeling of screenshots or videos of mobile applications or web
pages, which lacks the capability to identify such nuanced patterns based on a single interface.
Coincidentally, both detection tools and datasets cover 50% of the dark pattern taxonomy but exhibit
slight variations in the types of dark pattern they cover. Both cover 31 types, with differences in
Friend Spam (detectable but not covered by the datasets) and Hidden Information (covered by the
datasets but undetectable by the tools).
The findings from our dataset analysis carry significant implications for the domain of dark
pattern detection using machine learning techniques. The evident data imbalance, with certain
patterns being over-represented and others under-represented or even absent. Models trained
on such datasets might exhibit heightened sensitivity to more frequent patterns and diminished
capability to recognize rarer ones. Additionally, the complete absence of 32 dark pattern types
underscores a glaring gap in current datasets, limiting the model’s capacity to detect a wide spectrum
of dark patterns. Hence, for reliable and holistic dark pattern detection, there is a pressing need for
more balanced and exhaustive datasets, encompassing the diversity of dark patterns encountered
in real-world scenarios.

Answer to RQ3: The four datasets collectively contain 5,566 instances, and only cover 32
distinct dark patterns, yielding a total coverage rate of 50% (i.e., 32 out of 64 types). The
omission of the other 32 dark patterns, which also make up 50% of the total, restricts the
ability to recognize these patterns.

5 DISCUSSION
5.1 Implications
Our research in constructing a comprehensive taxonomy for dark patterns has multiple implications
for both the academic and industrial communities.
• Standardization: The taxonomy provides a standard nomenclature for classifying dark pat-
terns, thus reducing ambiguity and facilitating better communication among researchers and
practitioners.
• Impact Assessment: By annotating the dimensions of user impact and potential application
scenarios for each dark pattern, we offer a nuanced understanding that can guide design
decisions and policy interventions.
• Tool Evaluation: Our annotation of the detection capabilities of existing tools serves as a
benchmark for evaluating and improving dark pattern detection mechanisms.
• Data-Driven Insights: With the inclusion of the number of available instances for each type
of dark pattern, our taxonomy becomes a data-driven tool that can guide future research and
applications.

5.2 Threats to Validity

5.2.1 Internal Threats. The manual labeling process for identifying the dimensions of user impact
and application scenarios is prone to subjectivity and potential inaccuracies. Although we mitigated
this risk through expert consultations, the element of human error cannot be completely eliminated.
The study relies on industry surveys to validate the taxonomy. The quality of this validation is
highly dependent on the respondent’s understanding and interpretation of the taxonomy, thereby
introducing another layer of subjectivity. The methods used to evaluate existing dark pattern
detection tools may not cover all the possible configurations and nuances, thus affecting the
completeness of our evaluation.

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:18 Nie et al.

5.2.2 External Threats. First, our validation is based on surveys from industry professionals. How-
ever, the samples may not be fully representative of the diverse practitioners in this field. Second,
the study focuses primarily on industry professionals who are more likely to be familiar with
western design paradigms. The taxonomy and its implications might not be universally applicable
in different cultural and geographical contexts. Third, we evaluated existing dark pattern detection
tools based on their capabilities at the time of our study. These tools themselves are subject to
updates and improvements that might affect the relevance of our assessment in the future.

5.3 Challenges
The research landscape of dark patterns is complex due to advanced algorithms, data challenges,
and tool usability. Our study provides a foundational framework but faces several avenues for future
research and challenges. First, the quality and quantity of data are essential for the robustness of
our taxonomy. Despite efforts to ensure accuracy through expert consultations, the manual labeling
process still bears the risk of error and subjectivity. Second, the development of open-source tools
for detecting dark patterns is crucial but comes with evaluation challenges. Current methodologies
may not fully capture the diversity and subtleties of dark patterns, risking bias, and undermining
generalizability. Third, longitudinal studies[12], while valuable, are complicated by extended time
frames, analytical complexity, and the dynamic nature of digital interfaces. These factors could
compromise the reliability and generalizability of the findings. Lastly, ethical considerations are
paramount. With increasing regulatory focus, agile ethical guidelines are needed to accompany
our taxonomy, which requires a dynamic approach to its development.
In summary, our study is a stepping stone in the field of dark patterns, and ongoing multidisci-
plinary efforts are needed to refine methodologies and tools to adapt to the ever-evolving nature of
dark patterns.

6 RELATED WORK
In this section, we describe previous research related to dark patterns, with a focus on their
classification and detection. We begin by summarizing the work that classifies dark patterns
through the proposal of taxonomies and that suggests techniques for detecting these patterns
within GUIs. Subsequent subsections elaborate on the classification and detection aspects.

6.1 Classification of Dark Patterns

Numerous empirical studies have explored the concept of dark patterns on various digital platforms,
including websites, mobile apps [23, 31, 50], and diverse domains such as gaming [8], social media
platforms [36], e-commerce websites [50], cookie banners [33, 53], and advertisements [27]. The
typical approach in these empirical studies involves the collection and examination of user interfaces
from desktop or mobile applications. The researchers then consolidate their findings, identifying
and categorizing the types of dark patterns embedded in these interfaces.
This journey began in 2010 when Harry Brignull [17] established the concept of dark patterns,
defining them as manipulative techniques employed by websites and apps to force users into
unintended actions, such as making purchases or signing up for services. Brignull also launched a
Twitter account [16] for users to report and discuss instances of dark patterns they encountered in
their daily digital interactions. Subsequently, Gray et al. [31] conducted manual data collection from
popular online platforms over a two-month period. They expanded Brignull’s concept by classifying
dark patterns into five strategic categories: nagging, obstruction, sneaking, etc. Building on these
taxonomies, Mathur et al. [50] executed a large-scale empirical study, focusing on around 11,000
shopping websites. Their approach involved simulating user shopping behaviors and clustering
segments of the collected websites. In contrast to Mathur et al.’s concentration on e-commerce

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
Shadows in the Interface: A Comprehensive Study on Dark Patterns 10:19

platforms, Di Geronimo et al. [23] examined the presence of dark pattern elements in 240 widely
used Android applications. They achieved this by recording 10-minute usage videos for each
application and adopted taxonomies derived from Gray et al. [32]. Their work identified 16 new
dark pattern types comprising 31 cases.
The research most relevant to our study is conducted by Gray et al. [32], who developed a
comparatively comprehensive taxonomy of dark patterns. Unlike their work, in this paper, we
synthesize all previous taxonomies through a systematic literature analysis and, building on the
taxonomy established by Gray et al., introduce 20 important and previously unrecorded types.
Furthermore, based on the taxonomy we constructed, we meticulously mark and analyze the
recognition capabilities of existing detection tools and the coverage rate of datasets.

6.2 Detection of Dark Patterns

Various strategies have been employed to detect dark patterns, ranging from manual exploration to
semi-automated clustering methods and basic text-based classification techniques.
Detection of dark patterns was initially based on manual exploration, conducted by domain
experts [17, 23, 31, 36, 38] or ordinary users[16]. To address the limitations of manual exploration,
Mathur et al. [50] introduced semi-automated techniques. These methods simulate user behavior
to uncover dark patterns within shopping websites. They employed clustering techniques to
group related UI patterns, making the detection process more efficient. Based on their findings
and dataset, some researchers [60] explored classification techniques to detect dark patterns in
real-world scenarios. However, these techniques are text-based and limited in scope, possibly
resulting in numerous false positives. Additionally, they rely on labeled training data for dark
patterns. Recently, Daniel [38] developed DarkDialogs, a system designed to autonomously extract
consent dialogs from websites and identify the existence of 10 dark patterns. After that, more and
more automated dark pattern detection tools were proposed. Andrea et al. [22] combined natural
language processing techniques with web crawling and scraping techniques to design and develop
a framework to automatically detect potential instances of web-based dark patterns. For the mobile
app, Hrushikesava et al. [58] designed an application through which any identified dark pattern
advertisement can be viewed and warned through a dialog box. Dmitry et al. [52] proposed to
implement dark pattern detection on user interface sites using a cluster analysis algorithm that uses
both hierarchical and k-means methods to cluster many dark patterns in the application interface.
In the latest dark pattern detection tools, image processing has emerged as a key technique to
improve the capability of dark pattern detection tools. Mansur et al. [48]proposed the AidUI tool by
combining techniques such as visual processing and text analysis, which demonstrated excellent
performance for dark pattern detection. Chen et al. [20] proposed UIGuard by extracting the
UI interface and then combining the text analysis of the UI interface along with icon semantic
understanding and UI color analysis, obtaining good results. Based on previous work, our paper
assesses the capabilities and limitations of existing tools for detecting dark patterns in the taxonomy
we built. We also examine various benchmark datasets used to detect dark patterns.

7 CONCLUSION
In summary, this study represents a significant milestone in the rapidly evolving field of dark
pattern research. Through a meticulous systematic review, we have identified 76 seminal papers,
allowing us to construct the most comprehensive standardized taxonomy of dark patterns to date.
This taxonomy not only includes types and descriptions of each dark pattern, their categorizations,
but also annotates the impact on users and potential application scenarios. Furthermore, we have
marked the capabilities of existing dark pattern detection tools and the availability of current data

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:20 Nie et al.

within this taxonomy. Our taxonomy serves as both a standard nomenclature for classifying dark
patterns and a practical tool for researchers, practitioners, and policymakers alike.
The road ahead is fraught with challenges that require a multidisciplinary approach and ongoing
refinement of methods and tools. The landscape of dark patterns is in constant flux, influenced by
rapid technological advancements and shifting societal norms. As such, future research must focus
on developing more robust methodologies, including open-source tools and ethical guidelines, that
can adapt to this ever-changing landscape.

ACKNOWLEDGMENTS
We thank the reviewers for their insightful comments and suggestions. This work was partially
funded by the National Natural Science Foundation of China (61972359, 62132014), Zhejiang
Provincial Key Research and Development Program of China (2022C01045), Zhejiang Provin-
cial Natural Science Foundation of China (LQ23F020020), the Education Department of Hunan
Province (21B0313), the Natural Science Foundation of SZTU Top Talent (GDRC202132), the SZTU-
Enterprise Cooperation Project (20221061030002, 20221064010094), the Inner Mongolia Key Project
(2021ZD0044), and the SZTU Project (20224027010006).

REFERENCES
[1] 2017. Dataset:Rico. Retrieved Sept. 28, 2023 from https://www.interactionmining.org/
[2] 2019. Dataset:AGM. Retrieved Sept. 28, 2023 from https://darkpatterns.cs.princeton.edu/data/
[3] 2019. Tool:AGM. Retrieved Sept. 28, 2023 from https://github.com/aruneshmathur/dark-patterns
[4] 2020. Dataset:LLE. Retrieved Sept. 28, 2023 from https://zenodo.org/deposit/3601501
[5] 2023. Dataset:ContextDP. Retrieved Sept. 28, 2023 from https://github.com/SageSELab/AidUI/
[6] 2023. Tool:AidUI. Retrieved Sept. 28, 2023 from https://github.com/SageSELab/AidUI/
[7] 2023. websit:darkpatternsok. Retrieved Sept. 28, 2023 from https://sites.google.com/view/darkpattern
[8] Jacob Aagaard, Miria Emma Clausen Knudsen, Per Bækgaard, and Kevin Doherty. 2022. A Game of Dark Patterns:
Designing Healthy, Highly-Engaging Mobile Games. , 438:1–438:8 pages. https://doi.org/10.1145/3491101.3519837
[9] Gerald Albaum. 1997. The Likert scale revisited. Market Research Society. Journal. 39, 2 (1997), 1–21. https://doi.org/10.
1177/147078539703900202
[10] Competition Market Authorities. 2022. Evidence Review of Online Choice Architecture and Consumer and Competition
Harm.
[11] European Data Protection Board. 2022. Guidelines 3/2022 on Dark patterns in social media platform interfaces: How to
recognise and avoid them. https://edpb.europa.eu/system/files/2022-03/edpb_03-2022_guidelines_on_dark_patterns_
in_social_media_platform_interfaces_en.pdf Technical Report Version 1.0.
[12] Niall Bolger, Gertraud Stadler, and Jean-Philippe Laurenceau. 2012. Power analysis for intensive longitudinal studies.
(2012).
[13] Sebastian Boring, Saul Greenberg, Jo Vermeulen, Jakub Dostal, and Nicolai Marquardt. 2014. The dark patterns of
proxemic sensing. Computer 47, 8 (2014), 56–60. https://doi.org/10.1109/MC.2014.223
[14] Christoph Bösch, Benjamin Erb, Frank Kargl, Henning Kopp, and Stefan Pfattheicher. 2016. Tales from the Dark Side:
Privacy Dark Strategies and Privacy Dark Patterns. Proceedings on Privacy Enhancing Technologies 2016, 4 (2016),
237–254. https://doi.org/10.1515/popets-2016-0038
[15] Christoph Bösch, Benjamin Erb, Frank Kargl, Henning Kopp, and Stefan Pfattheicher. 2016. Tales from the dark side:
privacy dark strategies and privacy dark patterns. Proc. Priv. Enhancing Technol. 2016, 4 (2016), 237–254.
[16] Harry Brignull. 2010. Twitter: Deceptive Design@darkpatterns. Twitter (2010).
[17] Harry Brignull. 2010. Types of deceptive design. (2010). https://www.deceptive.design/types
[18] Harry Brignull. 2011. Dark Patterns: Deception vs. Honesty in UI Design. Interaction Design, Usability 338 (2011), 2–4.
[19] Harry Brignull, Marc Miquel, Jeremy Rosenberg, and James Offer. 2015. Dark patterns-user interfaces designed to
trick people. In Proceedings of the Poster Presentation, Australian Psychological Society Congress, Sydney, NSW, Australia.
21–23.
[20] Jieshan Chen, Jiamou Sun, Sidong Feng, Zhenchang Xing, Qinghua Lu, Xiwei Xu, and Chunyang Chen. 2023. Unveiling
the Tricks: Automated Detection of Dark Patterns in Mobile Applications. In Proceedings of the 36th Annual ACM
Symposium on User Interface Software and Technology (UIST). 1–20.
[21] Federal Trade Commission. 2022. Bringing Dark Patterns to Light Staff Report. https://www.ftc.gov/system/files/ftc_
gov/pdf/P214800%20Dark%20Patterns%20Report%209.14.2022%20-%20FINAL.pdf Technical Report Version.

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
Shadows in the Interface: A Comprehensive Study on Dark Patterns 10:21

[22] Andrea Curley, Dympna O’Sullivan, Damian Gordon, Brendan Tierney, and Ioannis Stavrakakis. 2021. The design of a
framework for the detection of web-based dark patterns. (2021).
[23] Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto Bacchelli. 2020. UI Dark Patterns and
Where to Find Them: A Study on Mobile Applications and User Perception. , 14 pages. https://doi.org/10.1145/3313831.
3376600
[24] Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto Bacchelli. 2020. UI dark patterns and
where to find them: a study on mobile applications and user perception. In Proceedings of the 2020 CHI conference on
human factors in computing systems. 1–14.
[25] Gregory M. Dickinson. 2023. Privately Policing Dark Patterns. https://doi.org/10.2139/ssrn.4389572
[26] Madison Fansher, Shruthi Sai Chivukula, and Colin M Gray. 2018. # Darkpatterns: UX practitioner conversations
about ethical design. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. 1–6.
https://doi.org/10.1145/3170427.3188553
[27] Liza Gak, Seyi Olojo, and Niloufar Salehi. 2022. The distressing ads that persist: Uncovering the harms of targeted
weight-loss ads among users with histories of disordered eating. Proceedings of the ACM on Human-Computer Interaction
6, CSCW2 (2022), 1–23. https://doi.org/10.1145/3555102
[28] Leo A Goodman. 1961. Snowball sampling. The annals of mathematical statistics (1961), 148–170.
[29] Colin M Gray, Lorena Sánchez Chamorro, Ike Obi, and Ja-Nae Duane. 2023. Mapping the Landscape of Dark Patterns
Scholarship: A Systematic Literature Review. , 188–193 pages. https://doi.org/10.1145/3563703.3596635
[30] Colin M Gray, Shruthi Sai Chivukula, and Ahreum Lee. 2020. What Kind of Work Do" Asshole Designers" Create?
Describing Properties of Ethical Concern on Reddit. In Proceedings of the 2020 acm designing interactive systems
conference. 61–73.
[31] Colin M Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L Toombs. 2018. The dark (patterns) side of UX
design. In Proceedings of the 2018 CHI conference on human factors in computing systems. https://doi.org/10.1145/
3173574.3174108
[32] Colin M Gray, Cristiana Santos, and Nataliia Bielova. 2023. Towards a Preliminary Ontology of Dark Patterns
Knowledge. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–9. https:
//doi.org/10.1145/3544549.3585676
[33] Colin M Gray, Cristiana Santos, Nataliia Bielova, Michael Toth, and Damian Clifford. 2021. Dark patterns and the legal
requirements of consent banners: An interaction criticism perspective. In Proceedings of the 2021 CHI Conference on
Human Factors in Computing Systems. 1–18. https://doi.org/10.1145/3411764.3445779
[34] Colin M. Gray, Cristiana Teixeira Santos, Nicole Tong, Thomas Mildner, Arianna Rossi, Johanna T. Gunawan, and
Caroline Sinders. 2023. Dark Patterns and the Emerging Threats of Deceptive Design Practices. (2023), 510:1–510:4.
https://doi.org/10.1145/3544549.3583173
[35] Johanna Gunawan, Amogh Pradeep, David Choffnes, Woodrow Hartzog, and Christo Wilson. 2021. A Comparative
Study of Dark Patterns Across Web and Mobile Modalities. Proc. ACM Hum.-Comput. Interact. 5, Article 377 (oct 2021),
29 pages.
[36] Hana Habib, Sarah Pearman, Ellie Young, Ishika Saxena, Robert Zhang, and Lorrie FaIth Cranor. 2022. Identifying user
needs for advertising controls on Facebook. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022),
1–42. https://doi.org/10.1145/3512906
[37] Shun Hidaka, Sota Kobuki, Mizuki Watanabe, and Katie Seaborn. 2023. Linguistic Dead-Ends and Alphabet Soup:
Finding Dark Patterns in Japanese Apps. In Proceedings of the 2023 CHI Conference on Human Factors in Computing
Systems. 1–13. https://doi.org/10.1145/3544548.3580942
[38] Daniel Kirkman, Kami Vaniea, and Daniel W. Woods. 2023. DarkDialogs: Automated detection of 10 dark patterns on
cookie dialogs. , 847–867 pages. https://doi.org/10.1109/EuroSP57164.2023.00055
[39] Barbara Kitchenham, O Pearl Brereton, David Budgen, Mark Turner, John Bailey, and Stephen Linkman. 2009. Systematic
literature reviews in software engineering–a systematic literature review. Information and software technology 51, 1
(2009), 7–15. https://doi.org/10.1016/j.infsof.2008.09.009
[40] Agnieszka Kitkowska. 2023. The Hows and Whys of Dark Patterns: Categorizations and Privacy. Human Factors in
Privacy Research (2023), 173–198.
[41] Tim Kollmer and Andreas Eckhardt. 2023. Dark Patterns: Conceptualization and Future Research Directions. Business
and Information Systems Engineering 65, 2 (2023), 201–208. https://doi.org/10.1007/s12599-022-00783-7
[42] Tim Kollmer and Andreas Eckhardt. 2023. Insights on Attention Capture Dark Patterns in Social Networking Sites.
BT - 29th Americas Conference on Information Systems, AMCIS 2023, Panama City, Panama, August 10-12, 2023.
https://aisel.aisnet.org/amcis2023/social_comput/social_comput/14
[43] Monica Kowalczyk, Johanna T Gunawan, David Choffnes, Daniel J Dubois, Woodrow Hartzog, and Christo Wilson.
2023. Understanding Dark Patterns in Home IoT Devices. In Proceedings of the 2023 CHI Conference on Human Factors
in Computing Systems. 1–27. https://doi.org/10.1145/3544548.3581432

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.
10:22 Nie et al.

[44] Cherie Lacey and Catherine Caudwell. 2019. Cuteness as a ‘dark pattern’in home robots. In 2019 14th ACM/IEEE
International Conference on Human-Robot Interaction (HRI). IEEE, 374–381. https://doi.org/10.1109/HRI.2019.8673274
[45] Piergiorgio Ladisa, Henrik Plate, Matias Martinez, and Olivier Barais. 2023. Sok: Taxonomy of attacks on open-
source software supply chains. In 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 1509–1526. https:
//doi.org/10.1109/SP46215.2023.10179304
[46] Jamie Luguri and Lior Jacob Strahilevitz. 2021. Shining a light on dark patterns. Journal of Legal Analysis 13, 1 (2021),
43–109. https://doi.org/10.1093/jla/laaa006
[47] Francisco Lupiáñez-Villanueva, Alba Boluda, Francesco Bogliacino, Giovanni Liva, Lucie Lechardoy, and Teresa Ro-
dríguez de las Heras Ballell. 2022. Behavioural study on unfair commercial practices in the digital environment: dark
patterns and manipulative personalisation. Publications Office of the European Union.
[48] SM Hasan Mansur, Sabiha Salma, Damilola Awofisayo, and Kevin Moran. 2023. Aidui: Toward automated recognition
of dark patterns in user interfaces. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE).
IEEE, 1958–1970.
[49] S M Hasan Mansur and Kevin Moran. 2023. Toward Automated Tools to Support Ethical GUI Design. In 2023 IEEE/ACM
45th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 294–298.
https://doi.org/10.1109/ICSE-Companion58688.2023.00079
[50] Arunesh Mathur, Gunes Acar, Michael J Friedman, Eli Lucherini, Jonathan Mayer, Marshini Chetty, and Arvind
Narayanan. 2019. Dark patterns at scale: Findings from a crawl of 11K shopping websites. Proceedings of the ACM on
Human-Computer Interaction 3, CSCW (2019), 1–32. https://doi.org/10.1145/3359183
[51] Thomas Mildner, Gian Luca Savino, Philip R. Doyle, Benjamin R. Cowan, and Rainer Malaka. 2023. About Engaging
and Governing Strategies: A Thematic Analysis of Dark Patterns in Social Networking Services. , 192:1–192:15 pages.
https://doi.org/10.1145/3544548.3580695 arXiv:2303.00476
[52] Dmitry Nazarov and Yerkebulan Baimukhambetov. 2022. Clustering of Dark Patterns in the User Interfaces of Websites
and Online Trading Portals (E-Commerce). Mathematics 10, 18 (2022), 3219.
[53] Trung Tin Nguyen, Michael Backes, and Ben Stock. 2022. Freely given consent? Studying consent notice of third-party
tracking and its violations of GDPR in android apps. In Proceedings of the 2022 ACM SIGSAC Conference on Computer
and Communications Security. 2369–2383. https://doi.org/10.1145/3548606.3560564
[54] Liming Nie, Kabir Sulaiman Said, Lingfei Ma, Yaowen Zheng, and Yangyang Zhao. 2023. A systematic mapping study
for graphical user interface testing on mobile apps. IET Software (2023), 249–267. https://doi.org/10.1049/sfw2.12123
[55] OECD. 2022. Dark commercial patterns.
[56] Daniel M Oppenheimer. 2006. Consequences of erudite vernacular utilized irrespective of necessity: Problems with
using long words needlessly. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in
Memory and Cognition 20, 2 (2006), 139–156. https://doi.org/10.1002/acp.1178
[57] Lorenzo Porcelli, Massimo Ficco, and Francesco Palmieri. 2023. Mitigating User Exposure to Dark Patterns in Cookie
Banners Through Automated Consent. , 145–159 pages.
[58] S Hrushikesava Raju, Saiyed Faiayaz Waris, S Adinarayna, Vijaya Chandra Jadala, and G Subba Rao. 2022. Smart dark
pattern detection: Making aware of misleading patterns through the intended app. In Sentimental Analysis and Deep
Learning: Proceedings of ICSADL 2021. Springer, 933–947. https://doi.org/10.1007/978-981-16-5157-1_72
[59] Brennan Schaffner, Neha A Lingareddy, and Marshini Chetty. 2022. Understanding account deletion and relevant dark
patterns on social media. Proceedings of the ACM on Human-Computer Interaction 6, CSCW2 (2022), 1–43.
[60] Than Htut Soe, Cristiana Teixeira Santos, and Marija Slavkovik. 2022. Automated detection of dark patterns in
cookie banners: how to do it poorly and why it is hard to do it any other way. arXiv preprint arXiv:2204.11836 (2022).
https://doi.org/10.48550/arXiv.2204.11836 arXiv:2204.11836
[61] José P Zagal, Staffan Björk, and Chris Lewis. 2013. Dark patterns in the design of games. In Foundations of Digital
Games 2013.

Received 2023-09-28; accepted 2024-01-23

Proc. ACM Softw. Eng., Vol. 1, No. FSE, Article 10. Publication date: July 2024.