From Immersion To Manipulation
From Immersion To Manipulation
Abstract
The continuing advances in Mixed Reality (MR) technology
have finally brought MR experiences to the consumers. How-
ever, the growing number of experiences merging the physical
and virtual worlds has also elicited a rise in the use of dark
patterns and manipulative design tactics intended to deceive
users into actions they might not otherwise take. This prelimi-
nary research investigates the mechanisms and prevalence of Figure 1: We analyzed the prevalence of dark patterns across
dark patterns in MR environments, providing a first glimpse 80 applications on various Mixed Reality platforms. Patterns
into manipulative practices. Analyzing 80 MR applications include button camouflage (left), hidden costs (center), or
across various MR platforms, we identified five primary dark false urgency (right).
patterns: Hidden Costs, Misinformation, Button Camouflage,
Forced Continuity, and Disguised Ads. Our ongoing analy-
sis highlights the impact of these patterns on user trust and Prior research has primarily focused on identifying and cat-
decision-making. egorizing dark patterns in conventional interfaces, highlight-
ing these patterns’ ethical and practical concerns. Recently,
research has started exploring the possibilities and implica-
1 Introduction and Background tions of tricking users by facilitating MR technology. To that
end, Eghtebas et al. [2] utilized speculative design to explore
Mixed Reality (MR) technology rapidly evolves, providing
deceptive design scenarios that may arise with pervasive MR.
immersive experiences by blending the physical and virtual
According to their research, potential future harms and risks
worlds. Finally, consumer devices are available, and users
could be caused by the situatedness of information, altering
can experience MR applications in various domains, such as
human perception, and the inherent ubiquity of sensing.
gaming, entertainment, education, or shopping. At the same
time, this trend causes new threats to individuals’ privacy and Similarly, Krauss et al. [5] investigated dark patterns in
security. One critical concern is the increasing presence of MR through expert co-design workshops. Based on several
dark patterns – design strategies that manipulate users into co-designed scenarios, they specified novel and confirmed
making decisions against their best interests. As MR becomes existing dark patterns. Given the unique technological proper-
more integrated, understanding and mitigating these deceptive ties of MR, they concluded that specific dark patterns could
practices is crucial to ensure user independence and trust in be amplified by exploiting MR properties such as spatiality
these advanced technologies [3]. and perception, leading to less effective traditional mitigation
strategies.
While these scenarios are speculative, they may not rep-
resent actual future uses of MR. Going beyond speculation
and co-design, Wang et al. [6] developed the first mockups
to study the effects of dark patterns in MR. They showcased
Copyright is held by the author/owner. Permission to make digital or hard that dark patterns can be effectively facilitated in immersive
copies of all or part of this work for personal or classroom use is granted scenarios. Despite understanding the unique challenges and
without fee.
USENIX Symposium on Usable Privacy and Security (SOUPS) 2024. malicious potential posed by the immersive qualities of MR
August 11–13, 2024, Philadelphia, PA, United States. technology that can amplify the effect of dark patterns [4],
there remains a gap in understanding the prevalence of dark and Disguised Ads, while Play Store applications often in-
patterns in current MR applications. cluded Hidden Costs and Button Camouflage. We further
This research aims to fill the gap by preliminarily assessing found manipulative practices in MR applications, exploiting
how prevalent dark patterns are in Mixed Reality applications user psychology through strategic placement of buttons, am-
across major platforms. Furthermore, through a systematic biguous messaging, and manipulation of default choices.
onboarding and walkthrough of available applications, we Our preliminary results already highlight the pervasive use
elaborate on which specific manipulative techniques are most of dark patterns in free MR applications across major plat-
commonly employed to influence user behavior. forms. Disguised Ads, being the most prevalent, emphasize
prioritizing financial gain over user experience. Yet, this find-
2 Methodology ing might be influenced by the application selection process
and calls for further analysis. The identified patterns pose
Our preliminary analysis involved a systematic, user-centric ethical concerns and call for regulatory measures to safeguard
approach to identify and analyze dark patterns in MR applica- users. Countermeasures or advanced mitigation strategies are
tions. Initially, MR applications were installed and explored particularly of greater interest given the current research and
from a regular user’s perspective, including potential registra- speculations on the future development of dark patterns to-
tion, onboarding, and thorough exploration of the key features ward ubiquitous MR.
or gameplay. During exploration, we logged occurrences of While our dataset is small (in contrast to the number of
dark patterns, paying particular attention to user interface lay- available applications) and has certain limitations, our pre-
out, screen transitions, and signs of persuasion or deception. liminary analysis shows already today that MR applications
Criteria for identifying dark patterns include deceptive user exploit cognitive biases and psychological weaknesses to de-
experience designs specified by Brignull et al. [1] and recent ceive or manipulate users’ actions intentionally.
speculative designs [2, 5].
Future empirical studies and in-depth analyses are nec-
essary to understand the development of dark patterns on
Application Selection: Our investigation focuses on four different platforms and to monitor past and future changes.
major platforms providing MR applications for handheld We argue for developing a standardized method to identify,
or head-worn MR devices: the Play Store (Google), App analyze, and categorize manipulative practices consistently.
Store (Apple), Horizon Store (Meta), and Microsoft Store Establishing a framework will facilitate comprehensive cross-
(Microsoft). For each platform, we selected MR applications platform comparisons and contribute to developing best prac-
that met three criteria: (1) availability, (2) free of charge for tices for ethical design to build trust in MR technology and
download, and (3) trending in the respective VR/AR/MR cat- ultimately enable the community to create adapted mitigation
egory. We installed and sampled the applications on a Pixel 8 strategies over time.
(Google), iPhone 13 (Apple), Quest 3 (Meta), and HoloLens 2
(Microsoft). We selected 20 MR applications from each plat-
form, resulting in 80 applications. We selected and analyzed
the applications between January and March 2024.
4 Conclusion and Future Work
3 Results and Discussion
Our preliminary study reveals the extensive use of dark pat-
Our preliminary analysis of the 80 selected MR applications terns in MR applications across major platforms. We identi-
revealed five leading dark patterns: Hidden Costs (unexpected fied five primary dark patterns: Hidden Costs, Misinformation,
charges revealed during application use), Misinformation (pre- Button Camouflage, Forced Continuity, and Disguised Ads.
sentation of false or misleading information), Button Camou- The prevalence of these manipulative designs, especially Dis-
flage (tricking users into clicking more prominent buttons), guised Ads, underscores ethical concerns, highlights the need
Forced Continuity (lack of straightforward and easy options to for regulatory measures, and calls for further research. Ad-
cancel subscription), and Disguised Ads (advertisements de- dressing these challenges is crucial to ensuring user trust and
signed like regular content or interface elements). Disguised ethical design practices in MR environments.
Ads emerged as the most prevalent pattern, mainly aimed at We argue for further analysis to deepen the understanding
financial gain through persistent advertising. Further, we iden- of current and future dark pattern dynamics in MR environ-
tified that dark patterns often co-occur, with Hidden Costs ments. With this work, we plan to discuss further the necessity
frequently leading to Misinformation. of anticipating the potential risks to users’ privacy and secu-
Interestingly, our analysis found variations in the preva- rity, gain valuable feedback from the research community
lence of dark patterns across the different platforms. For ex- during the conference, and motivate new approaches to miti-
ample, the Horizon Store heavily featured Forced Continuity gate dark patterns in MR environments.
References [4] Hilda Hadan, Lydia Choong, Leah Zhang-Kennedy, and
Lennart E. Nacke. Deceived by immersion: A systematic
[1] Harry Brignull, Mark Leiser, Cristiana Santos, and Kosha analysis of deceptive design in extended reality. ACM
Doshi. Deceptive Patterns - Types of Deceptive Pattern. Comput. Surv., 56(10), may 2024.
2024, https://www.deceptive.design/types.
[2] Chloe Eghtebas, Gudrun Klinker, Susanne Boll, and Mar- [5] Veronika Krauss, Pejman Saeghe, Alexander Boden, Mo-
ion Koelle. Co-speculating on dark scenarios and un- hamed Khamis, Mark McGill, Jan Gugenheimer, and
intended consequences of a ubiquitous(ly) augmented Michael Nebeling. What makes xr dark? examining
reality. In Proceedings of the 2023 ACM Designing In- emerging dark patterns in augmented and virtual real-
teractive Systems Conference, DIS ’23, page 2392–2407, ity through expert co-design. ACM Trans. Comput.-Hum.
New York, NY, USA, 2023. Association for Computing Interact., apr 2024. Just Accepted.
Machinery.
[6] Carlos Bermejo Fernandez Xian Wang, Lik-Hang Lee
[3] Colin M. Gray, Yubo Kou, Bryan Battles, Joseph Hog-
and Pan Hui. The dark side of augmented reality: Explor-
gatt, and Austin L. Toombs. The dark (patterns) side
ing manipulative designs in ar. International Journal of
of ux design. In Proceedings of the 2018 CHI Confer-
Human–Computer Interaction, 0(0):1–16, 2023.
ence on Human Factors in Computing Systems, CHI ’18,
page 1–14, New York, NY, USA, 2018. Association for
Computing Machinery.