[EVENT REPORT] Thinking about dark patterns, services designed to deceive people --- Special lecture on "The technologies of dark patterns and the ethical issues they induce" (Hosted on June 17, 2021)

As part of "Platforms and '2040 Problem' Project", one of the pillars of KGRI's "Research Project Keio 2040", we hosted our first study session on June 17th, 2021. We kicked off this session with a special lecture titled "The technologies of Dark Patterns and the ethical issues they induce."

"Dark patterns" are the designs of websites and apps that subtly trick consumers into doing things they did not mean to do. It is a term coined with the rise of online services and its techniques are rapidly advancing. These designs hinder people from freely making decisions and direct users to make unintended purchases. In our session, we discussed how dark patterns are being practiced and the ethical issues they induce.

We invited Dr. Atsushi Hasegawa from Musashino Art University as a guest speaker to share with us his expertise on this essential theme of "platform and people." This event report is a digest of Dr. Hasegawa's lecture and the group discussion held among Dr. Hasegawa and KGRI members.

GUEST SPEAKER: Atsushi Hasegawa
Professor at Musashino Art University and President of Concent, Inc. Born in 1973 at Yamagata. He earned his Ph. D. from the Department of General Systems Studies at The University of Tokyo Graduate School of Arts and Sciences. His focus of studies is the "design of understanding" for the social utilization of design, exploration of design approach possibilities, and construction of design-based social systems. He is the author of Information Architecture 100: Information Architecture Design for User Experience Design. He has also supervised the translation of publications such as THIS IS SERVICE DESIGN THINKING.

LECTURE: What are dark patterns and its problems?

Tatsuhiko Yamamoto (Professor at Keio University Law School): I would first like to thank everyone for participating in our first study session of "Platforms and '2040 Problem' Project", one of the pillars of KGRI's "Research Project Keio 2040." In today's special lecture of "The technologies of Dark Patterns and the ethical issues they induce", Dr. Atsushi Hasegawa, the leading expert of Dark Patterns in Japan, will share with us on what dark patterns are. Then we would like to discuss the impact that dark patterns may have on the development of platforms.

Atsushi Hasegawa:：Hello, everyone. I am Atsushi Hasegawa and I manage a design firm as an information architect while teaching at the Graduate School of Art and Design at Musashino Art University. I am one of the representatives of the Japan Chapter of Service Design Network (SDN), a non-profit institution for expertise in service design and a driver of global growth, development, and innovation within the practice, which I have founded with Dr. Masanao Takeyama (Faculty of Economics, Keio University) and Mr. Hironori Iwasa (Visiting professor of Musashino Art University Graduate School/Executive Officer of Reazon Holdings Inc.). I also serve the Human Centered Design Organization (HCD-Net), a non-profit organization in Japan to disseminate approaches for user-friendly designs incorporating factors such as user experience (UX), as vice director and have recently founded a working group that studies code of ethics.
This is the agenda for today's lecture on dark patterns.

(1) What are dark patterns?

"Dark pattern" is a terminology associated with online behavior. According to Wikipedia, it is defined as "a user interface that has been carefully crafted to trick users into doing things" such as "sharing more information than they intended to" and "using shame to drive users to act." This term was coined by UX consultant, Harry Brignull, in his website "Dark Patterns" which was launched in aim to spread awareness of these "tricks used in websites and apps that make you do things that you didn't mean to, like buying or signing up for something."

Dark patterns have gained wide recognition over the last few years and there are now more researchers and journalists studying this troublesome concept. We are already beginning to see laws that prohibit the use of dark patterns being enacted in USA. For example, the California Consumer Privacy Act (CCPA) banned businesses to implement dark patterns that obscure, subvert, or impair users to opt-out of sales or sharing of personal information when requesting for its services.

(2) Examples of dark patterns

"Dark Pattern at Scale: Finding from a Crawl of 11K Shopping Websites" by Arunesh Mathur, et al. published in Association for Computing Machinery (ACM) in 2019 reported that dark patterns were found in about 10% of the 10,000 EC sites around the world and that these patterns could be categorized as one of the following seven types.

A. Sneaking
This is a type of approach where the site sneaks an additional item into your basket. There are several known ways of sneaking, such as a greeting card added to your basket along with a gift you selected or EC sites like Amazon that lead the users to purchase an item as a monthly subscription instead of a one-time purchase.

B. Urgency
The design of the user interface urges you to make a purchase by making you think you will lose out on the situation if you do not buy with a limited time offer. In most cases, the site will have a banner that reads "Offer will expire in XX days" but will continue displaying the same banner past the specified date.

C. Misdirection
Examples include visual interfaces designed to purposefully focus your attention on one action (such as to click "Yes" and distract you from clicking "No" on the purchase screen) and web content purposefully worded in complicated expressions to lead you to put a check in the checkbox without reading the entire content (such as double negative expressions used in the web pages to consent the sharing of personal information).

D. Social Proof
In this pattern, the site is designed to pressure the user so that he/she cannot make an informed decision. For example, displays that read "There are XX people currently browsing this site." In some cases, the site posts fake customer reviews and in even worse cases, the site posts photos generated by AI as customer profiles.

E. Scarcity
This is an approach to emphasize scarcity of an inventory or that a product is offered for a limited time to encourage buyers' motivations. In one EC site, all products were labeled with "Most likely to be sold out" emphasizing that the products were popular.

F. Obstruction
The design makes it hard for you to get out of a certain situation. The most typical example is a subscription, where the user must view page after page and follow complex procedures to cancel it.

G. Forced Action
This type of approach forces you to take a certain action such as sharing personal information to browse through an EC site.

Though these patterns are based on a journal published in 2019, Professor Hasegawa thinks it is still valid. It has been pointed out these patterns are more often found in mobile websites because the small screen sizes of smartphones and devices force sites to curtail expressions.

(3) Structure of dark patterns

Let us now talk about how dark patterns came about and how they developed. There are mainly 3 trends that evolved to dark patterns as introduced in the aforementioned journal.

Deceptive practices in retail
Even before the rise of EC, the retail industry has a long history of deceptive practices. For example, selling a 100-yen product for 98 yen to make the consumers think its "cheap." This is a representative case of the technique known as psychological pricing. Other examples of manipulative practices are false claims of store closings with their "countdown sales" and the bait-and-switch approach which lures consumers with an exaggeratively featured product and make them buy a different product that is comparatively expensive. Researchers point out that consumers have begrudgingly accepted these practices.

Nudges employed for public policies
One of the hottest topics of literature of behavioral economics is nudging. A nudge is an approach that lightly alters the environment to trigger a person's cognitive decision-making process to favor a desired outcome. This theory was first employed to advance public policies, but soon implemented for commercial purposes.

A famous example of nudging is the comparison of organ-donation consent rates between European countries. While the consent rate is merely 10-20% in the UK and Germany where people have to explicitly provide consent for organ donation, the consent rate is close to 100% in countries like France and Austria where consent is presumed (people have to explicitly deny donation). Indeed, higher consent rates benefit the society more, but this striking difference has raised an alert that a significant number of people have unknowingly provided consent to such a grave matter.

Growth hacking in design communities
Growth hacking is a reflection of Silicon Valley values which puts aside short-term revenue-generating activities and focuses on achieving market dominance by driving the growth of user popularity and industrial share in aim to improve business performance. The best-known example of growth hacking is the strategy implemented by Hotmail. They increased their user share by added a registration link at the end of every outgoing email.

Of these 3 trends, nudging is based on the theory indicated below. In conventional economics, consumers were rational entities that were cautious and made informed decisions with self-discipline. However, in reality, our judgments are influenced by how we feel and strongly depends on the situation or occasion we are in. Thus, psychology theories have been incorporated when formulating market strategies, leading to the development of behavioral science and behavioral economics.

Led by the Ministry of Environment, the Japan Nudge unit "BEST" was founded in 2017 among other similar units established around the world. The slide above is a translation of their meeting agenda disclosed on the web. Though it is suggested for benevolent aim, I feel that it is controversial whether it is appropriate to urge unintended behavioral change. It should also be noted that there are several patterns of design and expression that are employed for growth hacking. The most typical method is the A/B test which is an approach that allows the most effective option to be selected. Nowadays, growth hacking patterns are being automatically generated.

(4) How to handle dark patterns

As you can see, the incorporation of behavioral economics to long-practiced retail strategies evolved into dark patterns and its development was further accelerated with the new concept of growth hacking. Or in other words, the business framework of increasing the number of users, driving them to consume more and extracting more information from them has become an optimal system to foster dark patterns. Law professor, Lawrence Lessig, explores ways to use laws, practices, market, and codes (architectures/configuration) as instruments for social control. I think it can be said the same for dark patterns--- we need to think about the codes that create dark patterns or otherwise we will not be able to stop the optimization of such malicious market strategies. On the other hand, it is also necessary to enhance the literacy of users through appropriate training.
Dark patterns are bound to further spread in our society. It is urgent for us to review how these patterns are generated in our social system and reinforce the ethics of service designers in order to prevent any more development of dark patterns. Above all, we need to raise awareness of dark patterns to be able to spot and avoid them in our decision-making process.

DISCUSSION: How can dark patterns be regulated?

Yamamoto: Let me take this time to introduce IT critic, Mr. Kazuhiro Obara, joining us today all the way from Singapore! He is known for his publication The Platform: IT kigyo wa naze sekai wo kaerunoka(The Platform: Why do IT companies change the world?) as well as his contributions to the launching of NTT Docomo's "i-mode." He has also served Google Japan and Rakuten as a board member.

Kazuhiro Obara: I have had the experience of trying to find the right balance of business and ethics when launching the "i-mode" and developing Rakuten's ID payment system. For example, my project team at Rakuten found that only sending advertisement emails of what the customers really needed increased the rate of repeated purchases. So, I know from that experience that in the long run, you can build stronger relationship with customers if you don't employ dark patterns. We should be like the sun in the fable of "The North Wind and the Sun"--- I think it is critical that we design more and more benevolent architectures.

Hasegawa: In reality, we are pressured to improve our performances through optimization and honestly, we don't have the leisure to even think about the impact that our services may have in the long run. So, I think it is extremely important that each employee working at service providing companies is aware that they will be losing trust from their customers if they continue to employ dark patterns.

Keigo Komamura (Professor at Faculty of Law, Keio University): It gives me the chills to even bring up the topic of nudging. Over the last few years, we have been witnessing authoritarianism, which completely neglects liberalism, being successfully implemented. To name a few, policies by former US President Trump and national measures taken in response to COVID-19. To me, nudging seems like another form of authoritarianism in disguise of liberalism. It is often associated with Libertarian Paternalism, a new concept of national governance which combines libertarianism, holding individuals responsible for the decisions that they make, and paternalistic governance. Libertarian Paternalism seeks complete rationality in decisions made and responsibilities held by individuals who can only act with partial rationality.
I think the key point of our discussion is in determining what kind of nudges are acceptable and what aren't. Perhaps it is effective to categorize the situations in which nudges can be used. Particularly for economic and political activities as well as for emergency situations. But it will still be difficult to draw those lines, because after all, deceiving and being deceiving are fundamental aspects of economic activities. As long as economy is based on free negotiations, we can't stop the development of advanced strategies.

Hasegawa: Exactly. In general, strategies implemented for advancing public policies are referred to as nudges and manipulative strategies used for economical behavior are referred to as dark patterns. As you say, not all nudges are malicious. I personally think nudging people to use stairs instead of escalators to promote their well-being is acceptable. But it's hard to say the same for organ donation. I have yet to make up my mind on whether it is appropriate for the government to attempt to regulate their people's behavior.

Yamamoto: I agree that more prudence will be necessary in certain occasions than others. Even in political use, we cannot say for sure that nudging is malicious. Because nudging used in elections have a completely different intention compared to nudging used to reinforce the enactment of a law that has been democratically approved. I think nudging in the latter can be acceptable under certain conditions like ensured transparency because democratic legitimacy is already guaranteed to begin with.

Obara: It is not any news that platformers like GAFA hires experts in cognitive psychology, statistics, and behavioral economics. But it should also be noted that these platformers disclose the initiatives they take towards ethics. For example, in Japan, the UX team at note inc., well known for their media platform "note", shares dark patterns as examples of "uncool content" and the designs that they view to be aesthetically beautiful.

Sotaro Ishizuka (Associate professor at Faculty of Law, Nihon University): Professor Komamura just mentioned about deceiving and being deceived. This concept is based on classical studies of constitutional law which assumes strong individuals. I think that the assumption of these individuals is what's making platforms vulnerable. In other words, is the cause of the problem in the fact that these strategies approach the vulnerability of humans? Perhaps we should think about how optimization of platforms can benefit the society as a whole.

Hasegawa: Obviously, spreading false rumors cannot be called good behavior. But the definitions of other online behaviors are gray. How far can one go to increase buyer motivations? It's difficult to decide judge what's OK and what isn't, because it drills down to "What is individual intervention?" As long as we are living, we will always have intervention from others and these interventions will have some kind of impact on our decision-making process, in one form or another. Which interventions can be considered as good nudges? From when do these interventions become dark patterns? There are no answers to these fundamental questions. It's really just a matter of how much we are intervened.

Yamamoto: I think that this issue needs to be further discussed across various disciplines. In terms of law, it's about how the Personal Information Protection Act can be interpreted and how the Consumer Contract Act can be amended. But we also need to review dark patterns from other perspectives such as design ethics and psychology for designing a suitable social system. Needless to say, interdisciplinary collaboration is the key. What better place to discuss this than KGRI? I thank you all for your participation and look forward in continuing to study dark patterns together.

Symposium held in hybrid form (onsite and online) at G-Lab (East Research Building, Mita Campus) on June 17th, 2021.
*Affiliations and job titles are as of the date the event was held.