By Sacha Robehmed & Nour Chaabene
This article will explore the potential of using existing data where users might share their lived experiences of deceptive design practices – namely app reviews – to identify deceptive design practices in the Middle East region.
2022 was a significant year for policy-making countering deceptive design, with both the Federal Trade Commission in the US highlighting the rise in deceptive design, and the European Union publishing legislation tackling deceptive design, the Digital Services Act.
However the situation is different outside Europe and North America. In South West Asia and North Africa (SWANA) - a region often referred to as the “Middle East,” anecdotally digital rights activists have shared experiences of deceptive design practices. The Web Foundation has noted in the Asia-Pacific region that: “the term [deceptive design] may be unfamiliar, but the experience is widely shared.” Many internet users, including those using non-English internet, have lived experience of being fooled, tricked, or deceived by deceptive design practices. Given the global dominance of US-based technology companies, many of the same websites where consumers in the US and Europe encounter deceptive design practices are also used in the SWANA region. Meanwhile websites and platforms developed within the region and designed for SWANA users may also feature deceptive design practices.
While we know anecdotally that deceptive design exists in SWANA countries, there is a lack of data, a lack of awareness from consumers, technology companies, and regulators, and a lack of policy to counter deceptive design. Consumer and data protection regulators in the SWANA - if they exist at all - are not yet tackling deceptive design practices. Technology policy advocates and activists focus more on issues like countering surveillance, freedom of expression, internet shutdowns, online harassment, privacy, and effective data protection across the region. With these priorities, and the very real threats to online users in some countries in the region, it’s unsurprising that deceptive design is low on the priority list. In other spaces where we might hope to find discussion of deceptive design, such as design education in the region, we find some discussion of the social impact of design and design ethics, but overall there is a lack of teaching and awareness of deceptive design.
Underpinning a lack of both policy and awareness on deceptive design in SWANA is the lack of data. Often Current data collection mechanisms do not address the region. The Dark Patterns Tip Line hosted by Stanford Digital Civil Society Lab is a method for direct reporting, while Deceptive Design Hall of Shame encourages people to use Twitter to highlight and call out examples. Together, they share hundreds of examples of deceptive design practices, but almost all of these examples – valuable data in highlighting and countering deceptive design – are about practices used by technology companies based in the US and to a lesser extent, Europe. This is not surprising given that the reporting sites are in English and are run out of the US and the UK, where there’s been the most awareness raising and press attention on deceptive design. While this data is undoubtedly useful, they do not capture the picture for users in the SWANA region. Even if users in SWANA are using the same web platform as in the reported deceptive design examples, they may experience a different interface which is designed differently for users in the region, or they may experience deceptive design harms in different ways. For instance, a SWANA consumer might experience greater obstacles with online payments, or face more challenges and costs to cancel a subscription if they have to call a US-based tech company, compared with a consumer based in the US.
Before there can be policy change and increased awareness in the SWANA region, there needs to be a way to identify, gather and share data on deceptive design practices. Where are SWANA’s internet users experiencing being tricked, fooled, or deceived by deceptive design practices? What types of deceptive design are they facing? And are technology companies doing anything about deceptive design practices - are they implementing new ones , or remedying existing practices? These were questions we were curious about, as identifying deceptive design practices and collecting data on instances and examples experienced by consumers in the SWANA region is the first step to beginning to counter such practices.
Through a conversation with digital rights educator Raya Sharbain on the topic, the idea emerged of looking at app reviews. App reviews constitute a repository of user-generated, self-reporting data, where consumers share their experiences of using an application - and might also potentially use this as an outlet or complaints mechanism in instances where they are fooled, tricked or deceived, and on the receiving end of a deceptive design practice.
Methodology and Approach
We sought to conduct exploratory rapid research, to understand to what extent deceptive design practices in SWANA can be identified through app reviews.
We approached this by examining reviews for 3 digital services that are popular in the region: ride-hailing app Careem, food and grocery delivery service Talabat, music app Anghami. We selected these as they are “home-grown” services, designed and developed in the region, and for their high usage and popularity across SWANA (though it’s worth noting that usage varies widely in the region, and Careem and Talabat are not available in all countries. In Tunisia for instance, where Careem and Talabat are not available, consumers are most likely to use Jumia for food delivery, and Bolt or InDrive for transportation services.) Careem has more than 10 million Android downloads and 1.27 million Play Store reviews; Talabat has more than 10 million Android downloads and 323,000 Play Store reviews; Anghami has more than 50 million Android downloads and 1.36 million Play Store reviews.
Data was collected through scanning app reviews on the Google Play Store using an Android mobile phone between 9 and 11 January 2023. Early on, we realised that the reviews we were seeing differed, and suspected this was due to language settings and geographic location. Through experimentation we learned that changing the geographical location had no effect on the displayed reviews, however changing the phone’s settings to French or Arabic language limited the number of available filters to sort through reviews – while in Arabic or French, reviews could only be sorted using “negative” or “ positive” filters, while in English an additional five to eight keywords labelles as “review topics” appeared for filtering reviews. For consistency, we therefore decided to examine the reviews setting Jordan as the geographic location and English for the language.
We scanned a total of 180 app reviews. We looked at the 60 most recent reviews for each app, by examining 20 reviews across three different text filters (labelled “review topics” on the Google Play store) that we suspected might be most relevant. For Careem we looked at 20 reviews each under “cancellation fees,” “scam” and “bad experience”; for Talabat, 20 reviews each under “scam,” “subscription” and “terrible experience” and for Anghami, 20 reviews under each category of “subscription,” “bad experience” and “critical”.
We selected these review topics as we suspected they would be potentially more likely to include reviews related to deceptive design (for instance: “cancellation fees”; “scam”; and “subscription”) and for each app also included one topic that would potentially reveal negative reviews but that will cover more general issues (e.g.: “bad experience”). With limited time available, we took this approach to try to identify the maximum number of reviews related to deceptive design practices. This avoided going through all reviews (many of which are positive reviews and do not highlight deceptive design practices). Using the “review topics” also meant that we could filter out reviews which might be complaints about a specific feature or update rather than something related to a deceptive design practice (for instance, “dark mode” on the Anghami app, with reviewers commenting that this had been removed in the latest update, or “driver” or “technical issues” in the Talabat reviews).
After filtering the reviews we scanned for deceptive design practices and classified them based on Harry Brignull’s deceptive design taxonomy that classifies practices into 12 types (Trick questions, Sneak into basket, Roach motel, Privacy zuckering, Price comparison prevention, Misdirection, Hidden costs, Bait and switch, Confirmshaming, Disguised ads, Forced continuity, and Friend spam).
Of the 180 reviews that were scanned, almost half (84) mentioned a deceptive design practice. A further 6 could not be classified into a particular deceptive design type but showed evidence of a customer feeling tricked or deceived. We have therefore labelled these as “Other”.
Of the deceptive design types identified in the reviews, there were:
44 instances of Hidden Costs (“You get to the last step of the checkout process, only to discover some unexpected charges have appeared, e.g. delivery charges, tax, etc.”)
24 instances of “Bait and switch,” (You set out to do one thing, but a different, undesirable thing happens instead.”)
9 instances of Roach motel” (“you get into a situation very easily, but then you find it is hard to get out of it e.g. a premium subscription”)
4 instances of “Misdirection” (“The design purposefully focuses your attention on one thing in order to distract your attention from another.)
2 instances of “Sneak into basket” (“you attempt to purchase something, but somewhere in the purchasing journey the site sneaks an additional item into your basket”)
1 instance of “Forced Continuity” (“when your free trial with a service comes to an end and your credit card silently starts getting charged without any warning. In some cases this is made even worse by making it difficult to cancel the membership”)
Six of the 12 types of deceptive design practices were identified in the data set. The design practices which were not identified related to ads disguised as other content,preventing price comparisons, privacy and being tricked into publicly sharing more information, guilting users, spamming friends, and trick questions.
Also of interest are the different instances of each type of deceptive design found in different apps. Careem’s reviews, for instance, account for 77% of the total instances of the “Hidden costs” pattern.
Analysis: Benefits and Limitations
This was a relatively easy to use and low cost data collection method, as the reviews are publicly available and free access. There is also an enormous amount of data already available (at least for popular apps), with millions of reviews. The reviews include data like when the review was posted, and overall data is recent and timely, with new reviews being posted every day with a short time lag for reviews to be published.
Perhaps the biggest limitation with this method though is it is only applicable for apps, not websites, and therefore only gives a window to deceptive design practices of some technology companies but not all. Other limitations are due to the Play Store itself - for instance, one cannot know the geographic location of a review unless the reviewer mentions it or implies it in their review (for instance by sharing the currency used). Given that regulators are typically focused on their own country, it may potentially limits the usefulness of review data for regulators and policymakers. A feature of reporting mechanisms like the Tip Line is that screenshots can be added and are even encouraged, to provide visual evidence of the deceptive design practice, but it’s not possible to add images to reviews on the Google Play Store.
Furthermore there are variations between the desktop and mobile versions of the Play Store which make it less useful for identifying deceptive design practices - on the mobile version reviews can be filtered by star rating and whether a review is positive or critical; by review topics (which are keywords that appear in reviews); and sorted by “most relevant” or “most recent”. Users can also select to see reviews for the “latest version” or “this device model.” The desktop version offers none of these ways to sort reviews, and it is unclear how the reviews are automatically sorted. Even when using the mobile version, the topics of reviews are sorted by a Google algorithm using machine learning. Reviews might contain information about more than one type of deceptive design, but it’s unclear if reviews can be sorted under more than one topic. It’s also not possible to use more than one filter at once.
The reviews can also be manipulated or are likely not fully representative. It’s unclear if reviews can be deleted by the company, and as anyone with a Google account can write a Play Store review, positive reviews may potentially be posted by users who are affiliated to or contracted by the company. (This is perhaps more likely to be noticeable and impactful if there are fewer reviews.) Equally, as reviews are user-generated data, certain groups are less likely to write a review. This could be users who are illiterate, or users who feel ashamed at being tricked and would not want to admit it in a review – it’s likely that users who are most vulnerable to the harms of deceptive design practices may be less willing or able to write reviews.
Another significant challenge is the quality of data. Some reviews are poorly written with unclear language, which makes it challenging to include them in a dataset. As reviews are user-generated and self-reported data without any verification of the information, and the reviews we were examining were often expressing dissatisfaction, there is the possibility that a customer may have exaggerated their experience or probably overlooked something. For instance, in the below example of a Careem review the customer mentions that they are not given a warning of cancellation fees. However, in the author’s experience using the app, a warning is actually given when a customer cancels the ride. The pop-up message that appears discourages the user from cancelling the ride, and is an example of “Confirmshaming.” While the data from app reviews highlights “hidden costs,” (for Careem, 34 of the 60 reviewed reviewed were related to ‘hidden costs’), the reviews did not mention instances of “Confirmshaming.”
Figure 1: Example of 'hidden costs' on Careem.
Figure 2: Screenshot of pop-up that appears when customer goes to cancel a ride on Careem; an example of "Confirmshaming"
The high number of reviews related to “Hidden costs” can partly be attributed to the lack of awareness of deceptive design types. People are reporting being feeling tricked or out of pocket rather than having experienced a deceptive design. In comparison, there is a low number of deceptive design related reviews on the Anghami app compared to Careem. Most of the negative reviews for Anghami were related to the app not allowing them to choose songs in the free version. Since they are using the free version there is no risk of being "robbed" by the app, which lowers reviews related to deceptive design.
The process has also surfaced some challenges with using Brignull’s deceptive design taxonomy for sorting app reviews. We have interpreted customer complaints of cancellation fees and feeling scammed as examples of “hidden costs.” Yet the company could make the argument that their costs are listed transparently under the Cancellation Policy in the Help section and are not hidden (even though from a user perspective the cancellation fees are flagged only when cancelling, not at the stage of ordering a ride). There is some flexibility and openness to interpretation in the definitions of each type of deceptive design, and a common agreement or more conclusive, definitive explanations within the typology might help to clarify. We also found examples where users felt deceived or tricked by the design, but the current typology did not cover this (these we labelled as “Other”).
App review data would become more useful to researchers, regulators and tech companies if users were asked questions about deceptive design types when they made their reviews. For instance, imagine if users in the Google Play Store could see a list of deceptive design types with explanations within the flow as they’re making a review, and could check off which of the types of deceptive design they had faced, if any, before or after making a written review. This would help to record types of deceptive design more accurately and hopefully improve the quality of data. It would also raise awareness of deceptive design among consumers. There is also the potential of using this to hold technology companies accountable.
What potential is there for researchers, regulators, and tech companies to identify deceptive design practices using app reviews?
For researchers: App reviews offer relatively low cost and open access to vast amounts of data. While this preliminary study looked at only a few hundred, for some apps there are hundreds of thousands of reviews available, and data science capabilities would be needed to examine these at scale. However, the biggest challenge is the quality of the data. Data is user-generated and there is high potential for bias, as app reviews are often written by disappointed customers. Reviews make claims which would need to be investigated further and not taken at face value.
App review data is therefore one possible method that researchers can use, and is particularly useful in countries outside the US and Europe where there is a lack of data. However we would recommend finding ways to verify the data, and using app reviews as a data source alongside other research methodologies.
For policymakers and regulators: The utility of app reviews for policymakers and regulators to identify deceptive design practices is limited. While they could maybe be used to suggest companies and areas of exploration, it seems unlikely that they could be used as reliable evidence. However, activists could use review content as part of an advocacy strategy, if there is an instance of a repeated deceptive design practice.
For technology companies: Currently, app stores are seen more as a way to promote digital products and redirect unhappy customers to a customer service funnel. For instance, all the app reviews had responses from the company, directing people to their customer service channel. However it is unclear to what extent companies making digital products are connecting customer experience and design, and sharing information on negative experiences (expressed in app reviews) with their design and product teams. For instance, while there could be a disconnect between the product and customer service teams, the company might choose to keep a deceptive design practice or something that benefits their business – such as cancellation fees – even if customers are complaining about it in app reviews. However, if they do seek to identify and remedy deceptive design practices, app reviews could be a powerful tool to help them. In order for this to happen, there is a need for a strong business case made in favour of trusted design practices.
Larger technology companies managing app stores, like Apple and Google, could could consider potentially make changes to the review process which would help in the easier identification of types of deceptive design – which could support consumer rights and help them earn favour with regulators and activists.
Download the full article here:
ABOUT THE AUTHORS
Sacha Robehmed is a design researcher based in Amman, Jordan. Through her research, she works with refugee communities and civil society organizations to develop rights-based and inclusive digital services, policies, and strategies.
Nour Chaabene is a legal and political science researcher based in Tunis, Tunisia. She is a Project Officer at Expectation State supporting the design and implementation of projects related to inclusive economic growth, innovative technologies, and digital rights.
Robehmed, S and Chaabene, N. (2023). Exploring the Potential of App Reviews to Identify Deceptive Design Practices in Arabic-Speaking Countries. The Unpacking Deceptive Design Research Series. The Pranava Institute <https://www.design.pranavainstitute.com/post/exploring-the-potential-of-app-reviews-to -identify-deceptive-design-practices-in-arabic-speaking-countries>