Mind control in its various forms—hypnosis, possession, brainwashing—has captured our imaginations for centuries in literature and in film. George Orwell’s 1984 famously conceptualizes means of ideological indoctrination that allow a government to brainwash its subjects, while Anthony Burgess’ A Clockwork Orange describes a fictional conditioning technique used to psychologically rehabilitate a serial criminal. Similarly, movies like those in the X-Men series construct storylines that affirm mental manipulation as the purest form of power. They depict mind control as the ultimate way to subjugate the opposition, by depriving them of the sentience that makes them human, and thus position it as something to be deeply feared.
As our understanding of human psychology has developed, mind control has rapidly expanded past the confines of science fiction and our imaginations into reality. Famously, the psychologist B. F. Skinner conducted an experiment in which he taught pigeons to tap a lever to receive food. He found that when he changed the machine to dispense food after a random number of taps, the frequency at which pigeons would tap the lever dramatically increased. His experiment established the idea that variable rewards could be incredibly addicting, and psychologists began to observe similar behaviors in people. They found that an element of unpredictability in the result of a given action made people want to do it again and again, which is partly why activities like gambling can be extremely addicting. Game makers began to use this knowledge to compel people to continue playing their games when they wouldn’t otherwise.
When it comes to methods used by product designers to create products that grip consumer attention beyond personal control, the idea of variable rewards is just the tip of the iceberg. In Hooked, Nir Eyal outlines what he calls the “Hook Model.” Eyal is one of the most prominent figures in the world of product design, having taught at Stanford’s school of design and founded a number of successful tech startups. His model for design, used by product designers in a wide array of industries, describes a four-step cycle to cultivate desired customer behavior. The four steps—trigger, action, variable reward, and investment—rely on a vast body of psychological knowledge to exploit recurring human thought patterns. To manipulate consumers, the process leverages detailed Pavlovian conditioning which describes the usage of rewards to positively reinforce desired behavior. The Hook Model also borrows heavily from the Fogg Behavior Model, the product of years of psychology research positing that the desired behavior is achieved in the simultaneous presence of a trigger and sufficient quantities of motivation and ability.
Product designers also utilize many of the heuristics or mental shortcuts we take in making decisions or forming opinions, like the “scarcity effect” and the “endowed progress effect.” The former explains the tendency for consumers to value the same product more when it appears less abundant and the latter describes how motivation increases when people believe they are closer to a goal. Since product designers want people to use their products, the knowledge of such irrational tendencies produces unbelievably effective products—indeed, nothing makes me click “purchase” faster than seeing “Only 1 left in stock!” on Amazon or finish filling out a profile like the halfway-full progress bar on LinkedIn (despite the fact that I’ve given them only my first and last name). Due to massive growth in the technology sector, product designers increasingly work at companies producing consumer tech, and with the power bestowed upon them by consumer psychology developments, they’ve ushered in a new era of addictive technology. Their influence on Generation Z, those born in the 1990s and 2000s, is undeniable. According to Common Sense Media, the average teen maintains a whopping 6 hours and 40 minutes of screen time a day.
The product designer is faced with a difficult dilemma. While they work in a variety of industries and come from a variety of backgrounds like computer science, art, or even economics, product designers all share the same employment objective. The companies that hire product designers want to generate revenue which then necessitates the increased usage of their products, so the nature of the job compels designers to create products that people will use as frequently as possible. It’s this part of the job description that is often at odds with ethical considerations; on one hand, product designers want users to use their products, but on the other, using their product might not be a good thing—and all the while, this bind is further complicated by the basic subjectivity of “goodness” itself. Take inKin, a social fitness app that takes advantage of the our need for peer affirmation in order to get users to exercise. Many may interpret the app’s friendly reminders that you’re running less than your friend Karen as a healthy thing, but others may dislike how it feeds into the need for external validation.
Samantha Dempsey and Ciara Taylor, two user experience designers who have worked on the interfaces of a number of healthcare companies, struggle with this very dilemma and are working to clarify the ethical considerations that designers should take into account. Some product designers have argued that those in the profession should be required to take an oath like the Hippocratic Oath, which physicians take to pledge they will provide each patient with the best medical care possible. In response, Dempsey and Taylor created the Designer’s Oath tool, which has evolved from a fill-in-the-blank agreement to role-playing software that enables teams to think more consciously about how the products they’re creating fit into their shared values. They told the College Hill Independent that engineering and product teams use the prompts given in the latest version of the tool to engage in discussion of team morals. In their words, the goal is to “help multidisciplinary teams define the ethical guidelines of their engagement.” But when probed further, they admit that it’s more of a stopgap measure, since it can be tricky to put these words into practice. Even if we have some standard set of ethical considerations were agreed upon across a given industry, what exactly do they look like in a product? It’s hard to say and harder yet to define a threshold where the guidelines are met by the design process. For apps like inKin, guidelines that may dictate that product designers create products in the best interest of consumers are impractical because, again, that “best interest” remains subjective.
Worse, product designers might not be inclined to care that their actions are ethical. Dempsey explained that not every designer will “believe they should share power with the people they’re designing for,” a chilling thought for consumers, since designers hold the potential to rot the brains of future generations. The instinct here might be regulation, but holding product designers and the companies they work for accountable is nearly impossible right now, given the legal limits. The same ambiguity in defining what’s ethical and what ethical considerations should look like in a product makes proving mal-intent out of the question—Taylor pointed out the difficulty in identifying culprits as an obstacle to regulation. She made the case that “the structure of how designers work with each other… it doesn’t seem as clear cut as if a surgeon cuts the wrong organ out of you.”
In the absence of regulation as an immediate viable solution, consumer conscientiousness may be our best bet. While our mass addiction to technology is driven by methods of product design, such methods are effective because they capitalize on our own lack of circumspection; they give validity to seemingly trite and overdone phrases like “be mindful” or “think before you act.” But when I asked how users might be able to resist these products, Taylor and Dempsey similarly talked about “living intentionally.” They pushed for “constant vigilance, or really being aware of how we want to live our lives and making conscious choices about that.” Their perspective offers insight into how product designers assume we can and should disrupt the orchestrated relationship between consumers and the products they use.
Right now, collective change may seem too far off or daunting to consider, regulation is logistically unfeasible, and products are increasingly persuasive in demanding our attention. Within this environment, it may be helpful to keep in mind that the effectiveness of addictive product design often hinges on our mental shortcuts—not on physiologically addictive stimulants, like nicotine. Smokers certainly can’t “just quit,” but when it comes to mind control, due diligence may, for the time being, be our most effective policy.
GRIFFIN KAO B’20 is trying to get his screen time down to 2 hours a day.