Bits don’t lie

So runs the slogan of the data analytics firm Kaiasm. It’s an example of the rhetorical technique chiasmus, in which a new insight is revealed by reversing the grammatical structure of a sentence, meant as a provocation. Historically, businesses have focused on “making people want things” by cultivating the desire to create a market for their goods, from sugar to Wedgwood pottery to cigarettes to the Model T Ford. Marketing and branding were developed to mold human desire to fit the supply of available products, often by promoting unnecessary items as status symbols, offering a better lifestyle, or enabling self-actualization.

As far as Kaiasm is concerned, too many organizations still operate in this outmoded “making people want things” paradigm. But how are they to start “making things people want” instead? In recent years it’s become more common for companies to aspire to “customer-centricity” in their product development. Typically they use market research techniques such as surveys and focus groups to discover the desires of their target customers and then design products to satisfy them. They may even develop “personas” to represent particular customer archetypes and then form hypotheses about other desires they might have.

There are two problems with these methods. First, they’re expensive. But more importantly, they are prone to various forms of “response bias,” the tendency for people to say what they think researchers want to hear instead of the truth. One kind of response bias is “social desirability bias,” where people lie to make themselves look good. A landmark sociological study from 1950 showed that Denver residents lied about a wide range of topics when surveyed on a wide range of topics, including whether they had voted in the last election, had a library card, or had recently donated to charity. To quote the data scientist Seth Stephens-Davidowitz, “People lie to friends. They lie to bosses. They lie to kids. They lie to parents. They lie to doctors. They lie to husbands. They lie to wives. They lie to themselves. And they damn sure lie to surveys.”

But over the past 20 years, new kinds of data generated as a by-product of our lives increasingly being lived online have made it possible to understand human desires more accurately. Stephens-Davidowitz and Kaiasm agree that the best way to find out what consumers really want is to abandon conventional market research in favor of one specific digital source of insight: search data. Every day, literally billions of searches are performed on search engines such as Google, Bing, and Baidu, creating a vast pool of data on human needs and desires. This data is accessible in aggregated and anonymized form through low-cost software-as-a-service platforms like AnswerThePublic and free tools like Google Trends. Search data is much cheaper than conducting surveys and reveals that what people want is often richer and more surprising than we are accustomed to thinking.

Search data’s great strength as a source of insight comes from how unfiltered we are when we use search engines. Consider some of the most common searches beginning with “Why does my boyfriend want…” and “Why does my girlfriend want…” among UK internet users at the time of writing:

Why does my boyfriend want

↳ to use condoms all of a sudden?

↳ to keep me a secret?

↳ to be alone?

↳ to get me pregnant?

↳ to see me every day?

Why does my girlfriend want

↳ to go clubbing?

↳ to bite me?

↳ to cuddle so much?

↳ my hoodie?

↳ me to post about her?

Search data is free from social desirability bias. As Sophie Coley from AnswerThePublic notes, people often ask remarkably personal questions through search engines that they wouldn’t even ask their family, friends, or doctor. They reveal subliminal fears of betrayal and abandonment, to insecurity and claustrophobia, to perplexing desires. It’s as if search engines have replaced priests as the modern world’s confessors.

For businesses, paying attention to search data can be a powerful source of commercial opportunity. In a project for the BBC, Kaiasm analyzed popular science and nature questions that Google search results weren’t addressing well. At the time, the BBC’s default approach was to commission programs that were similar to previous hits — but then Kaiasm found a pocket of poorly-served searches for content about a seemingly obscure topic: astronomy. Their analysis led directly to the BBC commissioning the show Stargazing Live. Filmed by famous observatories and hosted by the physicist Brian Cox and the comedian Dara O’Brien, the show aired for 7 years on BBC Two, attracting an average audience of 3.4 million at its peak.

The same approach helped DIY retailer Screwfix grow its sales to a billion pounds by aligning its buying and merchandising with how customers were searching for products like nail guns, kitchen lighting, and radiators. In the home and garden market, Worldstores focused on the demand for unusual products expressed through consumer searches to build a portfolio of niche e-commerce sites such as MattressesWorld, ShedsWorld, and TrampolineWorld, which it eventually sold to Dunelm Group Plc.

In financial services, startup ManyPets analyzed searches for pugs to identify the shortcomings of pet insurance policies for specific breeds. At a time when pugs were the most sought-after puppy in the UK, costing up to £5,000, most pet insurance policies capped coverage for theft and loss at £1,000 and excluded many conditions affecting pugs, such as hip dysplasia, from coverage. By designing pet insurance products to meet the consumer needs revealed by search data, ManyPets has grown to protect more than 600,000 dogs and cats in the UK, US, and Sweden. In entertainment, retail, and financial services, combining search data insights with the “long tail” economics of online business has proven to be a “killer app.” (Disclosure: the author is a shareholder in ManyPets).

Meanwhile, in academia, public health researchers are using search data to track and forecast the spread of infectious diseases like Ebola and Zika, a discipline known as “infodemiology.” This often involves monitoring the geographic distribution of searches for symptoms, as early warning signals of new outbreaks. In fact, during the Covid-19 pandemic, a model using symptom search data built by computer scientist Bill Lampos and his team at University College London was able to predict new cases 17 days faster than conventional epidemiological techniques.

Search data also offers fascinating insights into sexual and physical desire, including how it varies across space and time. Stephens-Davidowitz found that the top search starting with “My husband wants…” in India is “My husband wants me to breastfeed him.” While searches by wives about husbands the world over reveal a multitude of fantasies relating to childhood (such as the desire to wear diapers), the frequency of that particular breastfeeding search is four times higher in India than anywhere else. In the United States in 2004, the most common searches relating to changing one’s bottom were about how to make it smaller.

But between 2010 and 2014, searches for how to make bottoms bigger increased three-fold, to the point that by 2014 bigger-bottom searches outstripped smaller-bottom searches in every single state. Meanwhile, a surprising number of straight women seem to have the opposite preference when it comes to their partner’s anatomy: nearly 50% of penis-size-related searches by women expressed a desire for smaller penises, according to Stephens-Davidowitz’s findings. With fetishes and desirable body types seemingly in flux, it’s no wonder that the sex toy retailer Lovehoney has also used search data to understand market demand for its product range with Kaiasm’s help (apparently, most UK customers’ tastes are “reassuringly vanilla”).

It seems that people even turn to search engines when looking for answers about the waning of desire. They ask Google why they don’t want to “socialize,” “shower,” “get out of bed,” or “do anything,” and why their boyfriends, girlfriends, dogs, cats, husbands, and wives don’t want to “cuddle,” “hang out” or “sleep with” them anymore. According to Google Trends, global searches expressing anhedonia (the diminished ability to find pleasure in experiences that are normally pleasurable) have steadily increased over the past three years, especially in North America and Australasia.

As generative AI technology develops, search engines may evolve from being confessors to therapists, capable of both channeling desires to the places where they can be fulfilled and of diagnosing the causes of lack of desire. In February 2023, Microsoft released Bing Chat, which combines the functionality of a search engine with the natural language processing capabilities of OpenAI’s GPT-3 large language model. Rather than simply providing a list of links, Bing Chat answers questions directly — including personal ones. Here’s its answer to the question, “Why don’t I want to do anything?”

There could be numerous reasons behind this state of mind. It’s best to do some introspection and think about what exactly may cause your apathy. If you need help, ask for it. Sometimes not wanting to do anything can be a sign of fatigue and tiredness, but if this feeling persists for longer than a few weeks or months, then it can be a sign or symptom of depression. Feeling unmotivated and like you don’t want to do anything can be difficult, and there can be a range of reasons for feeling this way. Some of these feelings can be symptoms of depression.

(It signs off with a chipper, “I hope that helps!”)

Search data already has a lot to offer organizations that want to start “making things people want.” By analyzing the desires people articulate through search, companies can develop products and services that meet genuine needs — rather than just trying to create a market for what they happen to have to sell. Similarly, search data already provides fascinating insights into what people want (and don’t want) from sex and relationships. And with AI poised to supercharge the capabilities of search engines, it seems likely that in the future, search data will open up even more vistas into the deep recesses of human desire.