Japan, in a bid to boost its declining birth rate, recently pledged 2 billion yen toward artificial intelligence-powered matchmaking, which will be available across the country to help people find love, settle down, and make babies. This endorsement by a national government speaks to society’s assurance that the algorithms behind AI can accurately match people long term, borne out of the pervasiveness of dating apps and matrimony websites quickly becoming the go-to for people seeking love and stability.

But basing a public health and economic initiative on the efficacy of matchmaking algorithms, begs the question — do they even work?

Since matchmaking algorithms became popular a decade ago, they have only gotten more and more complex. If at first, they matched people based on age and location, now algorithms use everything from sense of humor, likes and dislikes, hobbies, and interests (as Japan promises its matchmaking algorithm will include). But the constant ‘evolution’ of these algorithms is antithetical to what research says about them: the simpler the better. Because after all, love is not a formula. But on online matchmaking sites, it can be a simple volume game.

The crux of algorithms developed by matrimony sites or dating apps is to compare information on the user against hundreds of thousands of other users to find matches. This happens either by matching people’s characteristics — social background, income bracket, age-range, gender — which is a relatively straightforward algorithmic exercise, or by matching how they behave on the website itself. The latter is embodied by dating apps such as Hinge, Tinder, or Bumble, in which the algorithm notes whom a user says yes or no to and then compares the pattern to others who exhibit similar patterns and preferences.

However, regardless of these personality- and behavior-based algorithms, a user’s appearance remains one of the most important factors when it comes to pairing. A 2018 study shows dating app users message others on a “hierarchy of desirability,” tending to reach out to people on average 25% more attractive than they are. The desirability in this study was measured by who got the most messages and the desirability of the person sending the message. This user behavior reflects a now-outdated system called the Elo score, championed by Tinder. It ranked users based on how many people swiped right on them, with the rank increasing if the person who swiped right had also racked up a high number of right swipes for themselves. This, again, is predominantly determined by people using each other’s appearance to determine attractiveness and, as one study shows, even trustworthiness or friendliness.

Related on The Swaddle:

Of Ghosts and Draculas: Modern Dating Trends, Explained

This research shows that regardless of how complex and evolved an algorithm may be, people still base their selection on a superficial trait — appearance.

So, despite an algorithm’s complexity, or its ability to consider a range of issues — “from personal interests, education, language, career, family, lifestyle to horoscopes,” as a Matrimony.com (which owns Bharat Matrimony) executive told the Economic Times — it simply doesn’t work. Keeping the overbearing shadow of appearance aside, experts say an algorithm still won’t predict how two people will get along with each other and whether they will have a successful encounter (or relationship).

In one study, Canadian researchers asked people to fill out a questionnaire about their preferences in romantic partners, yielding more than 100 individual traits they’d like or not like in a potential match. The researchers then subtracted the person’s choosiness and attractiveness — how they see others and how others see them — from the experiment to figure out if the traits and preferences they had listed had any bearing on their romantic desire for those who seemed compatible with them on paper.

After the participants went on four-minute-long speed dates, the head of the study, assistant professor at Western University in London, Canada, Samantha Joel, PhD, found the algorithm her team used to match people to be “completely useless.” She told the BBC, “My take is that when two people actually meet they form a shared dynamic that is more than the sum of its parts and cannot be predicted a priori.” She added, “Their individual preferences do not make up the substance of what they find attractive. My rating of whether I found you funny after meeting you will predict whether I like you, but my desire for a funny person and your measure of whether you are funny do not because we might not agree on a sense of humor.”

Joel’s study shows user preferences are complicated — does a person value attractiveness over humor and, if so, what’s the weight given to both of these traits in an algorithm? Do two people have the same definition of what kindness looks like or what having an adventurous spirit means? If predicting romantic success is complicated, can we at least use deal-breakers to weed out the absolute no-nos? Another experiment by Joel tried to answer this question based on the deal-breakers reported by study participants, with an eye toward filtering people out, rather than zeroing in on them. Even there, Joel and colleagues found the weight given to undesirable traits — such as a smoking habit — fluctuated depending on what other information the participant had about the person. A major factor was whether the person being considered had a desire to meet the participant; Joel found 74% of people who had listed a particular characteristic as a deal-breaker were prepared to overlook it when they found the other party had expressed romantic interest in them.

This speaks to a discrepancy between what people think they want and what they are actually comfortable with in a real-life setting. As Joel surmised, “People don’t know what they want,” which makes having algorithms gathering people’s preferences based on self-evaluations, and then matching them with others based on those preferences, highly flawed. “It seems to me based on the data that preliminary filters don’t work,” Joel adds.

Other research also bears this out — online matchmaking algorithms force users to screen others by searchable attributes, such as income, religion, or any other such externality. Even if the algorithm tries to go deeper, it includes a reductive idea of what traits such as humor or worldliness could look like. People are thus spending “too much time searching for options online for too little payoff in offline dates,” a 2008 study concludes. This is because “there’s no evidence that a high match percentage reliably translates into a successful relationship,” sociologist Kevin Lewis tells JStor, adding how well a couple is matched on paper is “irrelevant.” Referring to OkCupid, he adds, “At the end of the day, these sites are not really interested in matchmaking; they’re interested in making money, which means getting users to keep visiting the site. Those goals are even opposed to each other sometimes.”

Related on The Swaddle:

Tindstagramming Is a Creepy Online Dating Trend That Needs to Stop

Matchmaking sites, however, especially those promising long-term relationships and happy marriages, routinely put out self-glorifying press releases touting the number of successful romantic relationships borne of their algorithms. But these results, experts say, are simply a factor of a large number of people using the sites, meeting up, and essentially chancing upon their life partner. After all, when someone meets the love of their life online, they are bound to focus on the fact that it finally paid off, rather than the number of people that didn’t work out.

A 2012 study shows an algorithm cannot do any better of a job “than the randomness of the universe could,” Vox reports, because “there are inherent limits to how well the success of a relationship between two individuals can be predicted in advance of their awareness of each other,” the study’s authors write. They add the strongest predictor of a successful relationship is in the way couples “respond to unpredictable and uncontrollable events that have not yet happened. … The best-established predictors of how a romantic relationship will develop can be known only after the relationship begins.”

Essentially, it’s not necessarily the algorithms of online matchmaking that are the main draw of such platforms — it’s the robust user base and the ease with which users can come across and communicate with each other. People want to believe in some magic algorithm that purports to know them better than they know themselves “because meeting one-on-one is exhausting,” Joel says. The promise of success is “something that single people want to exist – it’s the romantic equivalent of an easy weight-loss plan.” But without people investing time and energy into the individuals they swipe on, without people meeting others in real life and experiencing life’s various ups and downs together, there is no way of knowing if they’re destined to a happy ever after together.