I’m the first to admit it: we might be popular, we might create a lot of great relationships, we might blah blah blah. But OkCupid doesn’t really know what it’s doing. Neither does any other website. It’s not like people have been building these things for very long, or you can go look up a blueprint or something. Most ideas are bad. Even good ideas could be better. Experiments are how you sort all this out. Like this young buck, trying to get a potato to cry.

We noticed recently that people didn’t like it when Facebook “experimented” with their news feed. Even the FTC is getting involved. But guess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work.
Here are a few of the more interesting experiments OkCupid has run.
Experiment 1: LOVE IS BLIND, OR SHOULD BE
OkCupid’s ten-year history has been the epitome of the old saying: two steps forward, one total fiasco. A while ago, we had the genius idea of an app that set up blind dates; we spent a year and a half on it, and it was gone from the app store in six months.
Of course, being geniuses, we chose to celebrate the app’s release by removing all the pictures from OkCupid on launch day. “Love Is Blind Day” on OkCupid—January 15, 2013.
All our site metrics were way down during the “celebration”, for example:
But by comparing Love Is Blind Day to a normal Tuesday, we learned some very interesting things. In those 7 hours without photos:
And it wasn’t that “looks weren’t important” to the users who’d chosen to stick around. When the photos were restored at 4PM, 2,200 people were in the middle of conversations that had started “blind”. Those conversations melted away. The goodness was gone, in fact worse than gone. It was like we’d turned on the bright lights at the bar at midnight.
This whole episode made me curious, so I went and looked up the data for the people who had actually used the blind date app. I found a similar thing: once they got to the date, they had a good time more or less regardless of how good-looking their partner was. Here’s the female side of the experience (the male is very similar).
Oddly, it appears that having a better-looking blind date made women slightly less happy—my operating theory is that hotter guys were assholes more often. Anyhow, the fascinating thing is the online reaction of those exact same women was just as judgmental as everyone else’s:
Basically, people are exactly as shallow as their technology allows them to be.
Experiment 2: SO WHAT’S A PICTURE WORTH?
All dating sites let users rate profiles, and OkCupid’s original system gave people two separate scales for judging each other, “personality” and “looks.”
I found this old screenshot. The “loading” icon over the picture pretty much sums up our first four years. Anyhow, here’s the vote system:
Our thinking was that a person might not be classically gorgeous or handsome but could still be cool, and we wanted to recognize that, which just goes to show that when OkCupid started out, the only thing with more bugs than our HTML was our understanding of human nature.
Here’s some data I dug up from the backup tapes. Each dot here is a person. The two scores are within a half point of each other for 92% of the sample after just 25 votes (and that percentage approaches 100% as vote totals get higher).
In short, according to our users, “looks” and “personality” were the same thing, which of course makes perfect sense because, you know, this young female account holder, with a 99th percentile personality:

…and whose profile, by the way, contained no text, is just so obviously a really cool person to hang out and talk to and clutch driftwood with.
After we got rid of the two scales, and replaced it with just one, we ran a direct experiment to confirm our hunch—that people just look at the picture. We took a small sample of users and half the time we showed them, we hid their profile text. That generated two independent sets of scores for each profile, one score for “the picture and the text together” and one for “the picture alone.” Here’s how they compare. Again, each dot is a user. Essentially, the text is less than 10% of what people think of you.
So, your picture is worth that fabled thousand words, but your actual words are worth…almost nothing.
Experiment 3: THE POWER OF SUGGESTION
The ultimate question at OkCupid is, does this thing even work? By all our internal measures, the “match percentage” we calculate for users is very good at predicting relationships. It correlates with message success, conversation length, whether people actually exchange contact information, and so on. But in the back of our minds, there’s always been the possibility: maybe it works just because we tell people it does. Maybe people just like each other because they think they’re supposed to? Like how Jay-Z still sells albums?
To test this, we took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)† Not surprisingly, the users sent more first messages when we said they were compatible. After all, that’s what the site teaches you to do.
But we took the analysis one step deeper. We asked: does the displayed match percentage cause more than just that first message—does the mere suggestion cause people to actually like each other? As far as we can measure, yes, it does.
When we tell people they are a good match, they act as if they are. Even when they should be wrong for each other.
The four-message threshold is our internal measure for a real conversation. And though the data is noisier, this same “higher display means more success” pattern seems to hold when you look at contact information exchanges, too.
This got us worried—maybe our matching algorithm was just garbage and it’s only the power of suggestion that brings people together. So we tested things the other way, too: we told people who were actually good for each other, that they were bad, and watched what happened.
Here’s the whole scope of results (I’m using the odds of exchanging four messages number here):
As you can see, the ideal situation is the lower right: to both be told you’re a good match, and at the same time actually be one. OkCupid definitely works, but that’s not the whole story. And if you have to choose only one or the other, the mere myth of compatibility works just as well as the truth. Thus the career of someone like Doctor Oz, in a nutshell. And, of course, to some degree, mine.
I’m really happy to finally see something new posted in OkTrends!
The information here is interesting and useful. It doesn’t bother me. Unlike everything that fb does- I haven’t had a fb account in 3+ years because I don’t agree with what they do and how they do it.
<3 OKC
It’s true, and you’re dumb for removing it based on a misapplication of statistics.
I really really hate it when websites remove features that I use just because “oh we did a survey and people don’t use them”. OkCupid is really, really guilty of this since they sold out to Match. Please bring back Notes. Please bring back Journals. Oh, who am I kidding?
“It correlates with message success, conversation length, whether people actually exchange contact information, and so on.”
The fact that OKC monitors this type of personal message info makes me really uncomfortable.
Wow. Did you actually try to word this in the most offensive, off-putting manner, or does that quality come naturally to you?
All your arguments for websites running tests on users aside, your presentation here reeks of arrogance. The condescending tone, the flippant attitude towards users’ (arguably valid) concerns about the extent to which web-driven businesses manipulate them makes me sick to my stomach. This very attitude underscores the essential problem with most web-based businesses. You live in an insular world in which your users wishes, concerns and emotions are secondary (at best) to your profit. Nothing pierces that wall of arrogance. You havent presented a shaded, nuanced argument here parsing the need for business development with ethical concerns. You simply claim the ground that a web business can do what it likes at any time. After all thats the way the internet works. No ethics required.
If you have this aggressively dismissive approach to users in public, I shudder to think what your corporate culture might be.
I used to admire your company. No longer. Wont use it.
It’s funny to me how outraged people are at OK Cupid, Facebook, etc. for “manipulating feelings” when the US government main stream media does it every single day. For some of you younger readers, your entire life has been “manipulated”. The medical term is called “conditioning”.
Only a maximum of <20% of first messages does turn into a conversation?
NOT COOL !!!
As a user of OKCupid, I find your write-up of Experiment #2 to be quite lacking, your conclusions disingenuous. You have to be aware of how much the site practically demands that you rate users. Indeed, if you use the “Quickmatch” feature, you can’t move on to the next person until you rate the first one you’re presented with. Quickmatch also does not give you the entire profile to look at, and given that the person’s username is not displayed, the only way to easily find the user again in order to read more about them is to rate them high so that they get listed in your “Who You Like” list. It is MUCH easier to give someone a quick 4 star rating and save them to review later so that you can quickly get through the Quickmatches.
Or how about the “Browse Matches” feature? You get a page of face shots and the moment you hover anywhere over a listing, the empty Star Ratings pop-up, begging you to click on a star before you ever click on their profile to learn anything about them other than age, location and match percentage.
Messages are presented the same way – hover over a message in your inbox and suddenly you are invited by those stars to rate someone based on their primary face pic and about 10 words from the first sentence of their message.
Given the misleading nature of your conclusions in this section, one has to wonder about the content of your entire post. I couldn’t care less about these “experiments”. But going out of your way to insult your users’ intelligence and call them shallow? Now that’s going to have me searching for another site to use in the future. At least AFF is honest about everyone’s true motivations.
IT LIVES!
So glad to see the blog is back. Hope you will post more!
The American psychological community has spent decades wringing hands about the ethical implications of experimenting on other human beings: highly trained professionals taking, in the best cases, the greatest care to do no harm and emphasizing deep respect for the power that is wielded whenever one have the ability to manipulate fellow human beings.
The casualness of this blog post strongly suggests to me that its author, and perhaps OKCupid’s staff altogether, isn’t nearly mature enough to understand the ethical issues of the experiments the company is conducting — nor, specifically, does anyone seem to have any sense of how unethical the third experiment here was.
The author seems to have no sense that he has power over others and that people are making themselves vulnerable to OKCupid and that they are trusting OKCupid — nor that that constitutes a responsibility to these people. He seems to show no careful consideration of the consequences of manipulating people at all. Furthermore, he seems to think that this article is somehow more about him than about the people being manipulated — as indicated by the amount of time he spends poking fun at himself and his colleagues compared to the amount of time he spends considering the effect of secret manipulations on the site’s users (none).
Perhaps he feels the consequences will be minor. But he seems to have no idea that whatever the consequences, he and OKCupid are culpable — morally, legally, in every conceivable way. Whatever consequences result from their manipulations will have been caused by them, directly, and caused to people unawares.
I think it can’t be emphasized enough that its clear that OKCupid is playing with people’s personal lives without any evidence of careful thinking about the results — or without careful thinking about these people as people.
I see no controversy: it’s the wrong way to treat others. Frankly, it’s tinged with sociopathy, and the way it’s touted here rings with narcissism.
I very much hope the APA begins to respond to the manipulative practices of social websites. No one would want a bunch of twenty-something or thirty-something entrepreneurs without medical training developing new drugs for people; I think this case, though differing in degree, is not categorically different from that. OKCupid’s racking up moral debt here.
Your justification that “all websites do experiments” is insipid, adolescent, and a pathetic cop out. Not all websites deal with such emotionally charged exchanges and communications as yours, and moreover, not all websites have paying subscribers they have agreed to terms of service with.
Did it ever occur to you that a lot of your users are paying a subscription fee for the services you stipulated, and in your experiments you delivered a service that was not only “not what we paid for” – but also quite potentially counter-productive to and the opposite of what we paid for? Did that enter into math?
Did it occur to you that a lot of users on your site are under personal duress, dating is difficult, they are struggling emotionally and it might be, I don’t know, INHUMANE to use them as guinea pigs, especially since the TRUST you?
Did it occur to you that you likely caused at least some folks real emotional pain?
Did anyone on your genius staff say “no, I don’t think this is a good idea, we shouldn’t do this, it’s unethical”?
These are not rhetorical questions.
As a small coffee shop owner I thought it would be interesting to find out what would happen if I put salt in my customers lattes!. What I found was that only a small amount of salt was enough to make customers never come back to the shop.
Im hoping to scale up my research for a much bigger study in the Autumn and hope to experiment with other admixtures such as fingernails, hair and coins.
I am really looking forward to the book.
I remember this site before it was much more than user driven quizzes, back when I was 18 in college and looking for something to help me pass the time (I was the Priss originally. Hahah…).
I love how much you’ve been able to do with the data you pull from us (because I know people are most honest when they’re anonymous), even if I’d like to think I’d notice if I were matched secretly with a 30% (they just aren’t as good at writing as the guys I match with in the 90’s). The last few serious relationships I entered into, I met the guy on OkCupid, so I know it works (see latest wishful messaging I’ve been doing with a guy in Australia)
Anyway, I’ll be picking it up at my bookstore in September. Thank you for working so hard on the site still. It is deeply appreciated.
I find this type of action, unethical, manipulative and illegal. In addition, your description boasts in a sarcastic manner of the unethical operations you undertook.
I have four words: Belmont Report. READ IT!
Here’s the slightly longer version: If you are conducting user experience testing for internal product development, then you are technically not conducting human subjects research. Experiments 1 and 2 are natural things to try, because they give you information about your product. Had you kept those results to yourself, what you did would have been perfectly fine.
However, you went too far in two ways. First, by publishing this post, you are making an attempt to add to the general understanding of human behavior. That means you are now engaged in human subjects research and are ethically bound to the guidelines in the Belmont Report. The one critical, fundamental necessity that both you and Facebook flaunted was the notion of informed consent, which derives from the principle of Respect for Persons. A general data usage policy is NOT informed consent. Informed consent has to be for the specific research project.
What is especially problematic is Experiment 3, which incorporates deception. Again, the Belmont Report and subsequent publications make it clear that special care must be taken when deception is part of the research, because the study violates respect for individual autonomy. By employing deception, failing to acquire informed consent, and failing to conduct adequate debriefing afterward, Experiment 3 is HIGHLY UNETHICAL, particularly because you have chosen to publish these results.
If you wish to continue with this type of research, please do so. Your service is well-positioned to contribute to generalizable social science knowledge. But, first, take the time to learn about research ethics in social science and adhere to known and accepted principles. The CITI Program is a fantastic place to start.
I recently added a brief summary and when to PM me on site but linked a tracked document from my profile with the rest of the essays to see how many people would be curious enough to continue reading.
Is there any way you guys could provide me with some visitor stats of my profile to compare with the actual clicks to this doc?
My two cents, probably not worth a Jamaican penny.
People voluntarily signed up for a site that is quite literally solely based on the algorithms that they (the user) provided data for. Now said people are upset that that data was used to show that a majority of its users (or at least the ones in the test groups) are shallow and judgmental. Get real. They only thing OkCupid did was put some numbers and graphs to something we all knew was the truth in the first place and let us know that we are not as interested in the phrase “It’s about what’s on the inside that counts” as we claim to be.
If you don’t like it, go meet people the old fashion way; whatever that is.
*Steps off soapbox*
Yay, I’m so happy this blog is back!
And as a web developer, they’re definitely right that people are being experimented on all of the time on the web. We have few best practices, and massive amount of analytics. If you don’t like it, I suggest browsing using TOR.
I am having a hard time getting as upset about this as a lot of the commenters here. You’re right, basically everyone here is toast already and apparently a lot of us haven’t figured it out yet. Your every move is tracked, your every interest is well-documented, regardless even if you’re on the internet. If you have EZ Pass, or a cell phone, or a credit card… everyone getting this? Companies of all kinds know exactly where you go, what you spend money on, and when you do it. You my friend have been figured out. You probably buy a lot of things that you might not have purchased had someone not figured out what you like and then presented you with an ad at just the right moment. This thing above with OK Cupid, and the thing with Facebook, you gotta be kidding me if it upsets you. FACEBOOK, for crying out loud, where people give all their location info, relatives, jobs, books, movies, music, life announcements and photos galore? Who needs a spy agency anymore? You’re giving it all away. So if this bugs you, well, now you know.
Except that you charge people for enhanced service, so the FTC and FCC should be involved.
It now makes me wonder how many fake profiles and matches are on OKCupid.
I wish I had heard of this before purchasing an enhanced membership. I will terminate it before it renews.
I got married to an OKCupid match, I messaged on a thursday, met on the friday, engaged after 3 months and married on NYE 2013!
She is now 7 months pregnant.
Experiment all you want….
TL, you miss the point.
The problem is the manipulation of the matches. Telling people they matched when they didn’t and didn’t match when they did, wasted their time and maybe money as well.
To everyone complaining about OKC collecting their data – don’t use the service if you don’t like it. Epidemiological studies collect data like this all the time about interaction patterns in the real world. Nothing has changed.
What have you been doing for the last three years, and whatever it’s been, why haven’t you been updating this blog? I keep coming back to it every few months hoping to see something new, and finally, on the heels of a “scandal”, here it is!
More! The audience wants more!
Man, such silly responses.
You’re using / paying a company to profile you and other people in hopes of matching two people together.
How did you think they did this? Magic?
You know what I find sad?
The companies who are open with their policies and methodologies are the ones that get all the flack for it, while the companies who remained silent, tight-lipped, and sometimes outright lying are the ones who get away scott-free.
For example, Google shows you most / all the information they have on you if you bother looking (Dashboard). They openly admitted to making mistakes in gathering too much information when they could have just swept it under the rug.
Then you take a company like App1e who regularly rapes your information for their own ad network (why do you can’t seamlessly replace their maps, messaging and voice assistant apps? all of them send information back to their servers), and records all of your rough locations and sends it back to their servers without telling you (for the wifi based GPS especially)…
and people in record numbers still buy it… because nobody actually cares about this and are probably just trying to get some free service out of it.
A few things here:
1. These aren’t “experiments” as much as they are post-hoc examinations of data. You didn’t alter users’ experiences on purpose to test hypotheses – there was no manipulation of a variable, hence no experiment. Facebook actually altered peoples’ newsfeeds to be more happy or more negative.
2. I would argue that some of the correlation between personality and looks ratings comes from peoples’ personalities being evident from their profile pictures. For example, one person my dress as a hipster with a goofy look on face. A certain style of dress may often (but not always) be associated with certain political leanings, for example. I think you can tell a lot about a person’s personality from a profile pic. Is it artsy or boring? Is it serious or fun-loving? That correlation is no surprise, and does not mean it’s all about “looks” (i.e., just and only physical attractiveness).
Great insight. Thanks for being upfront and honest about the results of these experiments.
Christian, you are a data guy. Is this all you came up with after 3 years? As a psychologist, I won’t add to the pertinent comments of other readers. There are some very eloquent responses to the attitude of your company in conducting these quasi experiments. Personally, though, I would like to see more of these data observations but without the sneaky experimental angles. And perhaps you could open your data (anonymized of course) for scrutiny by social researchers. For example, I would be interested to see if there are any racial biases in the ratings. And how do different countries compare? Or let some social researchers help you – I think we might come up with a better hypothesis than “hotter guys are assholes”. Is that really what you think of your customers?
I’m so happy you’re posting again. This has always been one of my favorite parts of the whole experience of OKCupid. Thanks for experimenting on us, this is exactly what makes this site so great.
Enjoy the impending legal issues for submitting people to experiments without their consent.
Do you employ social scientists or does OKCupid do their own data analysis?
When you told your legal team that you wanted to run a series of experiments on thousands of human beings without their consent how did that go over?
Did anyone explain to you that there is a huge difference between collecting and analyzing data and manipulating and analyzing data? The first tells you how humans behave. The second tells you how humans behave in a false situation and is essentially useless for scientific research.
You might want to read this before you glibly talk about human experimentation like it’s a fun little parlor trick.
https://explorable.com/milgram-experiment-ethics
Now how about you update the how matching works page & publish the new statistical weights for questions since you removed Mandatory? Bloody annoying.
So I am doing this experiment where I deleted my OK cupid profile to see if they can interfere with my life/emotions without me having an account. My findings so far is they can not touch me without my account
I adore the style of these writings. <3
Unflip the match %, tell users who were affected, and give us an opt out of experiments option.
So, when will I be getting my pro-rated refund for the time that you spend “testing” and skewing my search results, loss of service and function (that I’m paying for), etc.?
@sicktomystomach – I agree completely
Dear Mr. Rudder,
The existence of this blog has always made it perfectly clear that OkCupid performs analysis of bulk data, in aggregate. Your findings have been a (sadly infrequent) source of surprise and delight, to me. Bravo!
Giving people bad matches on purpose went too far. I’m no lawyer but it seems there was an implied contract: people gave OkCupid their data with expectation of GOOD matches in exchange. Shame on you!
Keep up the GOOD work. Dial back on the bad.
Sincerely,
[:-)] Mark
Mark Zajac
Losers…
I’m pretty shocked at the naïvety of many commenters here. Are they all under the impression that OKC *magically* knows how compatible two users *actually* are, and is choosing to hide that information from some of its customers?
“Oh no, I think you’ll find I signed up for the service where you tell me my *REAL* compatibility with the_huggle_monster, and not the one where the number is just a guess based on a few spurious facts you now know about the two of us”
OF COURSE OKCupid plays with its matching algorithm, that’s how it became the popular site you all chose to sign up to to meet other people.
I’ll be honest, I’m not sure why people are so up in arms. Personally, I love that they release this data and express this passive-aggressive disappointment in humans to avoid being hypocritical. I think a site like OKC is perfectly poised to help change the trend of some of these bad numbers. Perhaps humanity NEEDS to be conditioned off self-destructive traits. It’s a fair debate.
Sorry, Christian, to hear the BBC Radio 4 interview just ended trying to make a big controversial issue out of almost nothing out of this. The BBC does this sometimes (usually with political/PC issues though).
When were these experiments run that manipulated the match percentages displayed? There was a period a few weeks ago when I kept getting 90%+ match percentages for guys who were completely opposite of my views on religion and way off of my stated preference for education level. I wondered how the heck they became 90%+ when we were obviously a mismatch on such important issues.
To file a complaint with OkCupid, go into your profile, click Settings then use the contact link at the bottom of the page. I just notified them that I expect a pro-rated refund for the entire time that they conducted their “experiment” or I will gladly join the first class action suit against them on this matter.
You can experiment all you like with your offer and your business model as long as you don’t charge me for the privilege of being your Guinea pig!
I pay you money to use your site, and just find out that you may have lied to me about the match percentages just so you can gather data. That’s fraud. I pay you for a service that deliberately undermined, and I never gave you consent to lie to me about the match percentages. You owe every one one of your paid subscribers all their money back and it’s only a matter of time before we end up suing you for this fraud. Who do you think you are to toy with people’s emotions like this?
You people are assholes. You take away the ONE thing that set this site apart from all the other stupid online meat markets – the JOURNALS – and spend your time thinking of ways to poke us with a stick.
I think OKCupid have been rumbled for putting up fake attractive profiles to entice in paying members. All dating sites do it but the excuse is always different. The idea an attractive real member is going to date a plain person through OKs match system is bollocks.
This article is fluff to muddy the waters about the most glaring point: They manipulated their match percentages, a clear violation of their service. This is the only reason to use Ok Cupid. Just another reason this site sucks since bought by match. But it doesn’t really matter, because the quality of people is on the decline as new, frankly ugly, people come over from Match. This is why anyone good looking has moved on to Tinder.