I’m the first to admit it: we might be popular, we might create a lot of great relationships, we might blah blah blah. But OkCupid doesn’t really know what it’s doing. Neither does any other website. It’s not like people have been building these things for very long, or you can go look up a blueprint or something. Most ideas are bad. Even good ideas could be better. Experiments are how you sort all this out. Like this young buck, trying to get a potato to cry.

We noticed recently that people didn’t like it when Facebook “experimented” with their news feed. Even the FTC is getting involved. But guess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work.
Here are a few of the more interesting experiments OkCupid has run.
Experiment 1: LOVE IS BLIND, OR SHOULD BE
OkCupid’s ten-year history has been the epitome of the old saying: two steps forward, one total fiasco. A while ago, we had the genius idea of an app that set up blind dates; we spent a year and a half on it, and it was gone from the app store in six months.
Of course, being geniuses, we chose to celebrate the app’s release by removing all the pictures from OkCupid on launch day. “Love Is Blind Day” on OkCupid—January 15, 2013.
All our site metrics were way down during the “celebration”, for example:
But by comparing Love Is Blind Day to a normal Tuesday, we learned some very interesting things. In those 7 hours without photos:
And it wasn’t that “looks weren’t important” to the users who’d chosen to stick around. When the photos were restored at 4PM, 2,200 people were in the middle of conversations that had started “blind”. Those conversations melted away. The goodness was gone, in fact worse than gone. It was like we’d turned on the bright lights at the bar at midnight.
This whole episode made me curious, so I went and looked up the data for the people who had actually used the blind date app. I found a similar thing: once they got to the date, they had a good time more or less regardless of how good-looking their partner was. Here’s the female side of the experience (the male is very similar).
Oddly, it appears that having a better-looking blind date made women slightly less happy—my operating theory is that hotter guys were assholes more often. Anyhow, the fascinating thing is the online reaction of those exact same women was just as judgmental as everyone else’s:
Basically, people are exactly as shallow as their technology allows them to be.
Experiment 2: SO WHAT’S A PICTURE WORTH?
All dating sites let users rate profiles, and OkCupid’s original system gave people two separate scales for judging each other, “personality” and “looks.”
I found this old screenshot. The “loading” icon over the picture pretty much sums up our first four years. Anyhow, here’s the vote system:
Our thinking was that a person might not be classically gorgeous or handsome but could still be cool, and we wanted to recognize that, which just goes to show that when OkCupid started out, the only thing with more bugs than our HTML was our understanding of human nature.
Here’s some data I dug up from the backup tapes. Each dot here is a person. The two scores are within a half point of each other for 92% of the sample after just 25 votes (and that percentage approaches 100% as vote totals get higher).
In short, according to our users, “looks” and “personality” were the same thing, which of course makes perfect sense because, you know, this young female account holder, with a 99th percentile personality:

…and whose profile, by the way, contained no text, is just so obviously a really cool person to hang out and talk to and clutch driftwood with.
After we got rid of the two scales, and replaced it with just one, we ran a direct experiment to confirm our hunch—that people just look at the picture. We took a small sample of users and half the time we showed them, we hid their profile text. That generated two independent sets of scores for each profile, one score for “the picture and the text together” and one for “the picture alone.” Here’s how they compare. Again, each dot is a user. Essentially, the text is less than 10% of what people think of you.
So, your picture is worth that fabled thousand words, but your actual words are worth…almost nothing.
Experiment 3: THE POWER OF SUGGESTION
The ultimate question at OkCupid is, does this thing even work? By all our internal measures, the “match percentage” we calculate for users is very good at predicting relationships. It correlates with message success, conversation length, whether people actually exchange contact information, and so on. But in the back of our minds, there’s always been the possibility: maybe it works just because we tell people it does. Maybe people just like each other because they think they’re supposed to? Like how Jay-Z still sells albums?
To test this, we took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)† Not surprisingly, the users sent more first messages when we said they were compatible. After all, that’s what the site teaches you to do.
But we took the analysis one step deeper. We asked: does the displayed match percentage cause more than just that first message—does the mere suggestion cause people to actually like each other? As far as we can measure, yes, it does.
When we tell people they are a good match, they act as if they are. Even when they should be wrong for each other.
The four-message threshold is our internal measure for a real conversation. And though the data is noisier, this same “higher display means more success” pattern seems to hold when you look at contact information exchanges, too.
This got us worried—maybe our matching algorithm was just garbage and it’s only the power of suggestion that brings people together. So we tested things the other way, too: we told people who were actually good for each other, that they were bad, and watched what happened.
Here’s the whole scope of results (I’m using the odds of exchanging four messages number here):
As you can see, the ideal situation is the lower right: to both be told you’re a good match, and at the same time actually be one. OkCupid definitely works, but that’s not the whole story. And if you have to choose only one or the other, the mere myth of compatibility works just as well as the truth. Thus the career of someone like Doctor Oz, in a nutshell. And, of course, to some degree, mine.
To test this, we took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)†
This is supposed to be okay because you “alerted the users of the correct match amount afterwards”?
Don’t you have to alert users BEFORE you perform manipulative experiments on them?
–someone who no longer users your site.
I agree with what Mac says. It is a shame that OK Cupid thinks it is okay to experiment with its customers without asking for their consent. Now they send out a newsletter to brag about it! Yes, it is good that they notified the people afterwards about the experiment; but, their careless attitude towards toying with people as an “experiment” makes me wonder what other “experiments” they are running. I joined this site to find a relationship, not to be a guinea pig in a social experiment.
Wow. What a shitty thing to do.
I’d like to see an in-depth study of how the overall quality of a collection of photos in a person’s profile relates to how “attractive” they are both online and offline, and especially, what the ideal combination of types of photos is. For example, if you think about how human instinct works, some part of how we discriminate is based on how well the individuals we evaluate fit a perceived representation of ourselves. Likewise, some part of it is based on how energetic/healthy a potential partner appears, and we very intuitively and subconsciously rate him or her as more or less attractive than us, as well as more or less attractive in general. But consider the degree to which camera lenses and focus, digital compression, display type, lighting, Photoshop, and so on and so forth, distort the representation of a potential partner from how they appear in three dimensions and in person. Do you find it reasonable that the vast majority of us make many assumptions of how appealing that person is to us because we cannot be sure of how accurately we can determine this simply by looking at photos, and more importantly, we aren’t good at figuring out what we’re assuming? Gee, I’m not surprised women seem to enjoy blind dates equally whether the men they’re dating is more or less attractive than they are. (Note the statistical anomaly of 92%. It seems the greatest common denominator like their men to be that much — slightly — “hotter” than they are. And I’d absolutely love to see just how much men enjoyed blind dates with women of various degrees of attractiveness, but that graph is not published for one reason or another.)
Also, something I find very bothersome about these studies is that they express very accurately a trend in a sampling of the entire OkCupid population in total, but it omits just how far any observed statistical anomaly deviates from this mean average. It allows readers to make far too many assumptions — if an incalculable possibility for, say, a demographic of people who are the complete opposite what the graphs show to exist, why aren’t we illustrating that? Why are we painting such a large number of people with such a wide brush that the very scant few diamonds in the rough, so to speak, are made invisible? To do so is — be it intentionally or unintentionally — misleading.
What’s up set me up with some good ratings. Jk I like the guinea pig thing that’s how you learn. You should hold another blind date day. I’m down. The rating things funny. Question do you think most people are on here to fuck or find a good person? You have studies on that? Lol
Christian, thanks for sharing this. The results makes complete sense, particularly given the medium.
First, we know that we are terrible at predicting our own behavior. We can’t predict what car we’ll buy, what TV show we’ll like, or what we’ll have for dinner tomorrow. So ask us to fill out a complex profile of who we are (we don’t know) and what we want (we only know it when we see it), and of course it’s not likely to work very well.
Second, we also know that when we’re together, non-verbal queues are the vast majority of communication. So if it’s picture vs. words, we’re all about the picture. It’s not shallow, it’s completely natural. But add more real information and we’ll use it.
But it works – so long as we don’t over filter the people we meet. As long as OKC gets us to meet enough candidate mates, we will eventually find one despite ourselves. Please continue to experiment – find ways to better measure our actual reactions and you’ll make a huge difference for us.
-Steve
So, is there anyway of knowing if I was used in your compatibility match-up?
I doubt it since I will reject a candidate if I think he isn’t politically progressive, hence I read every word of the profile and the answers to the OKC questions…I have a lot of deal breakers and racism or political conservatism are the Biggest deal breakers of all….
I’ve met some great guys on OKC, my first hit, a 98 per cent match was the greatest of all. He wasn’t good looking but he is a truly wonderful person….sadly, he met another. We are still friends.
Your site is fun and I like the interactions. But, I haven’t met The One. I don’t think I really want to. I am a widow and my husband was the greatest man I’ve ever met…others don’t compare.. Oh well
lollyv
Stop nagging about how unethical this is. Nobody got hurt, not even their feelings. In the worst case they were confronted with their human nature: you use your eyes first and that impression sticks.
Sky shattering science indeed.
What are people in the comments more worried about: the fact that OKCupid “experimented” on them or the fact that they are just regular ol’ people with the same shallow habits as everyone else.
And if you just let your personal feelings go for a second, realize that these experiments are actually very valuable. Given their sample size and the fact that everyone was unaware OKCupid could test behavior that is otherwise almost impossible to measure.
Well done guys!
I think the most relevant statistics is about the percentage of meaningfull relationships started here and thats all and very simple..simplicity is alwaysvthe answet
I think the most relevant statistics is about the percentage of meaningfull relationships started here and thats all and very simple..simplicity is always the answer
THIS IS A PRETTY BIG PRICK MOVE AND I THINK THE PEOPLE THAT RUN OKCUPID SHOULD BE FIRED AND REPLACED. OBVIOUSLY THEY ARE ASSHOLE PRICK ASS MOTHER FUCKING PIECES OF SHIT. YOU GOT THAT YOU TWO BIT CHEAP ASS MOTHER FUCKING CUNTS? NOW DO THE WORLD A FAVOR, AND QUIT BEING MANIPULATIVE LITTLE BITCHES AND PROVIDE A SERVICE INSTEAD OF A SCAM. AND FOR THE CHEAP TRASH BAG HO THAT RUNS THE SITE, GO FUCK YOURSELF YOU DIRTY TWO BIT CUNT.
I haven’t had a single visitor to my profile since this blog post. Coincidence?
I agree totally with this experiment,I believe it would help the matches to find their rightful search and also increase alot of real success of relationship,I believe its a good Test take and you intelligent in finding out all this…
Though I would be forced to agree that some of the article was a bit dry(a statistician’s wet dream vs. an everyday Joe’s mathematical lullabies, lol) I have absolutely zero problem with these “experiments”and assumed they were being done already. For one, any successful company does their homework. If OKC wants to use what works, they have to be able to define what, in fact DOES WORK.
To all of you indignant daters: who cares? They are not trading your personal information, stealing your identity, announcing your personal involvement in the testing…so what is the big deal? In my opinion, people just want something to bitch about. If this is all you can come up with, no wonder you can’t score a date. How boring!
Second, these tests are not conducted to attack; they are conducted to improve upon a site which, though it cannot match every person with his or her soulmate, provides options for those of us uninterested in bars, churches, being bothered while deciding on the cheapest peanut butter while looking our worst in grocery stores.. I’m thankful to OKC for helping me realize that there are still as many good people and bad people in the world. And bc of them, I don’t have to waste more than a few minutes finding out if I’m attracted to someone add opposed to eating entire evenings and begging my bff to call me and make something up to get me out of two more hours with the date who won’t shut up about how great he is with radishes, or unwanted sexual advances, or, my personal favorite horrible first date: the scary dude who asked if I would wear a gimp suit to our second date.
social experiements are always interesting, and although it has to be held secret for a time from people to ensure authenticity in its results, they do shed some light on the situation. it seems the overall goal for okc with these expirements was to make finding a compatable date more efficient and successfull. sure it required some manipulating, but the cost wasn’t too high. no one died, people found themselves becoming compatable despite statistical assumptions. i don’t find it too criminal.
Personally I could give a shit about the experiments but bottom line your findings stat boy.
P.S. Besides…this is a FREE site! If you believe e harmony or match.com does NOT perform their own testing/experiments, feel free(again, FREE!) to go pay ridiculous amounts of money to STILL find out you’re single on Friday. For those leaving comments full of obscenities and personal attacks: Gee, wonder why you can’t find someone with all that pathetic narcissism, cruelty and negative attitude. Rage is fairly unattractive…even on e harmony…lol!
I like that you’re fine-tuning your creation . At least you’re paying attention .
May i suggest having a “help button ” for common inquiries … i often want to check on or alter particular things & stumble into the answer quite by accident, if at all.
it’s rather frustrating. (And -lol-i am clearly already quite frustrated )
Sincerely ,
Prepared to die celibate
I may have met tne love of my life who was a 68% match. We laugh about it and he says that I just sicnr answer as many questions as he did.
I chatted with others who were a height % match and as you said tried to make it work because of the % match and was trying to convince myself that it was the right person and I had just not been looking for he right things until now.
Thankfully I ignored the math and actually followed what I liked in someone’s profile. And not even tne photos in this instance. Thanks OKC for leading me to love but not who ever showed up as a match so I’m just lucky I guess
No comment at this time. I just joined yesterday.
There is something called trust, especially when paying customers are involved, for your methodologies when we decide to opt in. You just broke that for a lot of us by reneging on what you had promised while taking our money.
And getting free press may be quite self-indulgent and gratifying I’m sure but portrays you in bad taste, if you care to know.
Interesting. i wish i was tested with this. I often wonder what causes people to message me: hey we are a match what is up. I give a thoughtful reply with interesting topics to converse and i never hear anything back. I wonder how the old scale would fair now a days. about the same or a different trend?
on a side note I really didn’t care what facebook did, nor do i care when companies do this. People are ignorant to the fact it happens daily by everyone, we are on camera everyday.
If tests like this bother you, than you really need to move out to an area of the world that has zero connections to the outside world or first world technology.
Oh yeah, one more thing. Most of the single women on OKCupid who’s profiles I’ve read, talked to and met are phony and shallow in the end.
I find this interesting because I’ve been complaining openly to friends over the past several months that OKCupid was not utilizing the criteria I have in place. Every time I saw my matches I only got upset. You can’t fool me. I think the experiment will backfire horribly. I have been tossing around the idea of leaving the site because of the poor matches. This new revelation by OKCupid has me one step closer to making that decision. Does the site exist to help people or does it exist for the entertainment of those who run it?
Don’t understand it all,but,keep working……
I actually think the experiment was a good idea people do react differently if they know there a match of coarse some of there matching dint make any sense to me but I was always curious how they matched people up. I might have to read the book iv always been interested in psychology and how the mind works. Of coarse they really should let people know before the experiment but how many people would be willing to do it then. One more thing how about people just read there profile and look at there pictures and judge for them selfs to see if there a match in stead of relying on what the website says that’s why online dating is so popular to get to know the person before you judge them and on a deeper level so your more comfortable when you meet in person. Not saying I’m perfect I can be shallow at times but for good reason doesn’t mean I’m not willing to talk to other women I would rather connect instantly see if we have have similar goals and beliefs instead of jumping into a reslationship out of only lust on the mind.
I am a Masters candidate and I thoroughly approve of this.
Good for you OKCupid,
You are the only Free Dating Site, that attracts people of substance.
Honestly, if someone is upset, then they can move over to one of
pay sites.
I hope your experiments bring about some new venue,
that is the next hottest thing. I ask that if you are going
public with it. OK cupid users get first crack at buying
your stock.
Good luck on your experimenting.
Spirited4u
Bollocks! OKCupid didn’t tell users they were part of an experiment. Here is what they sent everyone:
“Dear [user’s name]
Because of a diagnostic test, your match percentage with [false match’s name] was misstated as [fake%]. It is actually [real%]. We wanted to let you know!
Best,
OkCupid”
They tried to play it off as diagnostic test error.
I asked for a full refund and was refused. After an interesting discussion with customer service (through email) and their repeated attempts to try to justify to me that what they did wasn’t wrong, it was only a short time, and only about 1,000 users, I replied with this:
“It doesn’t matter if it was an experiment done with 1 user or 1,000 users or all OKCupid’s users. It doesn’t make it ok. What OKCupid did was unethical and wrong. If OKCupid cannot see that, then it needs to have a little sit down to re-examine its business practices.
Would it be alright for a car manufacturer to at some random point in time switch the function of people’s accelerator and gas pedal, even if just for a short period of time, just to see how a very small number of drivers would react? Only to send them this message afterwards:
Dear Driver,
Because of a diagnostic test, your brake pedal in your car was misstated as an accelerator. It is actually a brake. We wanted to let you know!
Best,
Your Car Maker
I guess that makes everything ok, right?”
I’ve been issued a full refund.
Wow, relax everyone. For those of you saying you joined okcupid in search of a relationship and not to be part of an experiment, what do you think dating is? Okcupid has done nothing wrong nor has it wasted your time, all they did was open up more opportunities for you. Look at this more positively and you will see that they aren’t screwing everyone over…they are just expanding your options and making the match making process better by doing these things.
Hilarious! I read this on Drudge Report & immediatly was going to my profile to change something, maybe put a photo of a lab rat! But, you beat me to it by posting the blog. Damn, thought I would get all kinds of likes! Honestly, I use this site kinda as an experiment myself. It’s totally true that I go right to the photo. You probably didn’t even need to track that one. Kinda obvious, especially for guys. Anyway, keep up the good work. Just don’t give the NSA all this info. Oops,,they probably have it anyway!
Spirited4u, a lot of the people that are upset about them undermining their own core selling feature (the match percentages), Paid for A-list, including me, so that’s a bad argument.
Anyway, I have no problem with data mining and even some minor experiments, but outright lying to the users like they did with the third experiment was wrong and broke all trust. I hope the person directly responsible will be fired and the company will give a formal apology to its users as well as inform all that were affected by that experiment. Otherwise I don’t see myself ever using this site again and from the reactions here, I don’t think I’m the only one who thinks like that.
Unbelievable. What you are doing is normal research procedures for any company in the world. These “aliens”, slinging mud and fowl(spelt right)language at you, obviously don’t know or realise exactly what info is being flung around the world via the internet as soon as any device with a computer/GPS is turned on, including camera devices. So through me in the cage, fellas, as long as that little hotty hugging the log is with me.
I’m waiting for a response to my request that you stop wasting my time by sending me “Matches” that aren’t even close to being matches.
I think you have a basically good system.
if you knew how many people were ready to jump ship on FB because of their manipulative shenanigans, you would think twice about lielng to people,
If you want to lie to people and try to improve your website, how about offering a “beta” version with the caveat that it may or may not be accurate. Let people have informed consent and it will be longer before you have to deal with lawsuits and government regulators.
Just sayin’ . . . a word to the wise -you might not be teflon for ever.
I love
Thank you for making this whole experience more interesting!

I love girls i love you eating i love dating i love sports and sex
My only comment is this, if you take away what people say in their profile I will not look at the picture alone because what i am seeking is someone who knows how to communicate so I feel disappointed and let down by your experiments. You are effectively reducing the possible connections for me! However I never rely on just one site for meeting people and this is a good reason to continue doing so and happier that I have not paid for the reduction in your service!
Mea culpa. You got busted.
brilliant. i just love this. well done everyone…well done.
Loved the article, loved the humour, loved the unapologetic sarcasm. Just a shame too many people think that OkCupid somehow *owe* them something. These experiments were clearly done to improve the user experience and to help develop better algorithms for future use. And besides, none of the above are even harmful experiments. You’ll also probably find you agreed to such things in the T&Cs of the page when you signed up (ignorance is not an excuse). Climb down off your collective high horses and stop crying foul.
I think your research is inadequate. I notice that when I moved from Oregon to California my ability to get a date dropped from decent enough to near zero. I also found the women I was paired with to be less interesting though much more attractive. I think there is a regional cultural element that you may not be capturing adequately. Of course it could also just be my bad luck.
So if we don’t use pictures to match on we are less shallow but if we don’t have one up on the site we’ll get approached less. Well that’s not a huge revelation, all dating sites bang that drum.
I’d rather see some more meta analysis like on the proportion of women who say in their profile “If you want to know more just ask” or “If you like what you see message me” or “I think the best way to know each other is to chat” and then never it seldom reply to messages. I would reckon there is a pretty high correlation between these phrases and ignoring messages.
The admition of the experiment totally explains what happen to me 2 weeks ago. They pretent to be users and contact account holders. I’m glad i didn’t fail to keep my personal rules. SO MANY CUPS keep waiting for those picture!!!!
Fortunately, I never take any notice of the % match suggestions. I look at photos, read profiles and form my own opinions.
I joined this site not because I thought OkCupid had all the answers but because it’s a good place to start if you want to find singles. Anyone who thinks that they can look at the compatibility percentages and find their soulmate is an idiot. Personally, I may take them into consideration but I read a persons profile to try to get a sense of who they are. All of us want someone that we find attractive and there is no shame in that. The problem I have is with men not reading my profile and then messaging me asking me the same questions I answered in it already. It’s frustrating but not surprising. In real life men usually could care less what I have to say. It’s all about what’s in my jeans. At least on okcupid I don’t have to be harassed by these pervs in person. I simply hit delete. Also I think it’s about time it was acknowledged that good looking people have a tendency to be jerks. I can’t speak for men but women have to be in order to survive everyday without being raped in the street. Or they are used to having everything their way and being worshipped not for their character but for their looks. So who’s fault is that? I will say that I personally do not respond if there is not a photto. I wouldn’t consider myself a shallow person but if I have a photo I expect to see yours. Ok Cupid can experiment all they want to as long as they aren’t hurting anyone. If they have the data then maybe they could unlock the match making secret. Seriously I wouldn’t trust my best friend of 26 years to match me with someone.
I have no problem with the testing since they are using it to improve the site, and I found the results quite interesting. I think this is one of the best free dating sites because of the matching % as I find I have more in common with people over 80%. I have talked to several people people from this site and one is now one of my very best friends (92% match). I became a paid member so I could delete every profile within a 100 mile radius that answered three ethical question in a non-Biblical manner. Saved me a lot of time, but eliminated basically everyone I was attracted to in this area… Doh! Still looking for someone who is right for me and I for her, but I am hopeful that she’s out there.
This is an awesome article. As a statistician, I wish more companies published their research.
This was a completely manipulative, deceitful and terrible thing for you to do. I was just about to pay to upgrade to an A-list user but then saw the news online, NOT on OKCupid, alerting everyone that your site had done this. It was unethical in the extreme, and your claims that “everyone on the internet” does it, as a justification, makes it even more despicable.
I am no longer going to recommend OKCupid to my friends…in fact, there were some who I’d already tried to win over to using OKC, e.g from Match dot com, who I will now tell them they were right to not use OKC.
Sincerely,
Gurpreet S. Jawa, M.D.
For paying customers, I understand they can feel bad about those experiments….
It’s a modification of service you didn’t agree to, which may be legally ok (the contract usually cover those) but is morally wrong (this type of contract will probably be forbidden soon, especially in EU).
Non paying users, not so much. Ok Cupid do not really get anything so it is not bind by a contract…anyway, I personally like those experiments, the scientific approach without trying to hide or sugarcoat unpleasant or un-pc findings is why I’m here….that and it’s free…
Match percentage seems correct, I like the profiles with high percentage while the low one often shows dealbreakers…and a good profile (or, more important, good answers) is an asset…very small compared to hot pictures, but enough to make a difference when the pics are in the maybe region…