I’m the first to admit it: we might be popular, we might create a lot of great relationships, we might blah blah blah. But OkCupid doesn’t really know what it’s doing. Neither does any other website. It’s not like people have been building these things for very long, or you can go look up a blueprint or something. Most ideas are bad. Even good ideas could be better. Experiments are how you sort all this out. Like this young buck, trying to get a potato to cry.

We noticed recently that people didn’t like it when Facebook “experimented” with their news feed. Even the FTC is getting involved. But guess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work.
Here are a few of the more interesting experiments OkCupid has run.
Experiment 1: LOVE IS BLIND, OR SHOULD BE
OkCupid’s ten-year history has been the epitome of the old saying: two steps forward, one total fiasco. A while ago, we had the genius idea of an app that set up blind dates; we spent a year and a half on it, and it was gone from the app store in six months.
Of course, being geniuses, we chose to celebrate the app’s release by removing all the pictures from OkCupid on launch day. “Love Is Blind Day” on OkCupid—January 15, 2013.
All our site metrics were way down during the “celebration”, for example:
But by comparing Love Is Blind Day to a normal Tuesday, we learned some very interesting things. In those 7 hours without photos:
And it wasn’t that “looks weren’t important” to the users who’d chosen to stick around. When the photos were restored at 4PM, 2,200 people were in the middle of conversations that had started “blind”. Those conversations melted away. The goodness was gone, in fact worse than gone. It was like we’d turned on the bright lights at the bar at midnight.
This whole episode made me curious, so I went and looked up the data for the people who had actually used the blind date app. I found a similar thing: once they got to the date, they had a good time more or less regardless of how good-looking their partner was. Here’s the female side of the experience (the male is very similar).
Oddly, it appears that having a better-looking blind date made women slightly less happy—my operating theory is that hotter guys were assholes more often. Anyhow, the fascinating thing is the online reaction of those exact same women was just as judgmental as everyone else’s:
Basically, people are exactly as shallow as their technology allows them to be.
Experiment 2: SO WHAT’S A PICTURE WORTH?
All dating sites let users rate profiles, and OkCupid’s original system gave people two separate scales for judging each other, “personality” and “looks.”
I found this old screenshot. The “loading” icon over the picture pretty much sums up our first four years. Anyhow, here’s the vote system:
Our thinking was that a person might not be classically gorgeous or handsome but could still be cool, and we wanted to recognize that, which just goes to show that when OkCupid started out, the only thing with more bugs than our HTML was our understanding of human nature.
Here’s some data I dug up from the backup tapes. Each dot here is a person. The two scores are within a half point of each other for 92% of the sample after just 25 votes (and that percentage approaches 100% as vote totals get higher).
In short, according to our users, “looks” and “personality” were the same thing, which of course makes perfect sense because, you know, this young female account holder, with a 99th percentile personality:

…and whose profile, by the way, contained no text, is just so obviously a really cool person to hang out and talk to and clutch driftwood with.
After we got rid of the two scales, and replaced it with just one, we ran a direct experiment to confirm our hunch—that people just look at the picture. We took a small sample of users and half the time we showed them, we hid their profile text. That generated two independent sets of scores for each profile, one score for “the picture and the text together” and one for “the picture alone.” Here’s how they compare. Again, each dot is a user. Essentially, the text is less than 10% of what people think of you.
So, your picture is worth that fabled thousand words, but your actual words are worth…almost nothing.
Experiment 3: THE POWER OF SUGGESTION
The ultimate question at OkCupid is, does this thing even work? By all our internal measures, the “match percentage” we calculate for users is very good at predicting relationships. It correlates with message success, conversation length, whether people actually exchange contact information, and so on. But in the back of our minds, there’s always been the possibility: maybe it works just because we tell people it does. Maybe people just like each other because they think they’re supposed to? Like how Jay-Z still sells albums?
To test this, we took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)† Not surprisingly, the users sent more first messages when we said they were compatible. After all, that’s what the site teaches you to do.
But we took the analysis one step deeper. We asked: does the displayed match percentage cause more than just that first message—does the mere suggestion cause people to actually like each other? As far as we can measure, yes, it does.
When we tell people they are a good match, they act as if they are. Even when they should be wrong for each other.
The four-message threshold is our internal measure for a real conversation. And though the data is noisier, this same “higher display means more success” pattern seems to hold when you look at contact information exchanges, too.
This got us worried—maybe our matching algorithm was just garbage and it’s only the power of suggestion that brings people together. So we tested things the other way, too: we told people who were actually good for each other, that they were bad, and watched what happened.
Here’s the whole scope of results (I’m using the odds of exchanging four messages number here):
As you can see, the ideal situation is the lower right: to both be told you’re a good match, and at the same time actually be one. OkCupid definitely works, but that’s not the whole story. And if you have to choose only one or the other, the mere myth of compatibility works just as well as the truth. Thus the career of someone like Doctor Oz, in a nutshell. And, of course, to some degree, mine.
Everyone keeps complaining that OK Cupid’s matches are crap and that they don’t appreciate being part of an experiment. That is all fine and good, but the experiment took place in 2012 and the “participants” were notified of the correct compatibility scores afterwards.
So, most of you were not part of the experiment or you would have been aware of it.
You are a day late and a dollar short on sending out this email. It is old news. What is sad is that Rudd admitted the whole “formula” magic was a joke. Your technique is nothing more than late night home shopping network type. Fortunately for me I never put much stock in OKC. Nor any money. I let the advertisers do it for me. What is pathetic is your magic ” algorithm cannot even add. “79% friend, 34% enemy. Ugh, shouldn’ t it all equate to a grand sum of 100% overall? The only saving grace here is maybe, just maybe OKC will lose a couple bucks immediately and most hopefully see a dip in their net growth moving forward.
Very entertaining. Not sure it helps to weed out the people looking to scam you! sex, money, and your time. It’s a crap shoot. lol.
I do applaud your effort to find answers and to give us a good laugh.
I agree with wood tables.esp. statement. Match me up…lol…
Caught the NPR interview. Nicely done!
This ‘experiment’ you’re talking about is very boring to read. Your data (that I skimmed) is just dumb. Who cares about that crap. Online ‘dating’ is overrated and you should not need the Internet to find love. You use the Internet to find a piece of ass for the night. Okcupid lied to me when signing up. I remember it saying that there are lots and lots of good looking people on this site…..liars. Here’s a tip…go to a grocery store to find someone. Or a park…or a pub…or the beach…or the..you get my point. Grow a pair and say HI for fuck sakes. Some will find ‘love’ on this ugly people site but most of the users will not.
Interesting experiment. I don’t use the % friend/enemy to decide whether to contact a person. I find the men’s answers to my questions to be a better indicator of compatibility. In fact, I think that is the one thing that makes OKC stand out. The profile is also important, but I realize that writing may not be someone’s strong suit.
Yes, physical attraction is part of chemistry, however, he could be the most handsome person in the world, but if he believes murdering people is okay, his profile is blocked.
Geez, I sure hope people are honest when they answer those questions…
It is a shame and it is even more shame that you are proud of it (and even send newsletters for it).
Shame on you people who think you can play with fellow humans just because you provided a free service. It is disgusting.
And I say that as a scientist who works with data …
I find physical attraction a must for any good relationship. If you can not even see yourself holding the guys hand..It’s going NOWHERE..& I do read their profiles & try to find something we have in common.Then contact him..I hope this helps you. CC Ride ER’ Cher
I have msg fifty guy on here and not yet found a match 7 has ask me for money and every body from the US are in the army or Africa only one or two in my area and only want sex .l want to met a nice guy and have a relationship with .
I just wanted to thank you for the data info. With the recent online news about the experiment I thought maybe it was done for a cynical reason. But I understand why now why it was done. I can’t say there was nothing new we don’t all already know. I would have been interested to know, besides looks how race played a roll in the message response percentage.
Experiment #3 only says that the first 4 online messages do not cancel the assumptions we make based on the match score. Well, they shouldn’t. A match score comes from hundreds of questions, after all. It goes deeper than a 4-message exchange. Give your users some credit, OKC. And give yourselves some credit, too: You’ve come up with a good scoring model.
I’m glad I’m not part of the shallow end of Internet dating…that experiment wouldn’t have worked with me. Unless you message to me either exhibits some reference to my profile, I don’t answer. The match up percentage means nothing to me. Your pic won’t affect my decision either. If I wanted to meet people based on looks, I could do that anywhere. Maybe if more people thought like that, there would be more successful relationships…on and off line.
You guys suck
hahahahaha
What joke!!
We do not even know who we are, imagining that virtual reality
is true, that anything written in the profiles is what really is.
Hey, Bill you are a genius : nobody knows what ‘is’ is.
They think that they”ve been experimenting on us. Haha, guys
WE ARE just a bunch of illusions, we are not real human beings.
Lets see what happens
I’ve talked to lots of girls and have gone on a couple of dates already ! Its all how u talk to people to get a reaction from another person , saying just hi doesn’t do anything lol ! Okcupid dose half the battle. the girls are there u just gotta talk to them ! Good pictures and talking will get u in the door ….
This is pretty despicable and will cost you a lot of users and credibility…way to lower our trust in your website!
All, please read and post concurrence if you agree…some points below:
– the reason you are likely admitting this, is because it was/would be leaked out anyway, just like all the dirty politicians who deny until caught in the act.
– when we signed up for the service, we agreed to certain procedures and methods per our understanding; this was changed midway without our knowledge, a breach of trust.
– this probably/surely has cost some people a meaningful relationship, and that’s very disgraceful…most people couldn’t sleep at night if they knew they did that to others!
– sadly, it’s easy to see your results didn’t really amount to anything substantial, even by your admission; you would have much more data if you simply asked each user why they accepted or rejected the last match or contact!!!
Now, some unrelated peeves…
While we realize this is a free site mainly, with the better perks (filter settings, notifications, etc) reserved for payers, it is still crappy to withhold matches from users based on fees. How does a paying woman feel knowing she is likely not shown the one or two men she is compatible with, because they aren’t paying members? (p.s. this would not be the site most people pay for anyway!)
Why is there no short distance setting (5 or 10 miles) for searches and such? You insist on sending matches that are 30 or 40 minutes away; a lot of people will not drive that far for a first meeting!
Finally, we hope this comment, and other critical ones, will help you improve the service and methods enough to win back some trust and users over the next months.
Regards,
former user
i support experimenting. it is necessary to bring better products. Thank You OkCupid
I think a good thing to try out is to reward or entice ladies to be apart of the matching process VS window shopping for the right guy. I also suggest rewarding men for proper date selection vs the “amount” of traffic on their profile or others profiles. I see a lot of one-sidedness where it is the man that is supposed to post himself bench pressing a car while achieving a doctorate degree and also submit hundreds upon thousands of hello’s and adds in hopes that a woman responds. Men tend to shotgun all adds in hopes to get one response with no regard of thinking or properly choosing their mate. On the flip side, women post a pretty picture with no words and expect to “fish” guys out as they chose. They truly window-shop without giving average man a chance to show his above average qualities. Women also don’t “try” to communicate unless there is something that can be gained or stands out which is unfair to the guys who just need to be given a face-to-face chance. It is old fashioned thinking on a platform that should advocate equal rights. OK cupid needs to formulate ways for increasing female participation and limit men’s shotgun methods. They should also limit the adds that receive all the attention forcing both sexes to look away from the adds that are so perfect that they are most likely scams.
You do realize that people sometimes live and die based on whether they can find love, right? You do realize that there are people who are really isolated who depend on sites like this to connect with someone, anyone? Would you be so smugly self- satisfied if you knew half a dozen people blew their brains out because OKC led them to believe there was no hope? Here’s an idea: how about you stop playing games and actually have the site be as most people expect— a place to find love and happiness.
I use the match percentage as a “first cut”. There are so many women that are at 85% or more I don’t feel the need to mine any deeper than that. I do occasionally find someone with a high match percentage that seems to be an awful match after reading their profile, but I don’t go looking for great profiles with low match percentages. I’ve gotten 3 long-term relationships (1 yr, 2 yrs, and 4yrs) from OKC, all with women with 90% match percentages and higher.
I don’t rate anyone. I can only speak for myself, and I don’t see how my rating on someone would be relevant to anyone but me.
I don’t feel particularly upset about the experiments involving lying to users about match percentages. You have to have some way of validating their accuracy and value.
I appreciate your removing the “mandatory” category for questions, as they were never truly “mandatory”. I would have preferred you actually making them work as I had expected, but this is an acceptable fallback position.
I wish that there was a way to rate questions, or at least down-vote them with a comment. Some of them offer answers that are too simplistic, or contain errors. If enough people down-vote them and the comments describe a relevant concern, the question could be deleted and possibly reworked.
Experimentation is necessary for accurate data mining and analytics. The question on OKC: Are you a dog or cat person? Well, I like dogs, I don’t own one because I don’t want to clean up their hair or come home early to let them out, but I would rescue one from a dangerous situation by a roadway and adopt it off. So does that make me a dog person or not? The answer is that when you let amateurs write questions for analytics the data integrity is compromised. I’m all for the OKC community writing their own questions. I think that if you want more accurate results you have to come up with a way to dig deeper into people’s lives. Make it a great day!
Not much an issue with me about the pics (I wasn’t on OKC back then anyway), as I do tend to read the profiles first before clicking on the pics.
What does make me want to cancel my membership is when I click on a Woman Seeking Men profile and the first thing I read is, “Hello, I’m a pre-op t-girl looking for that special guy”. They need to have their own category or something. I DO NOT want to meet anyone with an X & Y chromosome…
Just sayin…
When you feel contained on certain system, pressure group or power instance of any kind socially speaking, you need to find a gate that releases all this huge amount of energy; so for me appears to be means of expression but also balance and that’s when and image or an iconic reference like a photographical picture supports better and entire blog of contents at the end what part of yoouir consciousness needs to be tested? The semantical grammatical exercise of tip being words aimed on any plase? Or your personal ID credentials? letting know the audience that you aren’t a delinquent or an Illegal
Awesome article guys!
We’ll never really know what works and what doesn’t…
But then again, who cares?
Welcome back! Just so you know, I’ve been checking the blog monthly JUST IN CASE. Glad to know that it wasn’t (entirely) all for naught.
Would prefer to have a place to leave a post to warn and inform people of verified scammers, either on their profile or on a seperate reference section.
Any chance of that??
I really like using OkC and I think it’s a great website. I have been on a handful of successful dates here and I truly believe I will find the perfect girl here, if not in person first. I Love the match %, the locals swiping, the match suggestions, the profile layout, etc. If that means being the occasional guinea pig for things, I don’t mind because I know it will overall make the site better and will therefore get me 1 step closer to finding my girl. But, I personally hated the 1 day blind date thing. I understand that it was getting people to try something new and to promote your blind date site, but for those that don’t like blind dates, there was no opt out and it was frustrating. It’s hard not to sound shallow, but the truth is that we’re using an online medium to meet complete strangers which makes it already hard enough to establish that initial chemistry like you do in person-to-person contact. Then, the pictures are taken away which gets rid of any way to at least sort through the people you might feel at least an initial physical attraction. It’s not all about looks but honestly, ask yourself this: How many of these profiles that say “I like to laugh, travel, and hang out with friends. Ask if you want to know more. 6 things I can’t live without: air, water, iPhone, coffee, sleep, food. Message me if you’re interested.” can you read without at least having some sort of way to eliminate some of the possibilities. It’s like reading resumes in an HR department with no way to sort through the ones you know would not work right off the bat. At least the pictures can eliminate people that aren’t your type or at least add something valuable they didn’t mention in their writing: she has a cute dog and it looks like I see a guitar in the background, I wonder if she plays.
Anyway, I like using OkC but there are a couple things that would be nice to see (keep in mind I’m speaking for the OkC iPhone app)
1) For the “Locals” swiping, I’d like to see easier access to profiles that you Liked so you can read their profile and possibly message them. I found a few profiles with high match % that I was attracted to but could not find them on the searches / match. Opportunity lost for me, especially if the other person doesn’t participate in the swiping which could have lead to a potential Match and Visit Profile click.
2) More search results for the OkC app. Lots of times I see the same people for weeks in my search then get a message from a nice girl that I never can across in my searches.
3) I really only use Quick Match because it gives me access to profiles I haven’t seen before, but once you find someone you like, it would be nice to be able to also message them (since you can already view their profiles and pictures). Why aren’t these people showing up in other searches? It would also be nice to not have to use a completely separate module to just rate them. I’d ideally like to see these people in just regular searches, but if we are keeping the ratings, can we just merge them into just the people’s profiles?
4) I’d like to see some of the above mentioned features merged so that there’s a) easier initial messaging for people you like or swipe, b) more search results from matches, searches, locals, nearby, and Quick Matches, c) more search filters, d) easier to rate the people you like so you can indicate interest similar to a Like on Facebook. I just feel like I’m using 4-5 different OkC modules to find people because each one had different results but with different restrictions (inability to message them or visit their profile, or inability to rate, etc)
Honestly though, awesome job and this is definitely one of the better sites out there especially since it’s also free. I hope you consider these points.
Great writing. Extremely entertaining and informative. The pervasive self-deprecating style is absolutely refreshing!
Cool to read, and I like the humility
thanks for the verbal explanations for us “non-numbers” people.
Removing all my info from ok Cupid. Too experienced for these games!
Right… Geeks love to mess with the system. Tweak till meltdown. All that. But HEY! Kids, your are screwing with real people’s lives here! So cool it! The match% was always dodgey, even when you weren’t messing with it. Maybe the algorithm can be improved. But, that would require people to actually be honest about who, what, and where they are. How likely is that? That’s why other “Scientific” sites give you Psychological tests with questions designed by someone other than college sophomores. For all the good it does them!
Thing is, it ends up being the “questions” that let me decide if I will make contact or not, but then, I only red the questions from matches with a high match score. Usually over 90%. Then I go to the details. I will contact people with high match, even if they don’t have a pic., and will shy away from pics which are obviously photoshopped, or “too good to be true”, e.g. someone much younger than the age stated. Sure, we all want to look our best. But kids, don’t you think your date might, when that inevitable 1st date comes, actually NOTICE that you are many years older and many pounds heavier than your photo? Don’t want to seem shallow here, but I would question your credibility and personal integrity.
One thing I might suggest is that you let people rate the “questions” as to being appropriate. That would leg you lose the real stinkers, and maybe tweak the rest so that they allowed something other than ” How often do you beat up your girlfriend?” Don’t want to blow the “Truth or Dare” party atmosphere, but please think about it. For myself, most of the time I wish there was an “all / none of the above” choice. Instead, I find that mmkst every response has some sort of explanation, which I imagine 90% of the people either don’t read, or don’t understand. But, that is a filter of sorts…
Anyway, thanks for the great site. With all its warts, it is possibly the best dating site on the net.
So I Wang you very bad boys and girls to put on a funny hat and go stand in the corner. Because you ha e been VERY bad!
Great article, but I would like to know how the percentages are figured? Even the profiles say they are 90% match are not really a match at all.
I think we should have a speed dating night with guys or gals who looking for each other . There should be based on each other’s profile.
For example I like guys who are asian, Hispanic, indian or mixed guys.
Since I haves joined I have two dates and several responses but no one really wants to go on a date..
Sometimes it just frustrates me!
Thanks
John
Long live discreet human experimentation!
This fails to say if any of the findings are statistically significant. Or what the sample sizes were. Or anything to indicate that this is actually good research.
Thanks for letting us know! I’m canceling my account.
The matching part or picture part is not the whole problem it is the truthful ness that I look for. Pictures can be deceiving. Some people look better in person. You have a lot of guys on here who are not who they say they are & that should be a #1 thing to deal with
I still have not got laid from this site…!!
But maybe I will today
I know one thing for sure..
prostitutes Are cheaper than dates
And without all the bs of there baggage
Bull backtalk..
Shame on you people who think you can play with fellow humans just because you provided a free service. It is disgusting.
And I say that as a scientist who works with data.
You have no shame. I will close my account and do something else with my energy.
Hahaha! I met my now husband in 2012 so we may have been a part of this “experiment”. I would have never in a million years thought I would marry someone like him but boy AM I GLAD I DID!!!!
He is the most caring and thoughtful guy!!!
I just realized… in the Terms and Services or whatever agreement did we all agree to this??? Oh… there is no terms of services agreement…
So, have we been included in the study? People paying for the service instead received false information for a psych experiment they did not agree to.
Unethical and PROBABLY illegal.
Basically everyone on this website can now (or possibly before) SUE OKC for a number of reasons now.
Bad idea guys. Bad idea.
All this ‘intelligent’ analysis just to point out that it is the looks that matters? Everyone knows this is true everywhere. But in the real world, looks while still a major factor, is but one of several attributes, ranging from subliminal, chemical to physical and social! If you really wanna prove your statistical brilliance do an analysis in real life! OKC or any goddamn online site, on its own, will majority of the times only get you superficial success but true stories from start to end happen in the real world!
You a punk with situational ethics and your company sucks. Why not just admit you are an ass leading a bunch of asses and get on with it. They say that Congress is lower than pond scum, but you are much, much lower. Lie, cheat and steal is all you know how to do. You would sell your grandmother for a nickel. Go suck an egg.
I have missed these blogs! Good to see you writing more. Keep it up! Experiment away!
-Pat
Looking at the results shared in the experiment 3, I wonder if you are going to build a new feature in which you allow people to chose to see the Match score or keep it anonymous. E.g.: If I want to see my match score with others, I need to enable it (and then everyone can see their match score with me), however, if I don’t want others to see my score (while I am okay that I don’t see there score) then I should be able to toggle the switch. Worth the experiment?
I’ve been on OKCupid since it began on The Spark as a fake dating site…I was wondering if you’ve recreated the Date My Sister experiment in recent times?
What gives you the right to experiment on your users without their consent?! You… suck!
It bothers me when I message a girl and she massages me then checks out my profile and I never hear from her again. That’s probably the worst feeling it’s like they saw my face and said ewww
I really don’t like being lied to. Does anyone? This amounts to a deceptive practice, hidden in fine print and not defined as to what they are going to do. No matter the reasons…just ask your self, would you experiment on your pet dog? (Lets see if rover likes chocolate, what will happen), your wife? (Humm…lets see if her friend or sister are just as good in bed….and the her! Wonder what could happen), maybe your child? (Here lil Timmy, take this stick and wake that hornets nest what might happen).
In your tests, you might find people don’t like to be.
subjects of anothers curiosity.
time wasted.
Paired to a complete (imo) looser.
Paired to more scam and phishing attempts.
Missed oppertunities.
time wasted…oh did I say that already.
I do admit, I did once meet someone from here in person. WORST MATCH EVER.
she was taller then me (I am 6 foot 2), she was a bleeding heart liberal who was a head strong femi-natzi! She did not want kids (which I do), there were so many markers, I was thinking she lied, just in an attempt to trap a guy. NOW I know it was the “NOT-O.k.Cupid” experiment.
Really wasted my time with this site.
Remember, rule to go by: OPPOSITES ATTRACT, BEFORE THEY ATTACK.
just ask the people in Israel and Gaza.
With the deception such as this, my trust is broken. And this is a biggy for me. In my past I was cheated on and lied to in 2 relationships, therefore trust, is something that once lost, CAN NOT be fixed with a happ-hazard, half ass explanation. Saying “oh-hey, we experiment on our members for shits and giggles”. This dies not make it better.
If you want to experiment on people (mess with their feeling, time, and emotions)…then go to a college…take a few class and learn what you need to there. I am sure plenty of the college kids will love to be in a test group.
Good luck on your tests…I for an willing to go it with other sites.
Wonder if this was what they were looking for or expected out of fucking with people.