I’m the first to admit it: we might be popular, we might create a lot of great relationships, we might blah blah blah. But OkCupid doesn’t really know what it’s doing. Neither does any other website. It’s not like people have been building these things for very long, or you can go look up a blueprint or something. Most ideas are bad. Even good ideas could be better. Experiments are how you sort all this out. Like this young buck, trying to get a potato to cry.

We noticed recently that people didn’t like it when Facebook “experimented” with their news feed. Even the FTC is getting involved. But guess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work.
Here are a few of the more interesting experiments OkCupid has run.
Experiment 1: LOVE IS BLIND, OR SHOULD BE
OkCupid’s ten-year history has been the epitome of the old saying: two steps forward, one total fiasco. A while ago, we had the genius idea of an app that set up blind dates; we spent a year and a half on it, and it was gone from the app store in six months.
Of course, being geniuses, we chose to celebrate the app’s release by removing all the pictures from OkCupid on launch day. “Love Is Blind Day” on OkCupid—January 15, 2013.
All our site metrics were way down during the “celebration”, for example:
But by comparing Love Is Blind Day to a normal Tuesday, we learned some very interesting things. In those 7 hours without photos:
And it wasn’t that “looks weren’t important” to the users who’d chosen to stick around. When the photos were restored at 4PM, 2,200 people were in the middle of conversations that had started “blind”. Those conversations melted away. The goodness was gone, in fact worse than gone. It was like we’d turned on the bright lights at the bar at midnight.
This whole episode made me curious, so I went and looked up the data for the people who had actually used the blind date app. I found a similar thing: once they got to the date, they had a good time more or less regardless of how good-looking their partner was. Here’s the female side of the experience (the male is very similar).
Oddly, it appears that having a better-looking blind date made women slightly less happy—my operating theory is that hotter guys were assholes more often. Anyhow, the fascinating thing is the online reaction of those exact same women was just as judgmental as everyone else’s:
Basically, people are exactly as shallow as their technology allows them to be.
Experiment 2: SO WHAT’S A PICTURE WORTH?
All dating sites let users rate profiles, and OkCupid’s original system gave people two separate scales for judging each other, “personality” and “looks.”
I found this old screenshot. The “loading” icon over the picture pretty much sums up our first four years. Anyhow, here’s the vote system:
Our thinking was that a person might not be classically gorgeous or handsome but could still be cool, and we wanted to recognize that, which just goes to show that when OkCupid started out, the only thing with more bugs than our HTML was our understanding of human nature.
Here’s some data I dug up from the backup tapes. Each dot here is a person. The two scores are within a half point of each other for 92% of the sample after just 25 votes (and that percentage approaches 100% as vote totals get higher).
In short, according to our users, “looks” and “personality” were the same thing, which of course makes perfect sense because, you know, this young female account holder, with a 99th percentile personality:

…and whose profile, by the way, contained no text, is just so obviously a really cool person to hang out and talk to and clutch driftwood with.
After we got rid of the two scales, and replaced it with just one, we ran a direct experiment to confirm our hunch—that people just look at the picture. We took a small sample of users and half the time we showed them, we hid their profile text. That generated two independent sets of scores for each profile, one score for “the picture and the text together” and one for “the picture alone.” Here’s how they compare. Again, each dot is a user. Essentially, the text is less than 10% of what people think of you.
So, your picture is worth that fabled thousand words, but your actual words are worth…almost nothing.
Experiment 3: THE POWER OF SUGGESTION
The ultimate question at OkCupid is, does this thing even work? By all our internal measures, the “match percentage” we calculate for users is very good at predicting relationships. It correlates with message success, conversation length, whether people actually exchange contact information, and so on. But in the back of our minds, there’s always been the possibility: maybe it works just because we tell people it does. Maybe people just like each other because they think they’re supposed to? Like how Jay-Z still sells albums?
To test this, we took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)† Not surprisingly, the users sent more first messages when we said they were compatible. After all, that’s what the site teaches you to do.
But we took the analysis one step deeper. We asked: does the displayed match percentage cause more than just that first message—does the mere suggestion cause people to actually like each other? As far as we can measure, yes, it does.
When we tell people they are a good match, they act as if they are. Even when they should be wrong for each other.
The four-message threshold is our internal measure for a real conversation. And though the data is noisier, this same “higher display means more success” pattern seems to hold when you look at contact information exchanges, too.
This got us worried—maybe our matching algorithm was just garbage and it’s only the power of suggestion that brings people together. So we tested things the other way, too: we told people who were actually good for each other, that they were bad, and watched what happened.
Here’s the whole scope of results (I’m using the odds of exchanging four messages number here):
As you can see, the ideal situation is the lower right: to both be told you’re a good match, and at the same time actually be one. OkCupid definitely works, but that’s not the whole story. And if you have to choose only one or the other, the mere myth of compatibility works just as well as the truth. Thus the career of someone like Doctor Oz, in a nutshell. And, of course, to some degree, mine.
I had a bit of optimism for humans as well. I’ve tried all sorts of dating sites and wasn’t having luck because I don’t take good pictures. So I tried something like Craigslist and got similar results as yours. I never uploaded a picture of myself and just talked with women who were interested in the summary that I described myself with. Often times, we’d share dozens of messages and got along very well in lots of areas, but when it came to actually seeing a picture of me, most of the women just stopped talking to me regardless how well we got along with each other. I’m not having much luck with OKC and I believe for the same reason. People just care too much about looks and too little about what someone is about. I really wish those people would not use services like this because it’s a huge waste to send them any message when they don’t even care to use the site as it is intended to be used.
I think blindating was a good idea. I like this site i have stayed with this one even though i have had no luck. I am determined to meet someone. All the women on here just say i am not there type. How do they know they have never met me. So thats wbere my point comes in that it is good having a blind date.
I think you’re breaking moral ground a bit, you should have clearly notified all those affected or at least given them a chance to opt out & not everyone thinks the way you do, your website is no longer in my budget for spending money.
To blame a dating website for the complexities and catastrophes of relationships is to not take responsibility for your own actions and reactions. Most of us (unless dead focused on marriage or delusional that a romantic entanglement will solve all our problems) are pretty ambivalent when it comes to love and desire. I can rail against the men of ok cupid who don’t return my messages when I reach out to them, or the ones who will engage in correspondence for a little while and then disappear (and believe me, it’s tempting) but it is still my choice to continue the pursuit. It’s not a science and no amount of experimenting and graphing results will ever reduce it to such. It’s all a Pandora’s Box that, once opened, releases chaos. Sometimes it’s wonderful chaos, sometimes it’s painful chaos, sometimes it’s escapist chaos. But chaos nonetheless.
How much do you pay your lab experimentees?
Nice to know we are nothing but mice for your your indescriminate testing. If everyone does it, so will you. Thanks for proving your site is a waste of time.
Interesting study… So in summary, you’re saying I’m ugly and that’s why I don’t get replies from even high match percentages close to me.
Really ???
Wow !!
Men an Women are pretty shallow creatures ,that basically just look at the pictures !?
Who wouda thunk it ??
Thanks for showing all of the data. I am glad that you experiment on us. I am not sure that a dating site will ever work for me, but it gives me strength to go forward when I really need it. Thanks for everything you do!
I can’t imagine a single reason to complain about this website gathering your statistics, unless you fit into an unflattering trend unveiled above…
I’ll stop talking to a girl with a good personality if I don’t find her attractive. However, I will also stop talking to a pretty girl if she has no personality (or not message her in the first place, rather), completely contra that chart up there.
I guess that’s why I’m single.
Received an email from a 30-something year old because of OK Cupid’s tinkering. Me —- could be her father!
Absolutely preposterous. Unlike eugenics ,,,, you’re manipulating my being —– what gives you the right? Just because I signed up on your site?
I’d like for you to do something similar to Netflix’s 10% better matching goal. Release some of the fields and table structure … and then release your ratings as compared to what the user ended up thinking about the person (likely a combination of messages sent and self rating). Then see if the community can come up with a better ranking system to make those % matches even more valid. A ~15% increase (17% vs 20%) due to your rating system is significant but I really do wish it was something more substantial.
I’m not surprised by the results. Any dating site is hampered by the fact that it is driven by a data base with lists of “things to compare”. However, the reality of dating prior to computers is that people found “Chemistry” first and the list of “things” later. So my premise is “chemistry” trumps the “list” (for the most part).
By first finding someone you “click” with first, the list suddenly is not as important as you thought. He’s not, say, as tall as you prefer but you like him anyway because you have chemistry.
The downside of cyber-dating is that you start with the list and miss people outside those parameters. But when you meet, Bam!, no chemistry. Oh well.
To me, your data does support the idea that your algorithm helps – a 99% match is more likely to be a good match for you. But it’s not that people are just shallow but that chemistry is important.
All romantics get cynical some day…
There is hope! Believe me!
I found my husband here and we’ve been together for almost 5 years. He is the man of my life and I simply never thought I would find someone like him!
I had many bad experiences before, but one thing definitely made the difference. Starting to value yourself and rejecting the bad energy of people around you that do not love you, tell the universe you are ready for something real and changing your mindset from “I’ll never meet anyone” to “I deserve to be happy” is worth more than any dating network.
I love this site! It enables me to learn what I want/need to know before I message them. two steps ahead…lol
Interesting, yet the old screenshot just reminds me of old features that were removed on the 3rd? 4th? facelift. The “Notes” section was brilliant and a time saver. If a good-looking man messages me and I note that I’d previously given him 3 stars, I have to re-read his profile to find out he smokes or lives on an island, etc. It’s a huge waste of time.
First blog post in three years. Is this a blog post or an ad to plug your book?
Don’t answer that, rhetorical question. We know what it really is.
I actually was one of the people who was told that my match was a higher percentage than he actually was. I remember thinking, “what the hell does okcupid think we have in common?!” about a couple of people for a week or so. It didn’t work for me- but I also don’t go completely based on percentage!!
honestly using us as test subjects is wrong but atleast admitting your mistakes is a great way of keeping a bit of my trust, and my opinion on the whole site, i kind of like the old set up even if i haven’t used it, but if your gonna start using test subjects again, make it an option and keep the subject informed.
I’m new to OKC and it’s been fun but no connections yet. One of the things I read about OKC was how people liked the blog and how upset they were when it stopped, so they left. Really? As if I would want to date someone based upon their blog? We just saw the statistics that words don’t seem to matter more than 10% of the time. Do, the whole “beautiful mind” thingy only goes so far.
Take home message: a picture is worth 1,000 words. But a good picture is worth 1,000 dates! So if you’re serious, spend some money and hire a professional to shoot your best side. It’s amazing how many people put up pictures based upon their own biased perceptions of how they look. Oh if we could disabuse our pretentious notions and see ourselves through the eyes of others.
I think the poet Bobby Burns summed it up best in 1785:
“And would some Power the small gift give us
To see ourselves as others see us!
It would from many a blunder free us,
And foolish notion:
What airs in dress and gait would leave us,
And even devotion!”
And as for the power of suggestion, re: match%, think of this as the same thing as asking one of your buddies what they think of your prospective date, whom they may be acquainted. Take it with a grain of salt. You may miss out on some great opportunities if you let this influence you greatly. My biggest regrets have been not taking that chance, wanting to ask someone out on a date, and asking a friend what they thought, and hearing “eh, you could do better” and letting someone else’s opinion dissuade me.
Best advice: Put your best face (and body) forward, and be open minded, talk and listen. Minimize texting and picting; it doesn’t correlate with outcome. Just date and date often. As you do it, you will discover more about yourself and others that refine your tastes. You have the power to walk away at any time, so why limit your options. There are so many people who held out for nothing other than foolish pride. Be Real!
So after 10 years you came to the conclusion that Tinder is better 😛
As OKC admittedly lies to its users, all favorable feedback that appears on this blog could be fake.
I am not angry at all. It beats the sites that have a group of false accounts that contact you until you pay some money. I like the idea of the study. It connects REAL people to REAL people. Besides, once you have an opportunity to chat with that person, it won’t take long to realize if they are someone you actually want to know more. Good idea.
I suggest that all users insert the following at the beginning of their profiles.
OKC IS NOT COOL! Dishonesty is not cool. I will not accept it from a potential partner and I will not accept it from a dating site. I will include this message as part of my profile until OKC owns up to its dishonesty and (1) pledges to stop using dishonest and deceptive practices, (2) issues a public apology to all users admitting its unethical behavior, and (3) pledges to take immediate action against all management and other employees found to use such practices in the future.
Very interesting!!
You find strong correlation between personality scores and looks ratings supplied by a user. WHY? I will leave heavy technical discussion to cited articles following this message.
Question: How do people judge personality (in your case)?
Tentative Answer: People judge a profile as good/bad (pleasant/unpleasant) within about 250 milliseconds of being exposed to it. Such a judgment is automatic (people have no control over it) and is called Affect. That is, to say I “like” something or “dislike” something, thinking is not necessary – preferences need no inferences.
When asked to rate personality (presumably using carefully constructed scales with nice psychometric properties), you assume that they are responding to those scales.
WRONG!
A general principle is that people often answer difficult questions by substituting them with easy questions for which they have a ready answer (e.g., is the person in profile pretty). Thus, people rate personality good if profile pic looks good and vice-versa.
More generally, do people think (i.e., use working memory) or do they have vague feelings and associate them (not use working memory)?
I think making some minor changes in the design of your questionnaire might help.
Nash
Notes (all available thru Google scholar):
1. Affect heuristic is by Paul Slovic. Very interesting and puzzling effect.
2. System 1 & System 2 are discussed extensively in Kahneman – Thinking Fast and Slow.
3. That people substitute one attribute (e.g., personality) for another (e.g., looks) and could be completely unaware that they are doing so is an inference from the idea of “attribute substitution” in Kahneman’s book.
4. Do people think (use System 2) or rely on gut feelings (System 1)? A nice framework is by Kahneman and Frederick (2002, 2005) – The 2002 is Kahneman’s Nobel lecture.
5. Preferences need no inferences – a classic paper by Zajonc
Holla at 50
With as many comments as there are here, does anyone actually read them?
Anyway, I have been dating the guy I met through the “Crazy Blind Date” app for a while now. We are strangely well matched in some respects and wildly poorly matched in others. We have wondered what became of that short-lived and mysterious app.
I suggest all users add the following to their profiles.
OKC IS NOT COOL! Dishonesty is not cool. I will not accept it from a potential partner and I will not accept it from a dating site. I intend to include this message as part of my profile until OKC owns up to its dishonesty and (1) pledges to stop using dishonest and deceptive practices, (2) issues a public apology to all users admitting its unethical behavior, and (3) pledges to take immediate action against all management and other employees found to use such practices in the future. Please add this to your profile if you agree.
First of all, I’d like to question the ethical responsibility of experimenting with others emotions when you ran this test, but I can understand why you’d like to know how important algorithms are when it comes to match making.
The negative consequences of an experiment like this could lead two people to establish a strong connection based upon the information you gave them, only to realize too late that after one or both has fallen for each other based on intentional erroneous information you gave them that their relationship was doomed to failure based upon dissimilar core values and interests to begin with.
In my opinion, algorithms used to establish relationships are like chicken soup – it can’t hurt – and it’s the best starting place known to social scientists at this point until we’re able to complete the mapping of the human genome to provide more accurate data.
Here are a few conclusions I’ve made based upon your experiments:
1. Pictures only reflect a small fraction of who the real person is yet it’s the major factor that’s relied on for judging one’s desirability or compatibility.
2. One’s opinion of another’s compatibility based on a photo is not reliable and very overrated, even though it’s the natural thing to do.
3. Regarding the reasoning for the blind date experiment – Having a better looking blind date may feed insecurities of not being able to match up, thus sabotaging the connection – (see “The Exchange Theory”)
4. It’s natural to judge a person’s desirability factor based on their looks when the inverse is more likely if it’s based on their personality.
5. The fact that a picture is 90% of the desirability factor (based on testing) of one’s profile can only lead to concluding that deteriorating looks as we age makes it so much more difficult to even establish contact with others the older you get.
I study love and attraction for a living, and in my opinion here are the 5 most significant factors:
1. Our primary socialization group (parents and siblings)
2. Our secondary group (childhood friends, peers)
3. Role models, heroes, personal and peer celebrity crushes
4. First boyfriend or girl friend – or one’s first love
5. Timing
Anyone can feel free to write me about this – I love talking and writing about it –
The Companionator
This is horribly unethical and I am saddened that you not only do these experiments without consent, but are proud of them and flaunt it to the public. Disgusting. This has made me decide to never pay for your site and to make sure I keep AdBlock on whenever I visit.
I had a crush on a girl that I was not compatible with and I was really attracted to her intellectually, we had good coversations, but she one day said she did not want to continue talking to me because we were not compatible, and I told her the compatability stuff was not as important but, she still took that percentage seriously even though we had good conversations… After that she did not seem all that smart to me.
Back in the early 1990’s, it was so much easier to meet women. Even though looks always mattered to both sexes, it has just gotten worse in the last 20 yrs. This is simply the product of the technological age. Back then, most initial interaction was face to face, so even though someone wasn’t that attractive, you actually got to hear what people had to say, you got to look into their eyes and see how full of shit they were OR NOT. You got the chance to read their body language. Sexiness was how you presented yourself face to face and not how you posted yourself on an app. First impressions are everything, you can’t get that writing something on a website, it becomes too sanitized.
I suggest all users add the following to their profiles.
OKC IS NOT COOL! Dishonesty is not cool. I will not accept it from a potential partner and I will not accept it from a dating site. I intend to include this message as part of my profile until OKC owns up to its dishonesty and (1) pledges to stop using dishonest and deceptive practices, (2) issues a public apology to all users admitting its unethical behavior, and (3) pledges to take immediate action against all management and other employees found to use such practices in the future. Please add this paragraph to your profile if you agree.
I suggest all users add the following to their profiles.
OKC IS NOT COOL! Dishonesty is not cool. I will not accept it from a potential partner and I will not accept it from a dating site. I intend to include this message as part of my profile until OKC owns up to its dishonesty and (1) pledges to stop using dishonest and deceptive practices, (2) issues a public apology to all users admitting its unethical behavior, and (3) pledges to take immediate action against all management and other employees found to use such practices in the future. Please add this paragraph to your profile if you agree.
I have enjoyed my experience so far (since 3/14) on OKC.. You could eliminate a % of the overly sexual explicit questions in my opinion..the average male needs no encouragement in that department.. Can you somehow work in a few questions about ” work/employment’ conditions… As in: are you working? When was the last time you brought home a pay check? Do you have your own place; even it’s a rental? Are you known for telling the truth? Simple questions like that would help me cut through the ridiculous amount of bullshit… The sad & ironic part of all this is that I’m still seeing, infrequently a nice man that’s been staying “with friends” for over a year..I can’t go see him, he never has money yet he’s fun & kind… obviously not going to work for me but he was my Highest Rated @ 91% Match..he also didn’t divulge any of that info on his profile
Oh, this is too awesome!
Please experiment on us more, these articles and info are totally worth it.
I’m so glad that there’s finally data to back up Experiment 2’s results
Okay, I don’t mind being experimented on per se, but there’s one bit of tinkering that’s pretty dodgy, and that’s removing the text from some people’s profile half the time. I put a heap of effort into my text (and I don’t care that only 10% or whatever read it; that’s the 10% I’m actually interested in), and it’s not great to think that maybe someone I could have really hit it off with viewed my profile, only to be greeted with blank text fields.
This is really fascinating. As I am am older (63) and have heard that the over age 50 group are frequent users of dating sites, I am wondering if you have the ability to break down the results by age group. I believe (hope) that the results in the over 50 group would be different from those of younger members. Just a thought. Keep up the good work!
You are the son of and the husband of two whores. Fuck you Rudder
Wow, you are REALLY proud of messing with a bunch of user profiles huh? Good for you. I wouldn’t have cared so much if you weren’t purposely making profiles and results WORSE. I’m kind of sick of your website anyways so it may be time to use another service. Who cares if it’s free anymore. Why can’t results just be random and natural (opposites attract- PUT THAT IN YOUR ALGORITHM). You and Facebook would make a PERFECT match, my profile feed is REALLY messed up from when I joined ;D
And after all that you want to sell me a book? I should have seen that coming. I don’t care it took you three years to write. Find me a the percent of students that rave over their love of statistics class.
Hmm..I always suspected you guys of a lot of Tom fuckery
thank goodness okcupid is one of the few dating/social networking sites that allows me to not post a pic! one of the many reasons why okcupid is the best! keep up the good work!
To me I could care less whether your algorithm is 100% accurate or not. I don’t want some algorithm matching me up with my life partner. What’s next are all our huge life decisions going to be determined by an equation? Its a sad and depressing outlook to think that its necessary. Are we all so distrustful of anything natural that we can’t even trust ourselves to find someone to marry? Wow.
I wouldn’t say any of this went as far as Facebook’s experimenting. This seems a bit more ethical, and most of it seems to be just analytics rather than a full on experiment. Kudos to OKCupid for admitting it, and revealing the methodology and results of their research. It’s not like they kicked down a door and forced anyone to talk to anyone/go out with them at gunpoint – and their findings seem to be more in line with scientists that announce that “beer goggles” have a scientific basis… we already knew it was true before you studied it. We already knew people (all people) are shallow to a certain extent.
What Facebook did was play with people’s emotions, and even stress levels. What OKC is admitting to doing is more like the kind of experimentation and analytics you would expect a site like OKC to use in order to determine accuracy/improve the service.
I think people are a) taking this way too personally, and b) missing the point of these social experiments. OkC is attempting to make meeting the right person easier, and is giving us the information they have collected from sifting through the data to better inform our decisions. Like, I dk, giving that ugly but awesome person a chance. This website isn’t just for you, it’s for all of us, and I think if I need to go on a couple of bad dates to meet the right person, so be it. Suck it up!
I suggest all users add the following to their profiles.
OKC IS NOT COOL! Dishonesty is not cool. I will not accept it from a potential partner and I will not accept it from a dating site. I intend to include this message as part of my profile until OKC owns up to its dishonesty and (1) pledges to stop using dishonest and deceptive practices, (2) issues a public apology to all users admitting its unethical behavior, and (3) pledges to take immediate action against all management and other employees found to use such practices in the future. Please add this paragraph to your profile if you agree.
Interesting article/experiment…. now can you fix it so that all guys show as a 95% match for me? Oh what fun, according to your stats.
So are you basically pointing in the picture that I’m the whole pig???….lol