Minor point, but if you are using factual questions to filter out stupid people, asking for the "right" number of continents is probably not ideal. You (and I) might disagree with someone counting Eurasia (or North and South America) as a single continent, but it is not really an indicator of stupidity in the way that believing in astrology or thinking the earth is larger than the sun are.<p>According to the wikipedia article on continents, different models are taught in different countries, so you probably end up selecting for a different attribute than you thought.
Hi. I work at OkCupid.<p>I can't address all the particulars raised in this analysis, but the biggest is essentially correct:<p>> <i>The worst side effect of the current scoring system, is that a spammer could easily answer only the questions with obvious answers (basic facts and display of non-bigotry) and get a decently high match percentage with a lot of people. At which point, the spammer uploads a picture of an attractive guy/girl, writes some generic profile text, and scams away.</i><p>The algorithm as described in the FAQ does suffer from this problem. However, we have enhancements that address the issue very effectively. The FAQ is slightly out of date, and shouldn't be taken as a complete, exhaustive description of how we make matches.
> <i>The worst side effect of the current scoring system, is that a spammer could easily answer only the questions with obvious answers (basic facts and display of non-bigotry) and get a decently high match percentage with a lot of people.</i><p>That is exactly what I did which led to my meeting my (now long-term) girlfriend. I was receiving about 5 profile views/week with 500 questions answered. I scrapped them all, answered 20 or 30 questions with non-offensive answers, and skyrocketed to 60-100 profile views/week.
OKCupid would probably get better matches if it dropped the user submitted weights and ideal match preferences completely and instead used its database of people in relationships as training data for a proper machine learning algorithm.<p>The current approach is entirely oriented to give people what they think is important and what people think they want. It would probably be better to derive that from existing relationships (successes).<p>I would be a little surprised if people at OKCupid hadn't already thought about this. Whether there is actually any momentum to change the core matching mechanic or not remains to be seen.
As a person with very strong religious views I've found the matching algorithm to be great at filtering out those with incompatible views. I think matching based on ethical, political, and religious views is the matching system's strength. The true weakness of the algorithm is that it matches poorly for personality. Two people can have very similar beliefs but be a terrible match in terms of personality. I would prefer an entirely separate score for that aspect.
OKC has the worst matching algorithm on the Internet. I signed up for an account, spend a large amount of time to get my profile "100% complete", answered over 1,000 questions and was still consistently matched with people I had absolutlely no interest in dating at all. I can't count how many times I saw 0% compatibility scores, people hundreds or even thousands of miles from me(I'd specifically said no more than 30 from my zip code) and worst of all, several men(I'm not gay and didn't express and desire at all to meet men on the site). I deleted my profile after 6 months and when they asked me why I was leaving I told them how their matching process would be better using a random number generator. It's too bad because OKC is one of few free dating sites I know of that actually has a decent number of women using it.
I would have thought that OKC would use some dynamic weighting scheme where the weights are not constant, but depend on how commonly people answer that question. For example, questions that just about everyone lists as mandatory wouldn't be weighted with 250, but with some number proportionately reduced to reflect the banality of the question.<p>Hmmm, I'm a bit disappointed or maybe I'm just missing something.
I guess I should have read the FAAAQ. I had no idea "mandatory" was being applied to the matching algorithm in such a naive manner. That sucks.<p>Of course, it doesn't really matter for me, anyway. I don't use OkCupid as a dating site. I use it as a way to find things like local libertarians, programmers, and other essentially platonic things. In fact, for a few years, I haven't even used it for that -- but that's how I did use it, so dating criteria are kinda irrelevant, which means answering questions is kinda irrelevant too. (I've answered quite a few just for shits and giggles, though.)
I'd like to see an analysis between these two sets of data:<p>A) Dudes that have an opinion about the OKC algorithm<p>vs.<p>B) The amount of times they get laid each week<p>Jokes aside, I'm fascinated with this topic. For most of history, the likelihood of finding a mate was left to chance. Then OKC comes along and says it can leap past obstacles such as chance and geography to help you find your soul mate. That's an incredibly powerful idea.
They seem to have tweaked their algorithm after the Match acquisition. My ex, with whom I had about 65% match (she signed up after we broke up), suddenly one day became a 80% match. She claimed she hadn't answered any more questions, and neither had I.<p>I like OKC, but they don't do even basic filtering of profiles. If they just verified, say, a mobile phone, it would get rid of the vast majority of fake accounts.
Interesting maths analysis. I'd certainly noticed that their ability to reliably rank inside a window of say, 80% and 95% was rather low, along with the ability to filter out deal-breakers. Frankly that remained a manual process, no matter how much data people had supplied, I never could do that reliably from the numbers alone. As a liberal Christian, I end up with quite a bit of filtering of filtering out of people with very great differences in either religion or politics which the algorithms simply didn't pick up.<p>So anyway, some tweaking done. Let's see if it has any visible effect...
How about this: instead of showing you match %'s, they show you profiles and you rate them. Then use recommender systems [1,2] just like netflix. There are two immediate problems with this: you have the cold start problem [3] and also if you don't show random profiles, then you introduce bias into their rankings. The first problem could be solved with side information (you still answer questions) and the second with a nicely chosen exploration/exploitation trade off. PMF[4] for the win!<p>[1] <a href="http://spectrum.ieee.org/computing/software/the-million-dollar-programming-prize" rel="nofollow">http://spectrum.ieee.org/computing/software/the-million-doll...</a><p>[2] <a href="http://en.wikipedia.org/wiki/Collaborative_filtering" rel="nofollow">http://en.wikipedia.org/wiki/Collaborative_filtering</a><p>[3] <a href="http://dl.acm.org/citation.cfm?id=1352837" rel="nofollow">http://dl.acm.org/citation.cfm?id=1352837</a><p>[4] <a href="http://en.scientificcommons.org/42513739" rel="nofollow">http://en.scientificcommons.org/42513739</a>
This critique reminds me very strongly of Evan Miller's "How Not To Sort By Average Rating": <a href="http://www.evanmiller.org/how-not-to-sort-by-average-rating.html" rel="nofollow">http://www.evanmiller.org/how-not-to-sort-by-average-rating....</a><p>In both cases, the key observation is to realize that the <i>meaning</i> of the numbers is more important than the numbers (or how they're calculated, per se). This point is kind of abstract--especially compared to the salient examples in both pieces--but if you're looking to understand <i>why</i> this error is so common, look first to the fact that it's not an error of stupidity or insufficient maths.<p>You can sometimes avoid a big practical problem (e.g., an avenue to attack your system via diluting meaning ("spam")) by abstractly considering the meaning behind the structures available to you.
I suspect that "doug" in the articles comments has it right.
Users are happier to see dozens/hundreds of "matches" that they know aren't all that good despite saying 90% than if they saw 3 matches that were more correct followed by a huge drop down to 50-60%.
It is crazy to think that a matching algorithm based on answers to seemingly pointless questions can predict the success of a future relationship. There is no substitute for interpersonal connection in real life. Those looking for love and meaningful relationships should get off of their computers and meet people in person. This is the problem with the majority of dating sites out there-too much focus on the online experience of users rather than the offline interaction. The internet is a great way to network and meet new people but nothing can replace spending time with a person in real life to determine whether or not you will get along with them.
The other problem is that many of the questions are redundant, ambiguous, or otherwise poorly constructed. I end up skipping a lot of them just because of this.
I don't think matching algorithms will EVER work, short of human-level AI. The only matching algorithm that works is the human brain.<p>So, I think that the right way to go is to just increase the speed and accuracy with which a human can look for matches. Some ideas here: <a href="http://ideashower.posterous.com/idea-dating-site-slideshow-audio-voiceovers" rel="nofollow">http://ideashower.posterous.com/idea-dating-site-slideshow-a...</a>
This reminds me of the recent Dinosoar Comic. It seems like everyone thinks they can make a better dating site these days: "BaguettesAll4Me.co.uk is complete! ... I realized my perfect woman won't say 'ew that's weird' as she watches me eat a whole baguette"
<a href="http://www.qwantz.com/index.php?comic=2088" rel="nofollow">http://www.qwantz.com/index.php?comic=2088</a>
I've been a user of okcupid, on and off, for several years.<p>at this point, I dont really even look at scores anymore. I can tell based on profile text alone whether I'm likely to get along with a person.<p>but this is age and experience, not math.<p>I'd rather okcupid had a feature that tells me what things are going to cause fights between a person and myself after we date for a while :)
> At least they’re not using a non-linear Bayesian splitting tree didactogram<p>This seems like a weakness to me. I'd love to be able to choose between different algorithm for matching features and set some relevant coefficients. It'd be fun to see the profiles that have a minimum hamming distance from mine, or whatever.
it's probably not a big deal from their perspective, since 99% of people wouldn't realize to do that.<p>It's essentially good enough for them, since the mainstream audience wouldn't realize that, and if a few geeks figure out a way to cheat the system, then that's fine too, since they need all the help that they can get
Hmm, this does make me wonder about some people that I'm "almost perfectly" matched with. We potentially had too many mandatory questions matches, but a lot of the more subtle things got lost?
What's wrong with OKCupid's matching algorithm for me is that I can't get to use it at all! I just can't log in. I've been trying for the last 2 months, they just keep saying "Sorry, we're having technical difficulties right now. Check back later.". I have searched news for an explaination of a major OKCupid outage and found nothing. Is it just me?
Assuming OKC's target market is the relative majority in English-speaking countries it really doesn't take much crazy math to find matches, as in the matching facility is not going to be as important as the random number generator (or trained monkey) that is filling in the matches. All that really needs to be taken into account are features of the prospective match that the two individuals WANT to be taken into account(age/height/race/income level/political/religious). It isn't that complicated, especially since one of OKC's tactics is to use quizzes/'looking for friends' to break the stigma of online dating.<p>What is the stigma of online dating?<p>Obviously the implied answer is that if you are using online dating then you cannot find a date IRL. Or perhaps a kinder answer would be that you have too much going on to find a date IRL. This is the same principle as a meat-market nightclub/singles bar where the bouncer lets the guy with the two attractive women in, but the pack of forever-alones get locked out. Then the slightly wealthier gentleman pays $ to the nightclub to get in, for the opportunity to pay $ to one of the attractive women (in the form of a drink) for the chance to talk to her and (hopefully) get sex/affection/phone #.<p>Of course this is all crazy, but it is a way in which many bars/nightclubs/online dating works. If OKC can attract attractive women with quizzes/validation/'looking for friends' and associated kitsch, they have it made.
The only online dating site that was worth a moment's time was Facebook, and then they removed the search facility.<p>OKCupid was interesting, but not a magic bullet. It was then bought by Match, which consists primarily of marketing and poor website construction.<p>Online 'dating' is a social problem more than a computing problem and, as such, no good solution will ever exist. Anyone male thinking of sinking time into one would be better served by simply getting out more, taking classes, adopt a dog and walk it, etc.
Now a good way to single out the Hacker News Readers would be to identify males in the computer software industry who suddenly and significantly updated their profile questions shortly after November 23'rd 2011....would love to see some real data on how much action these users are getting!
What I hate about OKCupid is that all the girls I've met only wanted sex. I.e. we have sex, it's all fun, and then they leave saying to call them back whenever I feel like having fun again. It's just not serious.. I clearly checked "Long time relationship".