This is not only a problem with recipe sites, but with most user input. <a href="http://youtube-global.blogspot.com/2009/09/five-stars-dominate-ratings.html" rel="nofollow">http://youtube-global.blogspot.com/2009/09/five-stars-domina...</a><p>YouTube noticed a similar issue.
Getting users to rate on a star rating just isn't effective. You may have more luck with a simple 'yes/no' option. Would you make this again? And offer the opportunity for comments.<p>I find the comments in the reviews on epicurious to be very influential in how I follow the recipe, and if I cook it at all.
"So how does the user actually find the best recipe for a particular dish?"<p>Users may have better luck finding quality recipes by going to curated sites Cooks Illustrated rather than mob-rule sites like RecipeZaar. As with social sites (Digg, Reddit, HN), the chemistry of the community and the influence of the admins can have a powerful effect on the quality of the content. It also depends on what you're looking for. If all you want is cheesy enchiladas, recipes from the popular sites will work just fine.<p>There are many sites (not to mention books and periodicals) which have thought and care put into their recipes, often with supplementary material which helps you understand what you're doing and become a better cook, rather than just an ingredient-assembling automaton.<p>Another aspect to consider is that many people like to read (or watch) things about food but don't actually cook much. That's why the Food Network has become so popular (ever notice how all their commercials are for ready-made products?). A pet peeve of mine is to see 5-star reviews with a comment like, "Looks delicious! Will try making it sometime." Pretty annoying, but some people just like to watch. Maybe that's why recipes with pictures get higher ratings.
Yes, online recipe ratings are broken, which is why I don't pay attention to them. Of course, I'm a fairly good cook, so I know how to look at a recipe and tell whether it makes sense or not. Often, I'll end up with 60% of one recipe, 15% of another, 10% of a third, and 15% my own experience to pull together a recipe for a dish I want to try to make.<p>If you're looking for something on how to make a dish (like I did recently for baguettes) and find several recipes to choose from, I find a few things make it easy to choose from:<p>1. Real, individual ingredients are better than composite ingredients (that is, flour, baking soda, oil, etc. rather than Bisquick™).
2. Simpler recipes are usually better than complex recipes, especially if you've never made the dish before. There are exceptions, of course. (I'm <i>certain</i> that the King Arthur baguette recipe that I found would be better than the one that I tried, but it requires making a sourdough starter some fourteen hours in advance. I will try it some day.)
3. Well-written descriptions suggest better recipes. This is a bit like code comments, though—there is such a thing as too much description.
4. Recipes that depend on exact timing of multiple pots and pans are probably not a good idea to experiment with if you're not already good at making similar dishes.<p>That said, I'd pay attention to recipe ratings if there were only two yes/no questions: I made this; I'd make it again. Report the resulting value as a simple ratio of (repeat)/(made once). Seriously, I don't make recipes more than once if I wasn't enthusiastic about the results or weren't convinced that I could make it better by changing the ingredients or cooking instructions better. I don't know anyone who does. Star ratings are a waste of time.
I'd suggest that a bell curve in this case isn't a particularly good goal. Most people who cook already know what collection of flavours they're likely to enjoy and will make more dishes that combine these flavours in interesting ways.<p>A good example is dishes containing coriander, a large number of people hate it with a passion while others love it. People who hate it won't make the dish and won't review it. Those who love coriander will make it and most likely enjoy it.<p>Applying statistical analysis with the goal of getting a bell curve isn't the correct direction to reason in.
I don't think recipes themselves can really be rated; we probably have to rate the person who created the recipe or the person evaluating the recipe. Let me explain:<p>The number of people that come back and rate a recipe is probably small. And how many people who had a middling experience with the recipe would make the effort to come back and rate it? Or even a good experience, for that matter? A bad one? You bet they're going to rate it.<p>So right there, you have all kinds of bias problems with the data. And just to make it more fun, how many of those people rating the recipes prepared it properly? How many substituted ingredients that just completely ruined the dish? You see this all the time on these sites - "This recipe is horrible. I vomited for 3 hours straight. BTW I didn't have any red onions so I used pickled cocktail cherries instead."<p>Somehow you have to find a more indirect way of evaluating a recipe's worth. Maybe it could be a measure of how many times a person comes back to look at it (on the theory that each hit is presumably them coming back to look up that recipe again)? Or perhaps how many people chose to make it.
Recipes are interesting in that most reviewers actually alter the recipe, making star reviews misleading.<p>So someone might post a bad recipe and have other members alter it and rate it highly after their own modifications. You can only figure this out by reading the comments.<p>So some kind of yes / no on whether you have modified the recipe is pretty essential.
Slight tangent: anyone built a recipe site that works by presenting a common item (chicken, flour, milk, etc) with "Got this? Yep or Nup" and so on to build a list of search terms?<p>CookThing seems similar I guess, but not quite the same.
I do not use the sites listed in the article, but I have found that Epicurious (<a href="http://www.epicurious.com/" rel="nofollow">http://www.epicurious.com/</a>) has plenty of bad recipes that are rated as such. I have had good success following the ratings and checking the comments on the recipes before trying them.<p>Of course, with any crowd sourced system it is good to take the time to really read through who you are getting your trusted advice from. I still remember reading a review of a Korean restaurant on yelp that had two or three one star reviews because "they did not have sushi".
There is a factor at work in this recipe ratings systems that is a common bane to all rating systems which present a user with a range of choices. One commenter (pedalpete) noted already that people gravitate high or low. But the larger issue is the number of people that will gravitate toward the current average rating.<p>This is a natural desire to conform, and it isn't always conscious. If a user perceives that most people have rated this item X, he will naturally want to rate it X, and will even re-evaluate his judgment based upon the ratings of others.<p>Of course, for an online-rating system with instant feedback, this gives the earliest ratings the highest weight, immediately skewing all the results which follow.<p>But what effect does the average rating have on a person who disagrees? They will naturally tend toward the opposite extreme. They want to make their disagreement obvious. If the average rating is high, the negative rater will gravitate toward the lowest ratings, when in fact he might have gone 2 or 3 stars had he had no such prior feedback.<p>This is a rather intractable problem, since about the only way to prevent this sort of feedback would be to hide the rating until the person has actually tried the item in question and rated it themselves. But that, of course, makes a rating system useless.
Another thing you can't do with recipe ratings is prove the the rater actually made the item in question. At least on shopping sites, you can say "jrockway bought this item on 5/19/2009", so you have some idea whether or not the person has any clue.