I poked around a bit on 538 on the day it relaunched, but hadn't read anything from there since then. Is this representative of the quality of articles? It seems just embarrassing as an example of data-driven journalism. They are evaluating the quality of a "big data" prediction algorithm based on 32 samples. The experimental setup is very odd, with such a short period of price tracking, a bizarre baseline strategy, and ending the trials early (over half right at the beginning!).<p>Or consider this gem: "For the five routes in which there was a financial benefit to waiting, Kayak successfully reduced my fare in each instance. [...] This is a sure sign of intelligence". Yes, of course if you cherry-pick 1/6 of the trials (a whole 5 of them!), the system will look effective. What kind of value is there in "analysis" of this level?
I regularly book from Kayak, and I like their predictor.<p>A sensitivity setting would be nice. For example, when it says the fare is likely to increase in the next few days, does that mean increase by $40 or increase by $140? I'm more likely to give up a probable saving of $40 to be comfortable in knowing my flight is finally booked, but I would wait if it's $140. I imagine everyone's tolerance is different, which is why the sensitivity setting would be nice.
This analysis ignores the fact that it might be beneficial to buy tickets more than 14 days before your trip (an idea which always seemed like common sense to me). The idea that fares rise monotonically for the last two weeks doesn't mean that big data is dead.
tl;dr version<p>You will probably not save any money following Kayak’s price prediction algorithm and instead buying tickets two weeks ahead of your scheduled departure is a reasonable strategy. You should use prediction algorithms anyway, because they wont lose you money either and help you gain "piece of mind".
If Kayak/MSFT uses machine learning for price prediction and airlines are doing the same for predicting their revenue, doesn't it result in both sides quickly reacting to each other moves and countering opponent's actions? After all, they're all using the same ticket sales data. Maybe that's why recommendation engines don't seem to work?
Not included in the cost calculations: several days of remembering to check the cost of the air fare every day, and stressing about whether or not you'll end up paying a ton when the predictor erroneously tells you to wait. The price differentials cited in this article are well worth just buying it and having it out of the way, IMHO.
<i>Because following the algorithm isn’t going to cost you more money, and it might actually relieve some of the second-guessing that occurs when you’re left to your own devices.</i><p>Disagree. Airfare pricing is adversarial. It used to be "common knowledge" that buying tickets on Wednesday at 3:05am Eastern was optimal. Then, airlines responded by jacking up prices at that time. You'll also get different prices from an Incognito browser than one with cookies. I'd bet you get hosed if you use your iPhone, for obvious reasons.<p>If something becomes commonly known as "the right way" to get fair prices, it will surely lead to unfair prices as airlines take advantage of the herd.<p>Because it's adversarial, the landscape is constantly changing and there are no guarantees.