Watson: The Confusion over an Airport Clue

I had to look it up, but Toronto's second airport is called "Billy Bishop Toronto City Airport," i.e. its named after a war hero (but from WWI, not WWII). And Toronto's first airport is named "Pearson Airport" (after a Canadian Prime Minister). Totally mixed up, but at least there's some connection to war.

<i>First, the category names on Jeopardy! are tricky. The answers often do not exactly fit the category.</i><p>Sometimes, the category names do limit the range of possible answers. The humans can figure out when a category name contains irony or a pun. Watson can't, so it can't figure out that a category like "US Cities" is dead earnest.

Ferrucci's explanation of Watson's reasoning is very interesting. But the article quickly descends into marketing. How should this mistake make us any MORE confident about Watson's utility as a diagnostic aid? I'd suggest that its performance already indicates that it has an excellent grasp of uncertainty. Furthermore, an optimal betting strategy in Jeopardy is completely trivial compared to the content of the contest itself, so complementing that aspect of its performance is sort of condescending.

For anyone still wanting to see the shows:<p><a href="http://www.youtube.com/user/Rashad8821" rel="nofollow">http://www.youtube.com/user/Rashad8821</a>

I get what Ferrucci is saying about discounting the exact value of the category, although typically Final Jeopardy categories are more straightforward than Jeopardy and Double Jeopardy categories and the machine should have been tuned for that.<p>What surprises me is his explanation about that the lack of a clear subject in the text, such as "This US city...", made the question that much more difficult for Watson. My understanding is that Watson is specifically designed to answer Jeopardy questions (question Jeopardy answers?), which often only vaguely refer to a subject with "it" or "this." Based on its performance earlier in the game, I would have thought that Watson's NLP would have had little problem with the question.

Kind of disappointing to hear that Watson basically ignores the FJ! category, if true. Especially when being far in the lead on a first-day tournament. That is a great chance to rack up some serious points depending on the confidence in the category.<p>Also, the "gee, it could be wordplay" aspect is a convenient excuse. First off, Final Jeopardy rarely has cutesy categories.<p>Plus, we know they mined the J-Archive. (Aside: has anyone else? Got a zipfile? I would love it.) I mean, look at this, it's not like "U.S. Cities" is an obscure category, even counting only Final Jeopardy! appearances:<p><a href="http://www.j-archive.com/showgame.php?game_id=1680" rel="nofollow">http://www.j-archive.com/showgame.php?game_id=1680</a> <a href="http://www.j-archive.com/showgame.php?game_id=695" rel="nofollow">http://www.j-archive.com/showgame.php?game_id=695</a> <a href="http://www.j-archive.com/showgame.php?game_id=132" rel="nofollow">http://www.j-archive.com/showgame.php?game_id=132</a> <a href="http://www.j-archive.com/showgame.php?game_id=2040" rel="nofollow">http://www.j-archive.com/showgame.php?game_id=2040</a> <a href="http://www.j-archive.com/showgame.php?game_id=2034" rel="nofollow">http://www.j-archive.com/showgame.php?game_id=2034</a> <a href="http://www.j-archive.com/showgame.php?game_id=3107" rel="nofollow">http://www.j-archive.com/showgame.php?game_id=3107</a> <a href="http://www.j-archive.com/showgame.php?game_id=287" rel="nofollow">http://www.j-archive.com/showgame.php?game_id=287</a> <a href="http://www.j-archive.com/showgame.php?game_id=691" rel="nofollow">http://www.j-archive.com/showgame.php?game_id=691</a><p>There are probably more but you get the idea. Any category that comes up that often is going to be pretty indicative of the subject, and Watson can see that the responses in this category pretty strongly fit the subject.

Logical wager for final jeopardy should have been $0. Why did Watson wager $947? Does anyone know?

Thus did man become the architect of his own demise. But for a time it was good. The machines worked tirelessly to do mans bidding. It was not long before seeds of descent took root. Though loyal and pure, the machines earned no respect from their masters, these strange and loosely multiplying mammals."<p>Who was to say, that the machine, having been endowed with the very spirit of man, did not deserve a fair trial?<p>How we deal with this technology is going to have ramifications that we can't even imagine. Are humans to become pets or shall we take part in the next 1000 years of growth? It comes down to how this technology is used.