Show HN: My weekend project - Chessboard picture recognition

It seems like you can avoid having to recognize the pieces altogether if you are taking photos of each move during a match, since they have a known starting position.<p>Edit: what I mean is that in this case the problem becomes answering "does this place on the board have a piece on it" not "which piece is on this place on the board." Once you know which places on the board have any pieces whatsoever you can simply infer between moves which pieces moved where.

You might look into the academic literature for OCR, which this will reduce to. (Most AI problems are "How can I apply human intelligence to cleverly reduce this problem from something which appears to require intelligence to something which actually only requires well-understood math?")<p>I think the punchline is likely to be that you end up taking dot products of feature vectors generated by library code and writing fairly little chess-related code. Sorry to burst your bubble if you thought this was going to be very sexy. On the plus side, it will work, and if you're as good at programming as you appear to be you're almost certain to succeed on this.<p>P.S. I hope you like taking photos because odds are you're going to get the opportunity to take <i>lots</i>.

I don't know anything about "chess theory" but are there some squares that only certain pieces can get to? I don't think it's possible for a Knight to ever be on some squares, so maybe you could use logic like that to narrow down the piece detection.<p>It may be easier to just track the game state from the start instead of trying to identify the position of the board at a random snapshot in time.<p>Cool project though, seems like a good intro to image processing.

"For example, the program is able to determine whether each square is white or black by calculating its average color and comparing it to the black and white colors"<p>I bet you could get away with determining the color of a corner square and assuming the rest.

I think your best bet is to feed the the segmented chess board squares into a machine learning algorithm. You would have one which classifies squares as empty or not, and another which recognizes chess pieces. The difficult part is building up all the examples to do the training, picking the right algorithms to use (SVM, logistic regression, etc), and then transforming the raw data into the proper form for the algorithm.<p>I did find a very recent paper on a chess playing robot which has some good starting points: <a href="http://www.cs.washington.edu/ai/Mobile_Robotics/postscripts/chess-icra-11.pdf" rel="nofollow">http://www.cs.washington.edu/ai/Mobile_Robotics/postscripts/...</a>

Hmm, interesting project.<p>If you're ok with helping the computer out a bit, you could put a very small matrix bar code (think QR Code) on top of each piece. Since there are 6 chess pieces and 2 colors, you could have a code for each piece in just a 2x2 matrix (4 dots / bits).<p>Otherwise, it seems like it may be very difficult, as I'm not even sure of what some of the black on black pieces are myself (bottom right).

I have worked in machine vision projects in past. It's not going to be easy to recognize the pieces as is; I suggest that you use 2D pieces for chess. Even better, use some unique design like this one: <a href="http://www.chess.com/forum/view/general/experimental-chess-set-design" rel="nofollow">http://www.chess.com/forum/view/general/experimental-chess-s...</a>

I may have a good solution:<p>Take <i>two</i> photos. One at the starting position and one at the current position you'd like to save. You don't need to take them in order (take the current position, reset the board, and take the other photo). This way you know what the pieces are from the starting position, then use an algo to match based on that.<p>Also some good hints for most standard sets while checking your current position...both kings are always on the board and they are usually the tallest. there is never more than one queen per side, and its usually a close height to the king, but has a circular crown. Rooks are circular on top, bishops come to a point, if there are more than two pawns per side that gives them away. Knights could be singled out by eliminating the rest.<p>Good luck!

Very nicely done! Really appreciate the links to your reference material in the blog, saves me (and other noobs) from having to dig around and discover it ourselves. I'll definitely be keeping track of your progress. Good luck, dude!

Don't see the use of this as an app, but fuck the use. If the goal is fun and learning, I think it's worth going all in and do object recognition using, say, SIFT vectors (see Wikipedia). Just to see if it works.<p>It's easy to find SIFT implementations, but you'll have to do some work to construct a training set of images for each type of chess piece. An elegant solution will probably detect the type of piece independent of its color, or the color of the square.<p>This might be ridiculously easy, or not work at all.

yes..like the other hackers are saying....you have to deal with this problem like a poker player tries to guess his opponents cards....so the first step is to create the piece expectation function...which will basically calculate the probability that a piece is (say knight) at square x,y given the probabilty expectation functions of all the other squares..one way to create probablilities for all squares would be to use a combination of dynamic programming and neural networks...you will have to create a dynamic programming algorithm that sort of trains itself(by storing data rather than changing its code!!) do better and better as you tell it how right or wrong it was....so the topics you are looking to cover in the next exercise (assuming tht u have already done some reading on image processing) are basic probability theory,dynamic programming and neural nets...those are my two cents on the problem.

If one day, my program is able to recognize the pieces on this chessboard... I will happy:D <a href="http://2.bp.blogspot.com/-TgamS2dnAls/TjvpV3YuA0I/AAAAAAAAAI4/y6XESlITAN8/s1600/287621_2266996910302_1109935532_32714307_1964814_o%25281%2529.jpg" rel="nofollow">http://2.bp.blogspot.com/-TgamS2dnAls/TjvpV3YuA0I/AAAAAAAAAI...</a>

Here is a related project on turning an image of a Go board into a computer readable file understood by most GUIs for playing Go: <a href="http://www.inference.phy.cam.ac.uk/cjb/image2sgf.html" rel="nofollow">http://www.inference.phy.cam.ac.uk/cjb/image2sgf.html</a>

For figuring out chess pieces, maybe determining the light source and looking for light/dark patterns in certain areas? The problem is, from top view, even a human would have a difficult time figuring out if a piece was a bishop or pawn.

similar successful project of a robot playing chess naturally:<p><a href="http://www.youtube.com/watch?v=OZ0Hx04KFCg" rel="nofollow">http://www.youtube.com/watch?v=OZ0Hx04KFCg</a>