<i>"The neural network had learned to distinguish cloudy days from sunny days, instead of distinguishing camouflaged tanks from empty forest."</i><p>Reminds me of a story and ex-colleague once told me. He'd been working on a project to use neural nets to detect overcrowding on subway platforms using CCTV feeds. The NN worked with the training images but failed when run on live images. After a while they realised that training images showing crowding had been taken in a different season to those showing uncrowded platforms, and the NN was actually detecting the different type of clothes worn by the waiting passengers.
Looks useful for image problems. My personal interest is using CNNs for natural language processing. The same sort of tool that allowed viewing parts of the network as the sliding window on input text moves, etc. might be useful.