Information geometry is a very interesting research topic. In essence it allows to define a metric (and therefore a measure of distance) between probability distributions.<p>As a results, it has a lot of practical application in machine learning and has been use successfully for classification in Neuroscience, Radar signal processing and computer vision.<p>we can also note that Information Geometry can be seen as a sub-field of Riemannian Geometry, with some equivalence between metric. For example, the cannonical metric for symetric and positive definite (SPD) matrices in Riemannian geometry is actually equivalent to the metric for multivariate normal distribution obtained with Information geometry.<p>For some application, IG is very efficient. it has been used for multivariate time-series for classification of EEG signal and was at the center of the winning solution of 3 kaggle challenges : <a href="https://github.com/alexandrebarachant" rel="nofollow">https://github.com/alexandrebarachant</a>
Skilling wrote a nice critique of information geometry that's worth reading:<p><a href="http://djafari.free.fr/MaxEnt2014/papers/Tutorial4_paper.pdf" rel="nofollow">http://djafari.free.fr/MaxEnt2014/papers/Tutorial4_paper.pdf</a><p>The main point is that KL-divergence is not a metric, so imagining it as a distance in a space may give you some wrong intuitions. Its matrix of 2nd derivatives, the Fischer Information, works as a local metric, but then many people want to draw global pictures that try to extend this back to a global metric, which doesn't actually work.
Not an expert in the field. But I liked Centsov's theorem (when a metric comes from IG in the discrete case). I have not found a similar theorem for the general case. Amari's book is hard to follow. There is a serious lack of pedagogical intro to the subject, something like a starter : The main ideas + achievements of the theory + how to use it. There is something in the theory very deep but Amari just scratches the surface.<p>Can you imagine if GR metrics come from IG?
Much of this is due to Amari, who got really into merging theoretical neuroscience and IG, e.g. <a href="http://www.mitpressjournals.org/doi/abs/10.1162/08997660260293238#.WF4QE7YrKAw" rel="nofollow">http://www.mitpressjournals.org/doi/abs/10.1162/089976602602...</a>
Some interesting discussion here, particularly the refs in (6): <a href="http://mathoverflow.net/questions/215984/research-situation-in-the-field-of-information-geometry" rel="nofollow">http://mathoverflow.net/questions/215984/research-situation-...</a>
From out of left field, or thereabouts, see also pseudometric space and "Geometry of Logic": <a href="http://finitegeometry.org/sc/16/logic.html" rel="nofollow">http://finitegeometry.org/sc/16/logic.html</a> ...<p>While there seems like a potential well-balanced in-between to these complementing/contrasting philosophies and layers of view points, I feel partly well-suited to vent that from what I've seen, statistical data analysis seems to zealously want to gain understanding by using brute force, where self-ordering "shapes" simply want to flow which shows their Nature.