This paper extends a lot of the ideas in "A Unified Approach to Interpreting Model Predictions" (reference 19). The term unified is also used in the OP paper.<p>That reference paper is unifying in the sense that it could tie together a lot of frameworks such as LIME, QII, and others.<p>The paper in OP is unifying in the sense of the explanation games/game formulations (while also referencing QII).<p>The perspective of using contrastive explanations is very intuitive and becoming more common in different aspects of ML. My only wish is that I could try this FAE stuff myself. The authors of SHAP have a pretty immediately accessible python library. Could not find the one for this novel framework. It is not that I doubt the results. My point is a lot of good ideas like the ones presented in the paper often don't take off unless the authors also give easily pip-able libraries.<p>edit: I found the code[1], but it is not an easily conda-able library.<p>[1]: <a href="https://github.com/fiddler-labs/the-explanation-game-supplemental" rel="nofollow">https://github.com/fiddler-labs/the-explanation-game-supplem...</a>
This paper is neat!<p>More broadly, I am very concerned by how difficult it is to actually meaningfully run model explainability techniques on black-box models in the context of text/images in the real world. Most people are not AI researchers and do not want to deal with the hassle of trying random research code off of github (which usually is chalk full of bugs)<p>the ELI5 library implements LIME really well for text, and I have not found an equivalent with SHAP that has all of the same functionality (highlighting most important words and exporting in pretty HTML being the big one).<p>It's even worse when you start moving towards the neural network gradient explanations. There are libraries like Captum except in practice it fails on all the big-azz transformer language models I try to train. I'm still not sure as to how to run any of these gradient explanation methods on pytorch transformer models. Even if they do run, they don't display saliency maps with as much detail as the ELI5 library does...<p>There's so much good research in explainable AI, and so much engineering work which remains to be done! FFS I had to hack the hell out of ELI5/LIME to get it to work on clustering (and I still don't know why explainable clustering is not really done, and why I couldn't find an off-the-shelf solution for this.)
At Fiddler.AI, we're working on cutting-edge Explainable AI Algorithms, here is our latest publication that was recently awarded the Best Paper award at CD-Make 2020. #WeekendReading #ExplainableAI #ShapleyValues<p>Please let us know if you have any thoughts or feedback.