This is not a research paper. In fact, most ML papers aren't research papers. Compare the FB paper to the first result in biorxiv under the genetics heading [1]. There are basically no similarities other than being done in LaTeX. I never expect a research paper to talk about how the research affected business processes, but again this isn't research in any traditional sense.<p>What this is, is documentation of how Facebook implemented a technology stack that uses reinforcement learning techniques to do something. Namely: "Notifications at Facebook"<p>So what can other developers and business owners take from this? I don't see anything about the down stream product impacts. Does it impact conversion to paid rate for users? Does it reduce human labor? How does it improve benefits to users. All I see them write are two things:<p>"We observed a significant improvement in activity and mean-
ingful interactions by deploying an RL based policy for
certain types of notifications, replacing the previous system based on supervised learning."<p>I'm sorry but there is absolutely nothing rigorous in that statement. How are "meaningful interactions" defined? Hopefully they aren't still arguing the formula (more interaction = makes users better off).<p>"After deploying the DQN model, we were able to improve daily, weekly, and monthly metrics without sacrificing notification quality."<p>Improve for who? Well obviously Facebook and how much activity people have. Not necessarily if the user is actually getting more value from it.<p>What's the Return on Investment for this system?<p>Listen, I'm a huge fan of being open with business practices, research etc...I'm also obsessive about RL and making progress in the field.<p>What I can't stand however is lack of rigorous and tangible proof of how we're making things better for users or the society broadly with RL yet, or even in most cases getting positive ROI for the effort we're putting into ML/DL.<p>I've built these tools at scale so it hurts to say this, but the economics just aren't yet lining up here across the entire ML/DL industry and that has me worried that another AI winter is coming.<p>[1]<a href="https://www.biorxiv.org/content/early/2018/11/01/422345" rel="nofollow">https://www.biorxiv.org/content/early/2018/11/01/422345</a>