><i>California mental health officials have been in talks with two startups about developing a digital system that identifies when a smartphone user is about to have an emotional crisis, characterizing it as a “fire alarm,”</i><p>This whole concept of predictive software for mental health (or otherwise) is a new level of disconcerting.<p>In an era where both of the following are true, we should not be pursuing these wide-spread predictive applications:<p>- Inability (or simply not caring) of companies / agencies to properly secure their data.<p>- Systemic inaccuracies present in an infantile field<p>We're struggling to hold companies accountable for the data they lose. Some of the biggest companies and agencies in the world are having data breaches (or inappropriate access to data, etc.) We are only just exploring how to identify and remove bias from predictive software. And yet, they want to roll this out?<p>><i>But it’s still unclear how effective an algorithm can be for someone at their most distressed, and in the meantime, the distressed are providing a tech company with its most desired asset—a wealth of deeply intimate data.</i><p>That paragraph, at least, sums up how I am feeling.<p>Noble goal, I think. Terrifying execution.