I was working on a web frontend for fawkes (https://pypi.org/project/fawkes/) based on the research outlined here (https://github.com/Shawn-Shan/fawkes) which claims to be able to use image level perturbations to protect against AI image recognition systems. From the paper, the following claim is made:<p>"Our cloaking technique is not designed to fool a tracker who uses similarity matching. However, we believe our cloaking technique should still be effective against Amazon Rekognition, since cloaks create a feature space separation between original and cloaked images that should result in low similarity scores between them."<p>I was in the process of verifying this when I found that _no such guarantee_ can be made using fawkes. I documented my experiment here: https://github.com/Shawn-Shan/fawkes/issues/125.<p>Using Azure Face and AWS Rekognition, perturbed images regardless of the settings resulted in high degrees of similarity. In addition, I did a separate test using a perturbed image and another image with the same subject which _still resulted in a high degree of similarity_.<p>While the README claims "Python scripts that can test the protection effectiveness will be ready shortly.", no such scripts have been provided.<p>According to the fawkes website, their application has been downloaded over 300k times. If the protection they claim to provide is ineffective, then there are _a lot of people_ exposed out there with a misguided sense of being protected. This situation is further complicated by the apparent legitimacy of this tool due to its presentation at Usenix 2020 and its inclusion in pypi.<p>My question is this - has anyone actually validated the efficacy of this algorithm and have reproducible results to share? Or is fawkes yet another example of AI snake oil?
I think that generative adversarial networks make the method ineffective. During the training of a GAN, the same type of distortions are applied to its training data to force it to generalize.