AI-designed camera only records objects of interest while being blind to others

Something really interesting here if you read the title and comments first is that this is an <i>optical</i> thing. It&#x27;s not software running on the camera, it&#x27;s physical.

This is how the &quot;violet cusps&quot; from The Eyes of the Overworld were made..<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;The_Eyes_of_the_Overworld" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;The_Eyes_of_the_Overworld</a><p>&quot;There, Cugel finds two bizarre villages, one occupied by wearers of the magic violet lenses, the other by peasants who work on behalf of the lens-wearers, in hopes of being promoted to their ranks. The lenses cause their wearers to see, not their squalid surroundings, but the Overworld, a vastly superior version of reality where a hut is a palace, gruel is a magnificent feast, and peasant women are princesses — &quot;seeing the world through rose-colored glasses&quot; on a grand scale.&quot;

The paper: To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects<p><a href="https:&#x2F;&#x2F;elight.springeropen.com&#x2F;articles&#x2F;10.1186&#x2F;s43593-022-00021-3" rel="nofollow">https:&#x2F;&#x2F;elight.springeropen.com&#x2F;articles&#x2F;10.1186&#x2F;s43593-022-...</a>

&quot;Privacy protection is a growing concern in the digital era, with machine vision techniques widely used throughout public and private settings. Existing methods address this growing problem by, e.g., encrypting camera images or obscuring&#x2F;blurring the imaged information through digital algorithms. Here, we demonstrate a camera design that performs class-specific imaging of target objects with instantaneous all-optical erasure of other classes of objects. This diffractive camera consists of transmissive surfaces structured using deep learning to perform selective imaging of target classes of objects positioned at its input field-of-view. After their fabrication, the thin diffractive layers collectively perform optical mode filtering to accurately form images of the objects that belong to a target data class or group of classes, while instantaneously erasing objects of the other data classes at the output field-of-view.&quot;<p>This only works for the privacy-minded, naive among us. If you want to exclude something from a picture or video. Do NOT record it, at all, EVER! If it can record anything it can record the wrong thing.

So this is really cool and useful, but it&#x27;s important to keep in mind that since this is a diffractive structure, it probably only works with coherent light (what you get from a laser.) Most normal light sources produce incoherent light, and that tends to not work so well with complex diffractive structures.

And, with a bit of poisoning in the image training data, all of the security cameras at $Critical_Facility will be utterly blind to anyone who wears a North Korean Military Intelligence full-dress uniform...

I&#x27;m not sure the authors appreciate the impact of their own invention. This isn&#x27;t a camera that censors things: it&#x27;s a passive image segmentation model that runs in real time and consumes zero power. This would have huge implications for robotics applications.

If this AI had designed an animal&#x27;s optics it would have been the frog&#x27;s: <a href="https:&#x2F;&#x2F;indianapublicmedia.org&#x2F;amomentofscience&#x2F;eat.php" rel="nofollow">https:&#x2F;&#x2F;indianapublicmedia.org&#x2F;amomentofscience&#x2F;eat.php</a>

Previous discussion: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=32469117" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=32469117</a>

Is there an adversarial network that can take the redacted result and try to reconstruct what the camera actually saw?

&gt; Since the characteristic information of undesired classes of objects is all-optically erased at the camera output through light diffraction, this AI-designed camera never records their direct images. Therefore, the protection of privacy is maximized since an adversarial attack that has access to the recorded images of this camera cannot bring the information back. This feature can also reduce cameras’ data storage and transmission load since the images of undesired objects are not recorded.<p>That seems overstated. In the third example image pair, I can easily see a <i>shadow</i> of the input 5 in the output. I&#x27;m pretty sure the 9 is also there in the fourth pair, but the shadow is not as clear.

Get the digital equivalent of fnord in optical algorithms, feel free to rob&#x2F;~murder~ assassinate with no evidence. Bonus points for when implants become widespread, then people won&#x27;t be able to see you either!

What guarantees do you have that information doesn&#x27;t still bleed through -- e.g. that compressed sensing techniques could still recreate meaningful parts of the obscured image?

Dude this is insanely cool. It&#x27;s through light diffraction that it censors. You could make an ad blocker camera haha!<p>Very cool. Though I wonder if we&#x27;ll get a Eurion censor camera instead.

This was not designed by an &quot;AI&quot;, it was designed through gradient descent optimization. It is an interesting application but it has nothing to do with AI.

The real problem is that humans have always deluded themselves that some technology was a &#x27;truth technology&#x27;. It&#x27;s been done with everything from typewriters to cameras. However, the camera has always lied, always rendered a counterfactual to a hypothesized truth state, denied access to fundamental reality. That a camera may now lie in a slightly different fashion does not alter that.<p>Are you going to believe your lying eyes?

About 6 years ago I sat on a jury. I was told by the defense that the plaintiff, who was suing for being critically injured in a workplace scenario, was overstating his injuries. They showed evidence that saw the plaintiff washing his car. The defense pointed out that there were timestamps on those car washing videos and each took place over 4 hours because he had to rest due to the pain from his sustained injuries. Aside from the facts of the case which were clearly in favor of the plaintiff, this attempt at deception pushed the jury to award more money than it likely would have otherwise.<p>Now that storytime is out of the way, this particular AI reminds me of a photo taken at a lynching.<p><a href="https:&#x2F;&#x2F;lynchinginamerica.eji.org&#x2F;report&#x2F;assets&#x2F;imgs&#x2F;14_crowd_desktop.jpg" rel="nofollow">https:&#x2F;&#x2F;lynchinginamerica.eji.org&#x2F;report&#x2F;assets&#x2F;imgs&#x2F;14_crow...</a><p>If you obscure the top half of this photo from view, how does that change your perspective regarding what is going on at the time? IMO, recordings need to record what is, not what we believe we want to see.

No mention of Black Mirror in the comments yet - I&#x27;m surprised!<p>A lot of the other theorising here runs right alongside the premise of one of the stories told in the White Christmas episode[1].<p>IIRC the episode was very well done, as most of them are.<p>[1] <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=AOHy4Ca9bkw" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=AOHy4Ca9bkw</a>

Reminds me of this really cool video about using Fourier Optics for optical pattern recognition.[1] The video happens to have one of the best explanations of Fourier transforms I&#x27;ve yet encountered.<p>1: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=Y9FZ4igNxNA" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=Y9FZ4igNxNA</a>

&quot;AI-designed optical filter blurs out areas of an image that don&#x27;t match the pre-trained network design&quot; - seems to be a bit more on point.

That&#x27;s interesting, could it be used on medical imaging to erase noise or somehow highlight tumors or fractures, without software post processing?

&gt; AI-based camera design was also used to build encryption cameras, providing an additional layer of security and privacy protection. Such an encryption camera, designed using AI-optimized diffractive layers, optically performs a selected linear transformation,<p>Differentiable schemes do not generally make for secure cryptography.

Given the rise of UFO &#x2F; UAP sightings I always wondered why there wasn&#x27;t just an army of cameras pointed at strategic regions around the globe 24&#x2F;7 (that aren&#x27;t government owned). A camera like this would be great for catching only what&#x27;s really interesting.

It&#x27;s an optical MNIST recognizer.

It&#x27;s an interesting idea, but I doubt it could work well in the real world, where any angle or distance errors would likely throw it off.<p>In a production line environment with known objects, on the other hand... it could be great.

Wouldn’t the underlying data that the camera parsed to determine what to record still be recorded or be otherwise retrievable somewhere? Meaning everything is still recorded in some way?

Earworm by Tom Scott comes to mind: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=-JlxuQ7tPgQ" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=-JlxuQ7tPgQ</a>

Isn&#x27;t &quot;erased&quot; a bit misleading from the paper? I understand that the camera does not see anything but objects of interest in which it was trained and manufactured.

A fascinating article. I suppose we can add &quot;inspires biology&quot; and turn it around as more and more things emerge from nano- and bio-tech.

Reminds me of a book with a 4-digit title.

I believe my own brain works this way.

“…erasure of undesired objects”<p>This is not going to turn out well. Do we really want to edit reality in this way? This is like the printer that automatically watermarks your prints - for your security and protection! Coming to a child protection law near you real soon.<p>Want to take a picture of that Ferrari? That’ll be an extra $5.<p>No, you really can’t take photos in airports.<p>Thats a police officer(s). (Literally) Nothing to see here, move along.<p>Vampires&#x2F;Ghosts. A class of people who’s faces are in a master redact database. You know, like some real CIA Jason Borne stuff.<p>Military installation? What military installation? Replace with slave labor camp or, a more economically favorable rendition - “sweatshop.”

You&#x27;re saying this as if it were a good thing.