Learning to See in the Dark (2018)

As a photographer, the comparison to &quot;raw&quot; results without color balance or noise removal seems somewhat deceptive. The effects visible in the video seem easy to quickly replicate with existing techniques, such as the &quot;surface blur&quot; filter that averages out pixel values in areas with similar color.<p>This happens at the expense of detail in low-contrast areas, producing a plastic-like appearance of human skin and hair, and making low-contrast text unintelligible, which is why it&#x27;s generally not done by default.

Hi, I&#x27;m a developer at NexOptic[0] and we are a company that was deeply inspired by this paper when it was first published. We had a lot of early success when attempting to replicate the results on our own and ended up running with it, and extending it into our own product line under our ALIIS brand of AI powered solutions.<p>For those curious, our current approach differs in some very significant ways to the author&#x27;s implementation, such as performing our denoising and enhancement on a raw bayer -&gt; raw bayer basis with a separate pipeline for tone mapping, white-balance, and HDR enhancement. As well, we explored a fair amount of different architectures for the CNN and came to the conclusion that a heavily mixed multi-resolution layering solution produces superior results.<p>As other commentators have pointed out, the most interesting part of it is really coming to terms that, as war1025 pointed out, &quot;The message has an entropy limit, but the message isn&#x27;t the whole dataset.&quot; It is incredibly powerful what can be accomplished with even extraordinarily noisy information as long as one has a extremely &quot;knowledge packed&quot; prior.<p>If anyone has any questions about our research in this space, please feel free to ask.<p>[0] <a href="https:&#x2F;&#x2F;nexoptic.com&#x2F;artificialintelligence&#x2F;" rel="nofollow">https:&#x2F;&#x2F;nexoptic.com&#x2F;artificialintelligence&#x2F;</a>

It&#x27;s surprising how little code [1] is needed to do this. On the other hand I feel this is quite dependent on the specific camera models and might not work on the RAW data downloaded from my phone. Happy to be corrected.<p>[1] - <a href="https:&#x2F;&#x2F;github.com&#x2F;cchen156&#x2F;Learning-to-See-in-the-Dark&#x2F;blob&#x2F;master&#x2F;train_Fuji.py" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;cchen156&#x2F;Learning-to-See-in-the-Dark&#x2F;blob...</a>

A &quot;Two Minute Papers&quot; on this project from 2018: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=bcZFQ3f26pA" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=bcZFQ3f26pA</a>

The problem with techniques like this is that they fundamentally amount to ‘making a plausible guess as to what the image would look like’, since essentially they can’t extract information that is simply not there. There is a Shannon entropy limit here.<p>Machine learning is really machine-enhanced educated-guesswork, which has its place but also has its limits.

It&#x27;s a great result, but it&#x27;s not perfect. No need for the &quot;perfect&quot; hyperbole in the title.

Video from the paper here: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=qWKUFK7MWvg" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=qWKUFK7MWvg</a>

Why is the &quot;page suspended&quot; ? <a href="http:&#x2F;&#x2F;cchen156.web.engr.illinois.edu&#x2F;paper&#x2F;18CVPR_SID.pdf" rel="nofollow">http:&#x2F;&#x2F;cchen156.web.engr.illinois.edu&#x2F;paper&#x2F;18CVPR_SID.pdf</a>

Finally, a way to restore <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;The_Night_Watch" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;The_Night_Watch</a> !

I was just wondering a couple days ago why the image from my phone is so grainy, while my eyes+brain can see everything clear in the dark (it wasn&#x27;t completely dark, of course).<p>This seems to replicate the post-processing we do in our brain (which is also a giant neural network). I wonder if the process is similar?

Pretty cool but seems like there’s a big limitation on this for now<p>“ The pretrained model probably not work for data from another camera sensor. We do not have support for other camera data. It also does not work for images after camera ISP, i.e., the JPG or PNG data.”<p>Would be cool to see how they come up with better models that would allow them to overcome the above limitations

I doubt this image is showing the true raw data (a):<p><a href="https:&#x2F;&#x2F;github.com&#x2F;cchen156&#x2F;Learning-to-See-in-the-Dark&#x2F;blob&#x2F;master&#x2F;images&#x2F;fig1.png" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;cchen156&#x2F;Learning-to-See-in-the-Dark&#x2F;blob...</a><p>If you take the dark image (a) from that and balance its color, the information that is present in it simply cannot contain the text from the book covers and so on. In fact, it&#x27;s full of JPEG artifacts despite the image being a PNG. It would be useful if they presented a histogram equalized image of (a).

Discussed at the time: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=17064079" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=17064079</a>

I always wondered if you can &quot;trust&quot; an image that has been basically recreated. Could that kind of image be used as an evidence in court?

It would be good to get a comparison of a brightened version of the sample image compared with the CNN version. Right now the sample image just looks black, but if you scale up the brightness you get an image that looks more like the higher ISO image. That would be a better comparison since it shows what improvements the CNN gives over naive techniques like just bumping up the pixel values.

I wonder if photographic evidence &quot;enhanced&quot; by such a method would be admissible in court?

Some questions:<p>- Did they create a special network topology for this problem?<p>- Does the network need to see the entire image, or only an NxN subblock at a time?<p>- How did they obtain the training data? Is it possible to take daylight images and automatically turn them into nighttime images somehow?

Could something like this be done for night vision goggles or is there significant latency?

Interestingly, this is effectively an extreme version of solving the colorization of black &amp; white photos problem. I wonder what the results would be if you just threw some black and white photos into the model.

(2018) and a better title that doesn&#x27;t say CNN, c&#x27;mon

isn&#x27;t this just some tweaked raw processing algorithm?

I think I&#x27;ll faint seeing AI progress anymore.<p>I didn&#x27;t even think this was possible. Have people ever done this manually before? Like without AI?

Cool and integrating this into AR Glasses would make them almost a must buy! Turn night into Day ..see in the dark, etc!

What would happen if this were paired with license plate recognition? And would it be admissible as evidence?

I believe this should have a (2018) tag.

Just wondering why for something brand new they&#x27;d use python 2.7??

Funny. I&#x27;m walking down the corridor almost in total darkness trying to get my son to sleep. I get bored and with my free hand reach to my phone, open NH and stumble upon this title. Totally unrelated to its content but I had a (quiet) laugh :)

I wonder, how much can it improve over this: <a href="https:&#x2F;&#x2F;youtu.be&#x2F;c_0s06ORTkY" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;c_0s06ORTkY</a><p>X27 is also using some kind of neural algorithm to denoise and get maximum out of the CIS

What camera are they shooting at 409,600 ISO at?

why use iso 8,000 as input and not the camera native ISO ?

Now all they have to do is make sure the correct image displayed for the story

Impressive of the American news channel, CNN, to convert images in minus one second.

Ah, after decades of effort, we have finally replicated the effect of a candle.