Experimental Nighttime Photography with Nexus and Pixel

Amateur photographer here.<p>This should be the future of DSLRs. Provide some sort of API so that I can create recipes for my photography project. Bonus if the hardware is powerful enough for me to process the way <i>I</i> want to on it (as opposed to the builtin, mostly useless, features).<p>As a silly example, say I want to take 10 photos. I want the first photo to be 1&#x2F;30s. The next 1&#x2F;15s, and so on - doubling the interval each time. I just want to be able to program this, and assign it to a button&#x2F;menu item, so it will do it automatically.<p>Or I want to do custom focus stacking. It should automatically take N shots of predefined focal distances, and if powerful enough, stack them.<p>I&#x27;ve never coded Android apps, so I don&#x27;t know how much control over the camera is exposed to you, but why can&#x27;t camera companies provide the same level of control?

Once you know some basic principles of photography, this actually shouldn&#x27;t be all that surprising. The Google Nexus 6P uses a 1&#x2F;2.3 inch type sensor that measures 6.17 x 4.55 mm. That gives us ~28 square mm of sensor area. The initial example image was taken with a full-frame camera with a sensor that measures 24 x 36 mm, yielding 864 square mm. That gives the DSLR ~30x the sensor area of the Nexus. Then with the same amount of light per square mm per second (measured by the f-stop of the lens) the Nexus needs to expose the image for 30x longer than the full-frame camera to gather the equivalent amount of light. It just so happens that this approach used 32 exposures - it makes sense that the results look comparable to a full-frame camera because the phone gathered just as much light.

My Nexus 6p takes amazingly good photos - I rarely use an SLR now.<p>The main problem I have is that it shuts down after 10-30 minutes of use - no help from google. <a href="https:&#x2F;&#x2F;code.google.com&#x2F;p&#x2F;android&#x2F;issues&#x2F;detail?id=227849" rel="nofollow">https:&#x2F;&#x2F;code.google.com&#x2F;p&#x2F;android&#x2F;issues&#x2F;detail?id=227849</a> Makes me want an iPhone. (see below Huawei might be fixing)

I&#x27;ve been pretty happy with my Pixel XL&#x27;s camera (easily keeps up with the incremental improvements in cell phone cameras over the years) but I also appreciate how they didn&#x27;t make this into too much of a puff piece where they &quot;hand wave&quot; away the amount of work still needed in Photoshop or similar.<p>The takeaway is that with some effort and intelligent use of other software tools, you can put together a nice image with all sorts of lower-end cameras. The bit at the end about hopefully adding some of this functionality to software available on the phone was a nice touch, as I imagine all of the big players are always working on that sort of thing.

The photos are impressive. But considering the introduction, it would have been nice to have some actual side-by-side comparisons with dslr. And then just for fun, the same exposure stacking applied to dslr at max iso.

&quot;The camera cannot handle exposure times longer than two seconds.&quot;<p>What&#x27;s the reason for this? Is it a hardware restriction? I suppose an artificial software restriction could be removed by using root &#x2F; other camera software.

reminds me of the low light video camera that was on here ealier. <a href="http:&#x2F;&#x2F;kottke.org&#x2F;17&#x2F;04&#x2F;incredible-low-light-camera-turns-night-into-day" rel="nofollow">http:&#x2F;&#x2F;kottke.org&#x2F;17&#x2F;04&#x2F;incredible-low-light-camera-turns-ni...</a><p>Not as fast, but the results look fairly similar.

FWIW Olympus already provides in-camera shot stacking for night shots in its Micro Four Thirds cameras via their &quot;live composite&quot; mode. See <a href="http:&#x2F;&#x2F;www.duford.com&#x2F;2016&#x2F;08&#x2F;explaining-olympus-live-composite-mode&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.duford.com&#x2F;2016&#x2F;08&#x2F;explaining-olympus-live-compos...</a> for example shots.

Incredible results!! I would really love to be able to do this on a phone.<p>I&#x27;ve tried with iPhone to take long exposures and crank up the brightness&#x2F;HDR to bring out as much signal as possible. This is the best that I could do on a fully moonlit night: <a href="http:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;km1D9" rel="nofollow">http:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;km1D9</a>

What I don&#x27;t get is why JPEG is still so commonly used when it introduces significant artifacting (even at high qualities). When a more advanced format like WebP can deliver higher quality images for the same file size.

I think the timelapse style stars is a feature and not a defect, although it goes against the original challenge.<p>Although this article is about the challenge - decent photography with a mobile phone - it does outline how easy it is to layer up lots of images in Photoshop, median everything out and get a long exposure image. Taking out the sensor &#x27;median&#x27; was clever too.<p>So you could use this with DSLR images too, to take better long exposure images whatever the sensor, so long as everything is fully HDR and manual.<p>I think that I might just give it a go. With PHP Imagemagick so that I can automate the Photoshop part and tweak settings easily.

if this could become an app it&#x27;d be great

This is great that they are investing so much in camera. In modern smartphone, I do not feel any difference in performance between my moto play and s7 edge. It is just the camera that makes me keep coming back to s7 edge, despite the $300 price tag difference. It is just much more convenient and I have stopped bringing my dslr on trips now since the phone camera works really good.

I think it would be interesting to compare an image shot in the daylight with the one created with this processing scheme in the same daylight. I expect the normal exposure shot to be better than the processed one, but it would be interesting to observe the difference in quality.

for the stars shot the next obvious step would be to automatically segment the image with optical flow. Could also try solving the hand-held problem with ORB feature matching.<p>given the previous posts from the googleblog I kind of expected a bit more algorithmic involvement beyond image stacking.

I know on the XDA forums, at least for the 5x, there is an app someone threw together to do long exposure times. I played with it some and got a pixelated mess, but I didn&#x27;t try a tripod. Some people on the thread produced some pretty good pictures.

Interesting work. However, in order to reproduce the marin headlands DSLR image the smartphone would have needed to also stitch a panorama from multiple frames (because the lens used in the DSLR shot has roughly 2x the FOV angle).

&gt;Still, this may be the lowest-light cellphone photo ever taken.<p>Not so fast buddy. Some of us have been playing with this stuff for a really long time :)

Fascinating article. Thanks for all the work.

Why take the mean instead of the median of the images? Wouldn&#x27;t the median be less sensitive to outliers?

Cameras, like so many things, have been increasingly software limited during recent years.