(Mux founder here, but adding some additional community bits to the conversation)<p>Probably unsurprisingly, the (honestly absurd world of) color is a pretty consistent topic among video engineers. Color theory is one of those areas that people working directly with it see it as a completely unsolved problem, and everyone else largely takes it for granted. This is a small sample size, but these are all talks just from a local SF Video meetup and the Demuxed conference, and the speakers work at YouTube, Vimeo, Mux, and a fruit company.<p>Color (SF Video 2016) - <a href="https://www.youtube.com/watch?v=PiAiOl1Pvgk" rel="nofollow">https://www.youtube.com/watch?v=PiAiOl1Pvgk</a><p>Early Experiments in Color Vision and Their Applications to Modern Color Theory (SF Video 2017) - <a href="https://www.youtube.com/watch?v=fXd6HLqpoMk" rel="nofollow">https://www.youtube.com/watch?v=fXd6HLqpoMk</a><p>A Jaunt Through Color Technology in Video (Demuxed 2017) - <a href="https://www.youtube.com/watch?v=XMnvY7a4-As&list=PLkyaYNWEKcOfntbMd6KtHhF7qpL9hj6of&index=16" rel="nofollow">https://www.youtube.com/watch?v=XMnvY7a4-As&list=PLkyaYNWEKc...</a><p>Your browser and my browser see different colors (SF Video 2020, by the author of this post) - <a href="https://www.youtube.com/watch?v=9JXx0bao7ho" rel="nofollow">https://www.youtube.com/watch?v=9JXx0bao7ho</a>
> And many people know that specific colors are really just wavelengths in the electromagnetic spectrum.<p>Colors that can be stimulated using a single wavelength are spectral colors, which are only a small subset of perceptible colors. In the CIE xy model, spectral colors are on the curved boundary, while the straight boundary is the line of purples - impossible colors.<p>Edit: And because CIE XYZ is used as a sort of universal connecting and definition space, all our color spaces are defined in terms of it. But this leads to a the-map-is-not-the-territory fallacy: CIE XYZ was based on just a handful of observations and generously extrapolating. CIE XYZ does not define the set of visible colors. It's a map of colors.<p>---<p>This post is about conversion from the video's color space to a rendering color space, and the problems with that. Another fun question is whether that rendering color space is actually the display device's color space, or nah. While browser do output color management (i.e. they use the color profile of the output device and convert all their output to the ad-hoc color space that represents), I don't think they do that for video actually. In fact, there are virtually no video players supporting color management on the output side of things.
Open firefox tickets:<p>[meta] Proper colorspace support and color rendering for video playback
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1494381" rel="nofollow">https://bugzilla.mozilla.org/show_bug.cgi?id=1494381</a><p>Properly support ICC v4 profiles and enable it <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1500737" rel="nofollow">https://bugzilla.mozilla.org/show_bug.cgi?id=1500737</a><p>And many more
<a href="https://bugzilla.mozilla.org/buglist.cgi?quicksearch=color+management" rel="nofollow">https://bugzilla.mozilla.org/buglist.cgi?quicksearch=color+m...</a>
I've been working on a Vulkan rendering engine for an embedded bit of hardware, and even though I spend an insane amount of time getting the colour spaces correct (textures, tools, presentation format, etc) in the end the cheap LCD screen cheated with it's firmware (I wanted BT2020, but it kept on giving 601, I couldn't even get BT709). The firmware lied that it supported multiple formats (including P3), but it was just a menu option with no effect. My reference desktop monitor did show all the differences, so I know that engine code path was correct.
> And many people know that specific colors are really just wavelengths in the electromagnetic spectrum. [...]<p>> There are many systems involved to turn an RGB triplet value into a specific wavelength of light<p>I think this wavelenght-color connection, often perpetuated in high-school classrooms, confuses people more than helps. In particular when talking RGB colors, your display (or really, any part of the process) will not really change the wavelenghts of outputted light based on the input.<p>Also colors really are not just wavelengths in EM spectrum. Color is more of a perceptual phenomenon, something that happens in our heads, more than physical phenomenon. And there are many things that can impact the color perception, most obviously the other surrounding colors and ambient lighting.
Recently I went down the rabbit hole of JPEG encoding. The default encoder - libjpeg turbo - is going to write lookup tables for CRT monitors of yesteryear where the signal is analogue and the picture is magically made good looking by the 1280 X 1024 CRT. Trinitron was the gold standard then.<p>The Mozilla JPEG encoder mozjpeg assumes a high density digital display and encodes for that, typically with less banding but softer.<p>Not a lot of people cared for mozjpeg and the barely perceptible differences, even though file sizes were smaller into the deal.<p>This experience made me aware of how few people are working at the cutting edge of image processing, for images or video. It is amazing how much we take their work for granted.<p>Frustrating differences may be between screens, browsers and operating systems, we are lucky to have what we have got and also lucky to have such forgiving eyes. The whole shebang is nothing short of a miracle.
> Most people know some basics of color theory. [...] And many people know that specific colors are really just wavelengths in the electromagnetic spectrum. [...] There are many systems involved to turn an RGB triplet value into a specific wavelength of light.<p>A 5-year old asks you "The Sun is a big hot ball! What color is the ball?". What do you tell them? They ask "And sunlight?" You say?<p>Most first-tier astronomy graduate students simply get Sun color wrong. A widespread misconception, perhaps first learned in Kindergarten, then reinforced by incorrect textbooks, persisting unintegrated into grad school. But more interesting here, is that first-tier non-astronomy physical-sciences graduate students often answer with some variation on "it doesn't have a color; it's rainbow color".<p>Confusing wavelength and spectrum and optical color. As in TFA, and discussion here. So I suggest very few people have a firm grasp on even core basics of color theory.<p>OT (from an email draft in my other window): What if those students had instead learned Sun color correctly in Kindergarten? How might it then have been used over the years, to teach other topics better? For example, color is often taught preK-1. So how might color be better taught, with improved conceptions, progressions, and interactives, building in part on this firmer grasp of Sun color?
The article is focused on video, but fwiw (at least several years ago), the browsers do bad stuff with images, too. It is (was?) impossible to get Firefox and I think also Chrome to correctly render PNG, JPEG, and other formats, <i>especially</i> when they provided a color profile.<p>At the time (again, like a decade ago) at least, it was roughly: Firefox thinks every JPEG is sRGB and every PNG is “huh?”, Chrome got JPEG right (treat as sRGB when no profile is present) but couldn’t do Adobe RGB (and wouldn’t even open a TIFF), and Safari used Apple’s long-standing color code to basically get this right (though I don’t recall if it went with “assume sRGB if no profile”).<p>I’d be curious to reproduce those old results, especially given the cool “oh yeah, Chrome doesn’t show the same colors in software mode” shown here!
> First and foremost, ALWAYS set the colorspace metadata on your videos.<p>> If you are using ffmpeg and you don't have color flags set, you are doing it wrong.<p>This.<p>Set the color tag in your files. Please.
All-zeros isn't even really a valid ycbcr value, since even "full-range" "pc" is 1-255 for cb and cr! (Only full-range y has a range of [0,255])
For anyone looking to get a better understanding of digital color, I'd recommend "The Hitchhiker's Guide to Digital Color":<a href="https://hg2dc.com/" rel="nofollow">https://hg2dc.com/</a>. The tone is a tad rude at times (though it's not directed at <i>you</i>), but it's done a good job of explaining stuff to me so far (I haven't finished reading it yet).
I wish all browsers would heed color space hints consistently. Seems like some things are always going to assume the generic sRGB, and some thing will notice the profile and convert accordingly.<p><a href="https://www.littlecms.com" rel="nofollow">https://www.littlecms.com</a> can come in handy if you have tricky color space issues.
I made <a href="https://colorcontroversy.com/" rel="nofollow">https://colorcontroversy.com/</a>. At some point I should go through the logs and figure out which colors are the most controversial across different browsers.
There was a website that showed different browsers rendered CSS differently, it reminds me vaguely of a square / maybe divided in triangles. Anyone know the website? I have been looking for it but to no avail.