If you're interested in more "hands on" or practical video codec learning, I recommend writing a H.261 decoder. Only two supported frame sizes, no B-frames, and no intra prediction (effectively a subset of JPEG) make for a simple yet possibly quite rewarding weekend project that can be completed in around 700 lines of C (my attempt). Unfortunately not much existing media is available in 261, but I think watching a video being decoded entirely by code you wrote is a pretty fun experience, including all the weird and amusing distortions you can see when debugging; and from there you can move on to MPEG-1 with variable frame sizes and B frames (another weekend, assuming you reuse much of the 261 exercise --- I ended up with 1k lines total to decode MPEG-1), and that has somewhat more existing media you'll be able to watch.<p>Then you can try H.262/MPEG-2 and enjoy the intricacies of handling interlacing as well as being able to decode DVDs and a lot of existing content; and then there's H.263 which has intra prediction... I haven't gotten past the first two largely for reasons of time and other things to play with, but IMHO getting a basic implementation of a video decoder is not that hard especially when you're working from a standard.
One thing that I wish more folks did with DSP/media tech was to start with concepts instead of diving into details first.<p>Like, "how does a video codec work?" should start with: what is the problem? (reducing the bits per second required for a video stream, since it's big) and how? (don't send superfluous detail, since it's either redundant or imperceptible).<p>Than dive into the details of how color is represented, how images are structured, how 2d signal transforms work, the principle behind the DCT as a method of representing the same data with better energy compaction by decorrelating different components, and how that can be used advantageously to reduce the number of bits for a still image, then talk about shared data between images, etc etc.<p>I've noticed as a DSP guy that when I talk about things without concepts first that everyone's eyes glaze over. Although it is nice that everyone thinks it's black magic, good job security.
As much as I like Apple it seems like they are in the back pocket of big media when it comes to this topic. AFAICT Apple only supports h.264 and h.265 meaning 1.3 billion iOS devices have browsers that can't access open standards for video.<p>Yet one more reason why Apple should be required to allow alternate browser engines IMO. Some will claim it's a battery issue but Apple has the resources to add hardware support for other codecs and I'm only guessing could already handle it just fine with their current "Bionic" chips. I'm not sure what other executes can be dreamt up. I'm sure they'll follow in the comments below.
I wish there was an article on how hardware decoding works. Detailed but approachable.<p>I guess reading source code of drivers and popular open-source video players will give me an idea, but it is too much work. Does anyone have a technical article highlighting differences between hardware decoders and important details.
Good. Want more people to understand how compression works.<p>Things like motion estimation, encoding the differences in frames, all that stuff is magic to the uninitiated. :) But this stuff is absolutely worth learning and every engineer should show interest :)
I can also recommend this article on just how impressive H.264 at compressing a file while retaining good quality.<p><a href="https://sidbala.com/h-264-is-magic/" rel="nofollow">https://sidbala.com/h-264-is-magic/</a>
This is all very cool, but what annoys me the most is that, let’s say you create video authoring software, it creates a piece of unique content and you decide to use a codec that’s licensed; you’ve got to pay royalties. If you use another it’s not supported by default. .FLAC for example or .oggv<p>To not be liable to lawsuits and to support your own you need to come up with your own codec, resulting in your own platform and the circle starts once again.<p>Why in this age can we just not all work together and make something that’s usable by everything for everyone without any restrictions.
this is good, then cringed when I saw the diagram about DRM that had M$, for msft. i guess the other companies aren't about making money at all costs?