Just like the author, I too am quite particular about empty space in my MP3 files.<p>When I was ripping my CD collection, I religiously tagged the MP3 files with ID3v1. Later on, I had some tracks whose titles were just a tad longer than 30 chars so I had to use ID3v2 for those and I noticed the file size grew by _a lot_.<p>Frustrated by this, I opened them in a hex editor and learnt that ID3v1 was a fixed format of 128 bytes, but v2 was variable. I also found out that the software had added a 4KB zero-byte padding to the v2 tag, which was "necessary" because the tag is now at the front of the file, and this padding allows more tag data to be added easily later on.<p>I tried various ID3 tagging software at that time and all of them added a padding. So I learned about the tag format and wrote a tagger myself that didn't add any padding. It was a great learning experience, and I managed to shave those useless zero bytes from my MP3s.
I worked with the FLAC format and it also recommends padding to make it easier to edit metadata. I think libflac reserves 4 KB by default.<p>> PADDING: This block allows for an arbitrary amount of padding. The contents of a PADDING block have no meaning. This block is useful when it is known that metadata will be edited after encoding; the user can instruct the encoder to reserve a PADDING block of sufficient size so that when metadata is added, it will simply overwrite the padding (which is relatively quick) instead of having to insert it into the right place in the existing file (which would normally require rewriting the entire file).<p>-- <a href="https://xiph.org/flac/format.html#format_overview" rel="nofollow">https://xiph.org/flac/format.html#format_overview</a>
Ripping from cd reserves 0.05% (about 5kb) for metadata. I can imagine some communication breakdown where someone thought they meant 5% (about 500kb) when specifying the number 0.05 with a % next to it. You see percentage mixups all the time.
Apple’s album art has been janky for me for 20 years. I’m sure it would be less noticeable if I had mostly mainstream music tastes, but after years of doing album art manually, Apple screwed it all up multiple times and I stopped caring as much about fighting around the time I gave in and started streaming more music.
If only Apple had some way to store metadata for a file separately from the content. Something like files with a "data fork" and a "resource fork", maybe.
How often do users change the metadata of music bought from Apple Music? When ripping stuff by hand, this is understandable.. you make a typo, want to add a year, do some locale changes to add local characters, etc... so in this case i see a possible need to waste 5 kB of space to save a couple of seconds after an edit.<p>But with downloadad music with all the metadata (hopefully) correctly set, even 5kB is a waste of space.
<i>You can’t make changes to the metadata (e.g. change the spelling of an artist’s name) without reassembling the entire file to recalculate the offsets.</i><p>Split the file after the metadata block; change the data inside the metadata block (possibly changing its size); add the difference between the old and new size to all the offsets in it; recat the two pieces together.<p>You could even do this "streamingly", to inject album art etc., since you know what size the added content will be. Simply add the diff-offsets to the right fields as they get streamed out.<p>The above solutions didn't take much thought to come up with. I wonder why no one at Apple thought of that (especially the latter)?
In 2022, we shouldn't be reserving any area of a file as 'spare space' like this.<p>On the very rare occasion you adjust the artists name in a music file, the user expects the whole file will be rewritten. So just rewrite the whole file.<p>Whats next? Photoshop only writing out a quarter of an image when I edit my ex out of a nice photo? MS Word only touching a few bytes of a 100 page document when I make a heading bold?
It could be a form of "look, this file is X large for Y minutes, that's much more than Z, so it must be better!".<p>I have a rising suspicion that some game companies with fluid ethics do this - avoiding to optimize and strip unused content, in order to inflate game size.<p>"Wow, it's 102 GB, the game must be huge with boatloads of content!" (ie huge == higher chance of I'm getting my moneys worth). Which in turn becomes a problem with the latest generation consoles that have quite little drive space (Playstation ca 625 GB).
Because I was trying to distract myself from stuff I looked at some files in a hex editor, and really old DRM AAC / M4P files from iTunes (still have a few around...) don't seem to have the same huge "free" padding block at the start of them (you can still see smaller "free" chunks, so they don't seem to be hidden by the encryption). Which is weird.<p>DRM iTunes Store files were 128kbs, whilst non-DRM ones are 256kbs, as beyond removing DRM one of the selling points of what was called iTunes Plus at the time was better quality. So it's possibly back in circa 2009 someone made a fuckup with the encoding settings and no-one has noticed since. Or its some extremely weird conspiracy to sell larger iPods.<p>Or perhaps they wanted to make the filesizes just a bit bigger so that iTunes Plus files felt like quality, like the way expensive products have metal weights added just to make them feel heavier and more solid.[1]<p>[1] For avoidance of doubt, this last suggestion may be what is known as "humour".
The key to understanding this issue seems to be "-movflags +faststart". The TLDR is that Apple apparently isn't "fast-starting" their MPEG-4 files before distribution.<p>This became a thing when distributing QuickTime Movies on the internet became a thing (it's not an issue with random-access media), because one needed Movie metadata at the front of the file in order to support progressive playback. Because the MPEG-4 file format is effectively the QuickTime Movie file format, the need to put metadata at the front of the file continues if you want viewers/listeners to be able to play files as they're downloading.<p>That Apple isn't performing this extremely trivial pre-distribution process is extremely curious. It makes me wonder if this "common knowledge" was lost along the way, or if the people who would know this kind of super-obvious production step just aren't the same people in charge of Apple Music standards.<p>For anyone curious about what fast-start implementations look like:<p><a href="https://github.com/FFmpeg/FFmpeg/blob/master/tools/qt-faststart.c" rel="nofollow">https://github.com/FFmpeg/FFmpeg/blob/master/tools/qt-fastst...</a><p><a href="https://github.com/kanongil/node-faststart" rel="nofollow">https://github.com/kanongil/node-faststart</a>
This is what happens when you teach developers 'storage/hardware/compute is cheap' if you ask me. That plus the lack of interest in Apple Music development in general. (I cannot imagine that shitty music player received much love over the pas years. Still better then spotify though.)<p>Funny side note: I recently made it's frontend crash. Turns out it _is_ a JS frontend after all. I suspected as much because of how slow & slugish it is, but I got an actual JS error & it fell back to the old table interface
The iTunes store launched in 2003, when most users were on dialup. File sizes of music was <i>critical</i> to the success of the service. There is no way they would have had 500 kilobytes of empty space back then.<p>So when did this get added?
Glad to hear I'm not the only one who stuffs as much music into their phone as possible :) Not an apple user, but I'd also enjoy some sudden extra 6% for sure.
Two complaints I have with iTunes.<p>One is that it stores a "play count", the number of times you've played a song, directly in that song file. So it's always re-writing the source file of the track in the library.<p>That would be only mildly irritating -- radically blowing up the size of system backups, for instance -- but for the fact that it routinely corrupts the new song file stream.<p>I see so much bit rot in my iTunes audio files -- and nowhere else -- that the stupidity of not simply keeping some small database of play count veers over into uselessness.<p>-<p><i>The second complaint is that they decided that any file I had ripped from my CDs were pirated, and therefore would require an annual fee for me to keep. I understand that's a concession they agreed to from music industry in order to roll out their "iTunes Cloud" stuff.</i><p><i>But yes, I bought more than 400 CDs new and yes, I still have all of them, and indeed, I never loaned them out to anyone else (no one asked).</i><p>-<p>After a while, I just set my iTunes audio files to read-only, and a to-do item from ten years ago is to go through and re-encode my physical CDs. Still haven't done that, yet.<p>Stopped using iTunes, instead. Still need to rip them discs...
I do wonder if this was done because Apple noticed people change Album artwork more than any other metadata, and wanted to leave room so such an action would generally be quick, even on slower drives.
At first I thought it was some sort of fingerprinting for anti-piracy (which seems extreme), but seems like its just to store potential future album art :)
> You can’t make changes to the metadata (e.g. change the spelling of an artist’s name) without reassembling the entire file to recalculate the offsets.<p>Isn't this a really, really infrequent operation? And aren't hard drives fast enough these days that reassembling the entire file doesn't even take that long?
With the advent of streaming music as popular as it is (and surely Apple had to see that becoming the trend), this seems largely like a non-issue from a storage perspective. In fact, it's really only costing Apple more money in bandwidth because they still have to transmit that data - though I'd bet it compresses extremely well.
Taken alone this might be unfortunate, but given Apple's long-standing trend of upselling storage at hugely inflated prices it's pretty damning.