From time to time, I see attempts to serialize huge JSON objects to files, and in every instance, it seems like a code smell. Today, I encountered a file that made the 1GB figure look like a joke (it was much bigger than 1GB). Has anyone else found themselves working with files like this (especially JSON, but would be interested to here stories involving other serialization formats. Media files excluded)?
Yep. 3 GB JSON handler reporting in.<p>It all depends on what assumptions you can make about update frequency and the continuity of link.<p>If you can't assume 100% availability of the link, but you <i>need</i> to maintain operability in between link uptime windows, it's worth paying the price for a sizable data file download, and perform runtime deserialization of that in the abscence of uplink.<p>Remember:<p>The network being available should never, ever be taken for granted.
It makes sense if it's a file of JSON records, one per line
Less so, if it's a single blob.<p>DOn't forget that JSON isn't only a web interchange format.