Linus Torvalds: “I'm happily hacking on a new save format using ‘libgit2’”

On the game I&#x27;m currently working on, it&#x27;s built very heavily around Lua. So for the save system, we simply fill a large Lua table, and then write that to disk, as Lua code. The &#x27;save&#x27; file then simply becomes a Lua file that can be read directly into Lua.<p>This is absolutely amazing for debugging purposes. Also you never have to worry about corrupt save files or anything of it&#x27;s ilk. Development is easier, diagnosing problems is easier, and using a programmatic data structure on the backend means that you can pretty much keep things clean and forward compatible with ease.<p>(Oh, also being able to debug by altering the save file in any way you want is a godsend).

What&#x27;s with all the XML hate? Of course, doing everything in XML is a stupid idea (e.g. XSLT and Ant) and thanks heaven that hype is over.<p>But if I want something that is able to express data structures customized by myself, usually with hierarchical data that can be verified for validity and syntax (XML Schemas or old-school DTD), what other options are there?<p>Doing hierarchical data in SQL is a bitch and if you want to transfer it, well good luck with a SQL dump. JSON and other lightweight markup languages fail the verification requirement.

I think this title is wrong.<p>Firstly some clarification - this appears to just be about the persistence format for his dive log. It was XML, now it&#x27;s git based with plain text.<p>As someone who had to manage a system which worked with plain text files structured in a filesystem for a number of years in the 1990s, this is done to death already.<p>You now end up with the following problems: locking, synchronising filesystem state with the program, inode usage, file handles to manage galore and concurrency. All sorts.<p>Basically this is a &quot;look I&#x27;ve discovered maildir and stuffed it in a git repo&quot;.<p>Not saying there is a better solution but this isn&#x27;t a magic bullet. It&#x27;s just a different set of pain.

Back in the bad old DOS days, instead of creating a file format for saving&#x2F;loading the configuration of the text editor, I simply wrote out the image in memory of the executable to the executable file. (The configuration was written to static global variables.)<p>Running the new executable then loaded the new configuration. This worked like a champ, up until the Age of Antivirus Software, which always had much grief over writing to executable files.<p>It&#x27;s a trick I learned from the original Fortran version of ADVENT.

From the comments (Tristan Colgate) :<p>&quot;XML is what you do to a sysadmin if waterboarding him would get you fired.&quot;<p>Made my day :-)

What I like is the &quot;I dont start prototyping till I have a good mental picture&quot;<p>I am currently stuck on a project I want to start becasue I cannot get it to fit right in my (future) head. And I am glad I am not an idiot for not being able to knock out my next great project in between lattes.<p>(Ok, in direct comparison terms I am an idiot, but at least its not compounded)

I just realized that Linus&#x27; posts are the only reason I ever go to Google Plus.

I don&#x27;t quite get Linus&#x27; problem with XML for document markup (for anything else - config files, build scripts - sure, XML is horrible). Does anyone know any more details about what his specific gripe is? For me, asciidoc (which looks very similar, conceptually, to markdown) suffers from one huge problem: it&#x27;s incomplete. Substituting symbols for words results in a more limited vocabulary, if that vocabulary is to remain at all memorable.<p>Sure, XML <i>can</i> be nasty, but thats very much a function of the care taken to a) format the file sensibly b) use appropriate structure (i.e. be as specific as necessary, and no more).

<a href="https://github.com/torvalds/subsurface" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;torvalds&#x2F;subsurface</a><p>I didn&#x27;t really know what he was talking about but I think this is it.<p>The title does need changing though as it is definitely file formats under discussion not file systems.

What is it with HN commenters and their demented ability to send topics completely of track? I would have thought someone might have examined the code or what Linus is trying to implement and comment about it.<p>But here we have threads about Lua, why people hate XML and love JSON and all kinds if irrelevant issues which have been well hashed elsewhere ad nauseam. Why not restrict to an analysis of whatever it is Linus developing?<p>HN is getting truly annoying and sucky, if it isn&#x27;t so already.

&gt; &quot;I actually want to have a good mental picture of what I&#x27;m doing before I start prototyping. And while I had a high-level notion of what I wanted, I didn&#x27;t have enough of a idea of the details to really start coding.&quot;<p>This I like. The race away from the waterfall straw man has also stripped us of the advantages of BDUF.<p>While rigid phase-driven project management helps nobody, I think there&#x27;s still room for speccing as much as we can upfront within iterative processes.<p>Or you could run to the IDE and start ramming design pattern boilerplate down its throat the second you&#x27;re out of the first meeting ;)

he talks about a save file format, not a file system. or do we have different concepts of &quot;file system&quot;?

I don&#x27;t really understand what he&#x27;s talking about here (my ignorance, not his fault.) Is it something like <a href="https://camlistore.org/" rel="nofollow">https:&#x2F;&#x2F;camlistore.org&#x2F;</a> that is a content addressable (the git part) datastore?

&gt;&gt;So I&#x27;ve been thinking about this for basically months, but the way I work, I actually want to have a good mental picture of what I&#x27;m doing before I start prototyping. And while I had a high-level notion of what I wanted, I didn&#x27;t have enough of a idea of the details to really start coding.<p>This might be a tangential discussion. Earlier, I used to have a similar approach. Can&#x27;t code until I have the complete picture. But, it&#x27;s tough to do in a commercial world and you have deliverables. So, nowadays, I start with what I know and scramble my way until I get a better picture. There are times when that approach works. But, there have been days where I was like - &quot;wish I had spent some more time thinking about this&quot;.<p>I am curious how folks on HN handle this &quot;coding block&quot;.

This is what Linus does. He has strong opinions and he throws them around. You can&#x27;t let that get to you. Both XML and JSON are just fine if used properly.

And the actual description is here: <a href="http://lists.hohndel.org/pipermail/subsurface/2014-March/010592.html" rel="nofollow">http:&#x2F;&#x2F;lists.hohndel.org&#x2F;pipermail&#x2F;subsurface&#x2F;2014-March&#x2F;010...</a>

Why reinvent on-disk data formats when you can just make a file of protocol buffers? <a href="https://code.google.com/p/protobuf/" rel="nofollow">https:&#x2F;&#x2F;code.google.com&#x2F;p&#x2F;protobuf&#x2F;</a>

Current title that I see &quot;Linus Torvalds on implementation of human-readable file system&quot; is off. It&#x27;s about file formats, not file systems.

why do you need to view filesystem and make it readable for humans, you would interact it via commands &quot;ls&quot; or some gui<p>git as the basis of filesystem is interesting, hope we don&#x27;t need to manually make branches and commits to use it

Worked on a project a few years ago where we needed distributed sync capability. Using git (or bazaar or mercurial) was one of the options - store everything in it versus a database. Interesting to see the same thought &quot;coming back&quot;.

Why not sqlite or sexpressions? Linus states that databases can&#x27;t hold previous state but that&#x27;s not really true...<p>I&#x27;m not sure why git is the best tool for the job in this case, even after reading the post &amp; some of the contents.

erik-naggum&#x27;s most excellent xml rant: <a href="http://www.schnada.de/grapt/eriknaggum-xmlrant.html" rel="nofollow">http:&#x2F;&#x2F;www.schnada.de&#x2F;grapt&#x2F;eriknaggum-xmlrant.html</a>

At work we have a git backed document store that just saves as json - versioning makes keeping track of audit points nice and easy.

Title is entirely misleading. Tech support! TECH SUPPORT!!

xml haters!!! using other formats how can I define DTDs?