TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Rethinking Files

110 pointsby UkiahSmithalmost 6 years ago

13 comments

erichansonalmost 6 years ago
Nice to see folks rethinking files, as they&#x27;re a scourge on the planet and an antiquated anti-pattern that has been holding back the industry pretty much since its inception. I don&#x27;t know how anyone could take a look at &#x2F;etc for example, and consider it anything but archaic. The adduser command is some 1130 lines long, and all it does is do CRUD on files, to name just one example. Then there are countless config files that just have to be edited by hand and happily accept syntax errors and logical errors. No modern system would tolerate this.<p>The root of the problem with files is that they lack an information model, beyond just a sequence of bytes. They are unopinionated to a fault. All files have structure. Even if that structure is a &quot;non-structure&quot; like &quot;all these files are just a random sequence of meaningless bytes&quot;, then that is their structure. But this information isn&#x27;t present in the system, nor can it be enforced or constrained when that is desirable.<p>To me, the obvious alternative is the database, aka &quot;everything is a row&quot;. We have used the database (relational or otherwise, but mostly relation) to successfully model many many domains, and bring coherence and clarity to them. The cool thing about the relational database is that it&#x27;s based on an underlying relational algebra. The syntax of data in an RDBMS is really just one manifestation of a deeper layer of structure that is syntax-free, and these abstract structures can be (and are) manifested in multiple coexisting syntaxes.<p>I&#x27;m exploring this pattern (&quot;datafication&quot;, headshake) with Aquameta (<a href="http:&#x2F;&#x2F;aquameta.org&#x2F;" rel="nofollow">http:&#x2F;&#x2F;aquameta.org&#x2F;</a>) and written a lot more about why file-centric is holding us back (<a href="http:&#x2F;&#x2F;blog.aquameta.com&#x2F;intro-chpater2-filesystem&#x2F;" rel="nofollow">http:&#x2F;&#x2F;blog.aquameta.com&#x2F;intro-chpater2-filesystem&#x2F;</a>). Boot to PostgreSQL! :)
评论 #20080680 未加载
评论 #20080758 未加载
zokieralmost 6 years ago
&gt; People like Unix&#x27;s “everything is a file” approach because what it really means is “everything is exposed to the same nexus”. It means you need only ssh to a system and you have all the power to reshape all aspects of that system with a single interface, the command line, using a common set of highly composable tools<p>But at least in Linux there are ton of files that are <i>not</i> exposed to the &quot;same nexus&quot;, i.e. filesystem. The most common example would be network sockets. They are files, but do not exist anywhere in filesystem. In Linux file is more of an object handle.<p><a href="https:&#x2F;&#x2F;yarchive.net&#x2F;comp&#x2F;linux&#x2F;everything_is_file.html" rel="nofollow">https:&#x2F;&#x2F;yarchive.net&#x2F;comp&#x2F;linux&#x2F;everything_is_file.html</a><p><a href="http:&#x2F;&#x2F;events17.linuxfoundation.org&#x2F;sites&#x2F;events&#x2F;files&#x2F;slides&#x2F;fd_0.pdf" rel="nofollow">http:&#x2F;&#x2F;events17.linuxfoundation.org&#x2F;sites&#x2F;events&#x2F;files&#x2F;slide...</a>
评论 #20080022 未加载
评论 #20078018 未加载
评论 #20077953 未加载
Lowkeylokialmost 6 years ago
I found the URL addressing scheme of Redox to be fascinating, if perhaps slightly less user-friendly compared to files and file paths.<p><a href="https:&#x2F;&#x2F;doc.redox-os.org&#x2F;book&#x2F;design&#x2F;url&#x2F;urls.html" rel="nofollow">https:&#x2F;&#x2F;doc.redox-os.org&#x2F;book&#x2F;design&#x2F;url&#x2F;urls.html</a>
评论 #20077964 未加载
EmilStenstromalmost 6 years ago
I recommend opening up developer tools and adding this before reading this article:<p><pre><code> body { width: 40em; margin: 0 auto; font-size: 1.4em; line-height: 1.4em; }</code></pre>
评论 #20080577 未加载
OJFordalmost 6 years ago
What if the interaction were more like OOP - the File class wouldn&#x27;t necessarily make sense as the top parent.<p>Would be kind of interesting to call methods on objects rather than read&#x2F;write files, but it&#x27;s not immediately obvious to me that that really gains anything over the status quo.<p>And now that I&#x27;ve written that, I wonder is that what powershell&#x27;s verb-object does anyway? I&#x27;ve never come close to proficient enough (nor wanted to!) to know.
评论 #20079380 未加载
评论 #20079306 未加载
评论 #20077938 未加载
mpweiheralmost 6 years ago
That&#x27;s kind of the point of stores:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;mpw&#x2F;MPWFoundation&#x2F;blob&#x2F;master&#x2F;Documentation&#x2F;Stores.md" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;mpw&#x2F;MPWFoundation&#x2F;blob&#x2F;master&#x2F;Documentati...</a><p>and Polymorphic Identifiers:<p><a href="http:&#x2F;&#x2F;objective.st&#x2F;URIs&#x2F;" rel="nofollow">http:&#x2F;&#x2F;objective.st&#x2F;URIs&#x2F;</a><p>Hierarchical paths were a good idea, let&#x27;s use them. Objects were also a good idea, let&#x27;s use those. A small set of verbs (GET, PUT, POST, DELETE) was also a good idea. Let&#x27;s combine these!<p>Abstract from:<p><pre><code> Path + File + POSIX I&#x2F;O URI + Resource + REST Verbs </code></pre> Get:<p>1. Polymorphic Identifiers, which subsume paths, URIs, variables, dictionary keys etc.<p>2. Stores, wich resolve URIs, subsume filesystems, HTTP servers, dictionaries, etc.<p>3. A small protocol that essentially mirrors REST verbs in-process<p>See also: In-process REST, <a href="https:&#x2F;&#x2F;link.springer.com&#x2F;chapter&#x2F;10.1007&#x2F;978-1-4614-9299-3_11" rel="nofollow">https:&#x2F;&#x2F;link.springer.com&#x2F;chapter&#x2F;10.1007&#x2F;978-1-4614-9299-3_...</a>
评论 #20078082 未加载
syn0bytealmost 6 years ago
Your not solving anything, at best you are getting maybe one extra level of abstraction by shifting <i>potential</i> complexity in the application. It may or may not care about internal file &quot;schema&quot; and thus has no code for it. Shifts to <i>concrete</i> complexity in the system; Your application doesn&#x27;t utilize file schema but some applications might so everyone gets a schema field and there is a bunch of extra code and complexity to support it.<p>From a security&#x2F;reliability standpoint it sounds like a nightmare combining the worst of things like NTFS alternate data streams and share library loading into one.
leocalmost 6 years ago
See my earlier comment, <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14542595" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14542595</a> .<p>Lotus Agenda&#x2F;Chandler <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Chandler_(software)" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Chandler_(software)</a> is another part of this long Grail quest.
bayareanativealmost 6 years ago
Files are too finite, low-level and lose generate&#x2F;parsing knowledge that is implemented N times in N places. OSes should read and write message-oriented streams of records (pb, capnp or similar.) that are invisible to the user, while tools and code see data and data structures. This solves many problems of unnecessary and repeated effort parsing log files, log file rotation, proprietary file formats, portability, compatibility and extensibility.<p>Also, programs should be able to dynamically-serve the contents of &quot;files&quot; as well with an &quot;activation symlink&quot;, i.e.,<p><pre><code> &#x2F;etc&#x2F;resolv -&gt;* resolvconf </code></pre> The &quot;the everything must be plain text&quot; refrain is obsolete and unnecessary because it&#x27;s trivial to serialize anything to any format since it would already be an universally-supported data structure both in tools and code.<p>It&#x27;s not 1978 anymore.
评论 #20079846 未加载
O_H_Ealmost 6 years ago
Two sic projects that can help managing files until we get another system<p>TMSU - tags your files and then access them through a virtual filesystem from any other application<p><a href="https:&#x2F;&#x2F;tmsu.org" rel="nofollow">https:&#x2F;&#x2F;tmsu.org</a> -- <a href="https:&#x2F;&#x2F;github.com&#x2F;oniony&#x2F;TMSU" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;oniony&#x2F;TMSU</a><p>Tagsistant - Semantic filesystem for Linux, with relation reasoner, autotagging plugins and deduplication<p><a href="https:&#x2F;&#x2F;www.tagsistant.net" rel="nofollow">https:&#x2F;&#x2F;www.tagsistant.net</a> -- <a href="https:&#x2F;&#x2F;github.com&#x2F;StrumentiResistenti&#x2F;Tagsistant" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;StrumentiResistenti&#x2F;Tagsistant</a>
solidsnack9000almost 6 years ago
The examples given at the end, where verbs are commands at certain paths, looks a lot like a special file system. All the printers are under `&#x2F;print` and all the print commands are under `&#x2F;print`. One could imagine all the database tables being under `&#x2F;db` and all the commands being under `&#x2F;db&#x2F;bin`.
ubrpwnzralmost 6 years ago
Another site, can we please just add something like this:<p>&lt;style xmlns=&quot;<a href="http:&#x2F;&#x2F;www.w3.org&#x2F;1999&#x2F;xhtml&quot;&gt;" rel="nofollow">http:&#x2F;&#x2F;www.w3.org&#x2F;1999&#x2F;xhtml&quot;&gt;</a> body{ max-width: 600px; font-family: &quot;Calibri&quot;; margin-left: auto; margin-right: auto; }<p>&lt;&#x2F;style&gt;
tgbugsalmost 6 years ago
I&#x27;ve done some silly things [0] with python&#x27;s pathlib recently that seem related to the issues discussed here. Given that smalltalk message passing finally clicked for me durnig the process, I am attracted to an object-like solution for everything (or a file-object-like solution for everything, since the practical performance advantages are undeniable). That said there are some considerations both for the low level implementation, and for high level things like affordances for &#x27;file&#x27; operations.<p>In direct response to the suggestion about file paths for verbs. Allan Kay says in one (possibly many) of his talks something along the lines of &#x27;every function should have a url.&#x27; The one of surely many challenges is how to ensure that the mechanism used to populate file system paths with nested functionality (e.g. &#x2F;usr&#x2F;bin&#x2F;ls&#x2F;all to `ls -a`) don&#x27;t trigger malicious behavior during service&#x2F;capability discovery. Being able to more deeply introspect file data and metadata as if the file were a folder could potentially be implemented as a plugin, and I worry about the complexity of requiring a file system to know about the contents of the files that it hosts, or that the files themselves be required to know about how to tell the file system about themselves. Existing file systems adhere to a fairly strict separation of concerns, since who knows what new file format or language will appear, and who knows what file system the file will need to exist on.<p>Said another way I think that the primary issue with the suggested approach is that it is hard to extend. The file system itself needs to know about the new type of object that it is going to represent, rather than simply acting as an index of paths to all objects. If there is a type of object that is opaque to the current version of the file system that object either has to implement a file-system-specific discovery protocol (which surely would have fun security considerations if it were anything other than a static manifest) or the user has to wait for a new version of the file system that knows what to do with that file type.<p>Some thoughts from my own work. (partially in the context of OJFord&#x27;s comment below)<p>Treating files and urls as objects that have identifiers, metadata, and data portions and where the data portion is treated as a generator is very powerful, but the affordances around the expression local_file.data = remote_file.data make me hesitate. When assignment can trigger a network saturating read operation, or when setter doesn&#x27;t know anything about how much space is on a disk, etc. then there are significant footguns waiting to be fired and I have already shot myself a couple of times.<p>The more homogeneous the object interface the better. However, this comes with a major risk. If the underlying systems you are wrapping have different operational semantics (think files system vs database transactions) and there is no way to distinguish between them based solely on the interface (because it is homogeneous) then disaster will strike at some point due to a mismatch. To avoid this everything built on top of the object representation has to be implemented under the assumption of the worst case possible behavior, making it difficult to leverage the features of more advanced systems. As with the affordances around local.data = remote.data, if I have a socket, a local file, a remote web page that I own, a handle to an led, a handle to a stop light, a database row in a table that has triggers set, the stdin to an open ssh session, and a network ring buffer all represented in the same object system, I have as many meanings for file_object.write(&#x27;something&#x27;) as I have types of objects, and the consequences and side effects of calling write are so diverse (from flipping bits on a harddrive to triggering arbitrary code execution) that it is all but guaranteed that something will go horribly wrong. At the very least there would need to be a distinction between operations where all side effects could be accounted for beforehand (e.g. writing a file of known length to disk has the side effect of reducing free disk space, but that is known before the operation starts), and operations where the consequences will depend on the contents of the message (e.g. DROP TABLES), with perhaps a middle ground for cases with static side effects (e.g. the database trigger) but that would not immediately visible to the caller and that might change from time to time.<p>The distinction between files and folders is quite annoying (non-homogeneous), especially if you want to require that certain pieces of metadata always &#x27;follow&#x27; a file. This is from working with xattrs that are extremely easy to loose if you aren&#x27;t careful. Xattrs are a great engineering optimization to make use of dead space in the file system data structure, but they aren&#x27;t quite the full abstraction one would want. It is also not entirely clear what patterns to use when you have a file that is also a folder -- do you make the metadata the outer file and the data the inner file? Or the other way around? Having the metadata as the outer file means that you can change the metdata without changing the data, but that the metadata will always &#x27;update&#x27; when its contents (the data) changes. However, when I first thought about using such a system, I had it the other way around, and a system with that much flexibility I suspect would have even more footguns than the current system.<p>Another issue is the long standing question around what constitutes an atomic operation. Everything is simple if only a single well behaved program is ever going to touch the files, but trying to build a full object-like system on top of existing systems is a recipe for leaky abstraction nightmares.<p>While I was working on this I came across debates from before I was born. For example hardlinks vs symlinks. There are real practical engineering tradeoffs that I can&#x27;t even begin to comment on because I don&#x27;t understand the use cases for hardlinks well enough to say why we didn&#x27;t just get rid of them entirely.<p>0. <a href="https:&#x2F;&#x2F;github.com&#x2F;SciCrunch&#x2F;sparc-curation&#x2F;blob&#x2F;master&#x2F;sparcur&#x2F;paths.py" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;SciCrunch&#x2F;sparc-curation&#x2F;blob&#x2F;master&#x2F;spar...</a>
评论 #20080321 未加载