The operating system: should there be one? (2013) [pdf]

I can&#x27;t help but think about what happened with graphics APIs - the older ones (DX9, OpenGL) held your hand more, and the new ones are super bare metal and not written for the faint of heart. I think most would agree this is better though; for an application developer you can simply use a much higher level library built on top, and for the library&#x2F;engine developer you have a lot more power and control. The old APIs were in an awkward middle ground of not being particularly easy to use, but also not being nearly low level enough. Nobody was happy.<p>I think modern OS stacks are in a similar boat, at least Windows and macOS. True low level control is kind of a pain, but the OS libraries are not particularly pleasant or high level enough either. So largely things like Electron or Qt are acting as the defacto OS anyway to varying degrees of success but undeniable levels of resource waste. I wonder if modern languages and tooling and such are advanced enough that the OS should become more primitive and minimal. In a way it feels like we&#x27;re moving there anyway with web tech becoming the defacto user space and toolkit. That&#x27;s fine, but god if it wasnt all built on 50 layer of cruft our machines would be screaming fast.<p>Smalltalk, if you get a chance to use it, really does act like its own OS in so many ways. It&#x27;s great as a programmers OS really, although it doesn&#x27;t really hit the right notes for average use. Still though, if someone had the urge to build it I could easily see a smalltalk based userland built on like a linux kernel (with no gnu user space) being a totally viable thing.

I would love more collaboration between operating systems. There should be more layers with standard interfaces.<p>For example BIOS used to be a driver layer back in the day, providing simple API to underlying OS. This concept should be expanded. I don&#x27;t see no reason for every OS to rewrite those drivers again and again. There should be some standard for common device APIs, like disks, keyboards, mouses, GPU and so on. I should be able to buy an Nvidia GPU, download and install UEFI driver and it should provide Vulkan API for any underlying operating system, Windows, Linux, OpenBSD, ReactOS.<p>Another example is file systems. All major operating systems are very similar when it comes to file systems, at least from the user PoV. Yes, there are Windows ACLs, Linux permissions, but it should be possible to abstract those away enough. So Windows could install its latest Windows 11 NTFS driver into UEFI and Linux should be able to just use it to access NTFS safely, without any gigantic efforts to reverse-engineer and reimplement everything from scratch.<p>At one point I thought about virtualization being an answer to those demands. These days you can write completely virtualized operating system which needs to implement virtblk, virtfs, virtio-* drivers and run it on any host operating system with perfect driver support from manufacturer. GPU is not there, but I don&#x27;t think that it&#x27;s impossible to implement. Of course running a full-blown ad-ridden Windows 11 just as a hypervisor seems like not the most elegant solution.<p>Unification will not play nicely with new progressive features like BTRFS, though. That&#x27;s something to think about as well.

Let&#x27;s start off with defining what an Operating System really is because depending on where you draw that line the outcome will be substantially different. Ignoring mainframes and other large installations for the moment, strictly from a &#x27;personal computer&#x27; point of view:<p>If you see the operating system as the task-schedular + IPC clearinghouse then the system is simple, well defined and it can be expected to perform in a deterministic and verifiably correct way (assuming working hardware).<p>If you define the operating system as &#x27;everything and the kitchen sink&#x27; including media players, browsers and so on then there is no way to make any guarantees and determinism is going to be impossible to achieve.<p>Personally I prefer the leaner-and-meaner operating systems that have a larger ring of userland software built on top of them to create as small as possible a core on which the actual end-user applications are run.<p>Those systems tend to find good middle ground between reliability, speed and security because the amount of surface area exposed is going to be the minimum at every level.<p>Something like Windows is on one end of that scale, something like MacOS is somewhere in the middle. Linux can be stripped (or more accurately: it could be stripped) to something a lot slimmer than the default distros, and something like IRIX (no longer used in production as far as I know) was roughly where MacOS sat (custom hardware, custom OS tied closely to that hardware).<p>An ideal OS that limits itself to scheduling and IPC is currently not on the market for desktop usage, and the arrival of the mobile device market has blurred many of the lines that we thought were drawn pretty clearly in the past.<p>The &#x27;lets move everything into the kernel&#x27; philosophy is as far as I&#x27;m concerned fatally broken and likely things will have to get a lot worse before they will finally get better, and it would not surprise me if everything that we do today will be discarded once that happens.

&quot;an operating system is a collection of things that don’t fit inside a language; there shouldn’t be one&quot;<p>I&#x27;ve wondered the same thing about databases. Just like an OS, a database has to manage data on disk and figure out when to cache it. The files &amp; folders metaphor is arguably a relic of pre-computer times--instead of searching through folders, it is easier to search and sort based on particular attributes, something that databases are well-suited to. I think it&#x27;d be interesting to start with a database and try to build an OS around it, see how far you could go.

I&#x27;ve had some experience with running applications without an operating system. First and foremost I like the simplicity. Instead of having state distributed between the kernel and the application all the state is in your application. No longer having this dance where an application and the kernel interacts there is only one place where state is kept.<p>This can lead to some spectacular results for things like TCP. Having the ability to serialize a TCP connection and sending it over the network means you&#x27;re able to transfer a TCP connection from one host to another, even with SSL on top.<p>Now, this can be done with a POSIX kernel as well, as long as you move the networking from the kernel into the application. But if you&#x27;re doing this you&#x27;re eroding away the benefits of the kernel, basically turning your OS into a bootloader. Same thing with scheduling.<p>Another benefit is that, at least on compiled languages, code that isn&#x27;t used isn&#x27;t there. So the linker can strip away everything that is never called.<p>This still feels like utopia. The average development environment is pretty complex and ripping out and replacing all the layers below your language runtime is too much effort.

I see the paper uses the term &quot;postmodern&quot; multiple times and references a paper &quot;Notes on Postmodern Programming&quot; which begins with a Manifesto. What does this term mean in this context? I don&#x27;t like what I smell.

Pardon the biological analogy, but diversity has 2 main benefits : resistance to epidemics (malware) and faster evolution (innovation).<p>I don&#x27;t want to converge towards a single OS, but welcome standards and bridges between them.

(Only read the abstract). I’m learning Emacs and it’s making me think that we missed on the correct abstraction level of the operating system. There are programs that are OS-like (Emacs, browsers, Electron, anything that takes plug-ins). Maybe operating systems could have been lower level, and we could have developed more&#x2F;better options on the pseudoOS level on top. This seems somewhat inline with the Smalltalk vision.

On one had - yes please, screw OSes, gimme something like Smalltalk environment (in it&#x27;s Pharo incarnation preferably).<p>On other hand - define OS. Isn&#x27;t that Smalltalk environment an OS if it runs bare metal?

IBM never had a plan for a mainframe operating system that made sense -- given the idea that the &quot;360&quot; was supposed to encompass the whole circle of applications.<p>What happened was some academics figured out it was possible to make virtual machines on the 370 and that you could run a few different operating systems at the same time. If you were doing software dev on the 370, for instance, you would start up a VM running VM&#x2F;CMS which was a single-user OS a lot like CP&#x2F;M or MS-DOS, work there, possibly using services offered by another OS in another VM by some kind of message passing.<p>This turned out to be the &quot;master plan&quot; in the long term, since they could have a few different mainframe OS, even some applications that run on &quot;bare metal&quot;, but have it all run on one machine or one cluster of machines and be managed under the same pane of glass.

I&#x27;ve been playing with Unikernels for Go server deployment - so far so good. I&#x27;m not sure if this counts as &quot;no OS&quot; - as I understand it (and I&#x27;m not sure I do) the Unikernel core provides the calls that the Go runtime is expecting.<p>But the central idea that the server system can only run one program makes a lot of sense to me. Obviously it won&#x27;t work for a desktop, but then the whole idea that desktops and servers run the same OS doesn&#x27;t make much sense to me.

OSes do have shared abstractions such as file systems and process input and output. I think the trick is to make the common abstractions richer in a way that enough people agree on and are willing to adopt. Most of the shared OS features are basically grandfathered in. So a big part of this is that this stuff is not really a rational choice usually. It&#x27;s just the way it&#x27;s always been.<p>So upgrading the capabilities and creating new common abstractions in an OS is not that hard. But almost impossible to get the majority to go along with it.<p>Browsers demonstrate an additive way of getting a lot of that. In that we now have essentially two highly compatible widely deployed VMs -- Firefox and Chrome. Which can be programmed in a number of languages that compile to JavaScript or web assembly and share a rich API.<p>Web assembly modules seem relevant.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;WebAssembly&#x2F;proposal-type-imports&#x2F;blob&#x2F;master&#x2F;proposals&#x2F;type-imports&#x2F;Overview.md" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;WebAssembly&#x2F;proposal-type-imports&#x2F;blob&#x2F;ma...</a>

If you have one job to do, and it doesn&#x27;t involve other computers, you don&#x27;t need an OS. MS-DOS or equivalent program loader&#x2F;file system API will work just fine. Windows, MacOS, and Linux work fine in a stand alone environment (unless an APT {advanced persistent threat, like the NSA} is after you)<p>On the other hand, if you&#x27;re even thinking about touching the internet, or any other computer, you need to have every single task limited ahead of time to a specified set of resources, like mainframes do. Doing anything less is foolish, and I certainly wouldn&#x27;t build a civilization on such folly.<p>Yes, in an era of mobile code, persistent internet, you need an operating system that doesn&#x27;t give any default permissions to any executing task, ever. It must also safely and securely multiplex access to the hardware resources, <i>all</i> of the hardware resources, in all of the subsystems in the SoC, etc.<p>There are no Operating Systems, as of yet. I weep for the future.

“Programs = data structures + algorithms”<p>If you have more than one program, they need shared data and access procedures, like the “user” and “the hardware”. Who manages them: The OS?<p>Maybe you want to share a bit more like some common higher level functionality. That becomes part of the OS, de jure (resident kernel) or at least de facto (loadable kernel modules).

As soon as two or more separate applications share some common body of code that helps them deal with The Computer, you have an operating system. It doesn&#x27;t matter whether that common body is a table of software traps that dispatch machine language routines, or whether it is Smalltalk objects.<p>Some game on a MS-DOS floppy disk that boots itself from the boot sector and uses BIOS calls has an operating system.<p>The set of requirements which describe the host environment (the &quot;virtual machine&quot; in the classic sense) that forms the backdrop against with an application is developed is the operating system.<p>The only way you don&#x27;t have an operating system is when the application itself does everything it needs with regard to the outside world by itself poking at I&#x2F;O registers, and handling every interrupt and so on. Anything independently organized for that purpose, given something resembling an API, is an operating system.<p>There is, or used to be, a sensible version of &quot;there should not be an operating system&quot; I first heard some decades ago. Namely this: &quot;there should not have be an operating system with a hardware-protected user&#x2F;kernel boundary&quot;. The idea is that there doesn&#x27;t have to be kernel which the hardware conceals using privilege levels and fences. The idea is supported by the notion that all security can be handled at the software level. The system, as such, is somehow verified to be correct. The only way to execute code is via compiling it via the system&#x27;s trusted compiler, which generates only safe instruction sequences. Those sequences can call functions in the system, and those functions sandbox what can be done. All notion of trust and privilege is software-level without any privileged instructions having to fall down a trap into a privileged kernel which meticulously validates arguments.<p>(All that rhetoric needs to be somehow adjusted for 2021, which finds us jaded by knowledge of side channel attacks, cache attacks and whatnot.)

Title should probably have a [2013] marker.

Maybe this is a hot take, but it seems like browsers have essentially become what the operating system should have been, and it&#x27;s the reason that everyone crams things into web apps &#x2F; Electron instead of building native applications.<p>Browsers have a set of APIs that are decided on by a standards committee, and you have to implement those standards if you want your browser to be compliant. As an application developer, all I need to do is check a page to see how widely a standard has been adopted and then use it - I can be sure that my code can ship to (nearly) every device and behave uniformly, with minimal tweaking.<p>If the operating system had gone that route, we wouldn&#x27;t have had to build more layers on top of it to abstract away the differences.

Title needs the [PDF] tag.

Unix generally has the correct basis for the design.<p>I agree that a filesystem as we know it in unix-like systems is a set of objects (directories) and files.<p>Would it not be preferable to retain this simple interface and instead conceive of interaction between programs and threads not as a stream of distinct objects in memory, but rather as a data tree within each program&#x27;s control which can be exposed as input or output for other programs to map against and write into &#x2F; consume?

Is there a demo showing how the Plan 9 is similar to Smalltalk regarding the “meta” stuff (programmabilty, message passing, ..)?

I think that there should be a humane operating system that operates for the human.<p>Operating systems shouldn&#x27;t get too hung up on supporting applications or browsers, instead they should focus more on serving humans.<p>Common operations that humans need should be standardized across the operating system.

When was this written? I cannot find any meta information in the paper itself.

Related question: do we need device drivers?

So...It Has Come To This...<p><a href="https:&#x2F;&#x2F;xkcd.com&#x2F;1022&#x2F;" rel="nofollow">https:&#x2F;&#x2F;xkcd.com&#x2F;1022&#x2F;</a>

i just want protocols between workflows of serverless services