“The Operating System: Should There Be One?”, 2013-11-09 (; similar):
Operating systems and programming languages are often informally evaluated on their conduciveness towards composition. We revisit Dan Ingalls’ Smalltalk-inspired position that “an operating system is a collection of things that don’t fit inside a language; there shouldn’t be one”, discussing what it means, why it appears not to have materialized, and how we might work towards the same effect in the postmodern reality of today’s systems. We argue that the trajectory of the “file” abstraction through Unix and Plan 9 culminates in a Smalltalk-style object, with other filesystem calls as a primitive metasystem. Meanwhile, the key features of Smalltalk have many analogues in the fragmented world of Unix programming (including techniques at the library, file and socket level). Based on the themes of unifying OS-level and language-level mechanisms, and increasing the expressiveness of the meta-system, we identify some evolutionary approaches to a postmodern realisation of Ingalls’ vision, arguing that an operating system is still necessary after all.
[Keywords: Unix, Smalltalk, Plan 9, metasystem, composition, binding, integration]
Writing in the August 1981 “Smalltalk” issue of Byte Magazine, Dan Ingalls set forth various design principles behind the Smalltalk language and runtime [1983, Smalltalk-80: the language and its implementation], and addressed the issue of integration with the operating system as follows [1981].
An operating system is a collection of things that don’t fit into a language. There shouldn’t be one.
Although not stated explicitly, we can infer that Ingalls’ vision for there “not being” an operating system would include gradually pulling more and more system functionality (eg. isolated processes, filesystems, network stacks) into the Smalltalk runtime, where it could be exposed in the form of higher-level abstractions (eg. as persistent and remote objects) rather than the byte-streams and raw memory interfaces of Unix—which seems unquestionably to be one system to which his “very primitive” refers.
It appears that this change has not happened—at least not yet. Was there a real benefit in whole-system design underlying Ingalls’ position? If so, is it achievable? If so, would there remain any need for a programmer-facing or user-facing operating system? In this paper, we make a case for answering all these questions in the affirmative, consisting of the following contributions:
We identify the potential benefits of Ingalls’ vision, and contrast these with parallel developments in Unix relating broadly to the concerns of composition.
We argue that the natural trajectory of the Unix design, extrapolated through Plan 9 and beyond, yields an object abstraction mostly equivalent to that of Smalltalk.
We map a large number of composition “point fixes” in Unix systems to features of a Smalltalk-like environment, each of which they are replicating for some set of use cases.
We argue that this “lurking Smalltalk” could be exploited to bring about many aspects of Ingalls’ vision in an evolutionary fashion, and sketch several approaches to this.