Chris Dickinson

you have to know what to wish for

Unknown — Thu, 26 Mar 2026 00:00:00 +0000
you have to know what to wish for</h1>
Last month, someone high up at `$dayjob</code> said that we should treat any line of code written by hand as a failure.`
`This hit me like a ton of bricks, to be honest. What does it mean to be a software engineer without writing code? Is this the end of software engineering?`
Let’s define terms. Programming is theory building</a>, as Peter Naur remarked. The job of the programmer is to build and maintain a mental model of the problem space the system they develop is written against. There’s a feedback loop there in the small: the programmer writes a program to solve a problem, and learns more about the program and its environment when they run the program1</a>. Change, edit, crash, swear, learn, repeat. ^1Excelsior! Chris wrote a bit about tests as feedback loops for interrogating systems in the last post</a>. </div> With luck, eventually the program does what the programmer intended. The programmer walks away changed from the experience. They know avenues not to explore, patterns to match on that let them skip steps, maybe something interesting about their tools. Writing the code changed their mental model of the program. This understanding, not the code, is the thing of value. The programmer could write documentation about the program, but their exact understanding at that moment cannot be captured in full. This is what Naur is talking about: the programmer has a theory of the system, developed in concert with that system. That theory is a mental model, a map of the system, problem space, and how it might react to changes in the future. Software Engineering is the act of building a socio-technical system that promulgates that theory to a group of people – “engineering is programming integrated over time”</a> 2</a>. This is to say, the goal of software engineering as an organizational capability is to model the problem domain and current solution, keep multiple people</a> in alignment on that definition, and use that understanding to integrate changes from the business. ^{2 Quoting Russ Cox quoting Titus Winters. Programming is also learning to dereference references! </div> Just as programmers anneal their theory of the system by changing programs to understand the effect of their changes; software engineering manages concurrent theories of the program held by everyone involved in changing, building, and shipping the software: product managers, engineering managers, executives, marketing, sales, etc., in addition to the engineers themselves. The goal is to align all of these different ways of understanding the system.}This is, of course, impossible3</a>. But trying matters: EVERY GOOD REGULATOR OF A SYSTEM MUST BE A MODEL OF THAT SYSTEM Roger C. Conant</a></li> </ul> </blockquote> This comes to us from the field of cybernetics. Here, a “regulator of a system” is an entity which, given a set of inputs into a closed system which may produce a number of outputs, a subset of which are desirable, takes action to steer the system towards desirable output. The theorem says that a good regulator is a model of the system under regulation – a homomorphism, potentially lacking the detail of the full system. Air traffic control models planes into the airspace around an airport for the purposes of routing and sequencing, but lacks the full detail of the airport being modeled: what is the temperature inside the gate? Is the nearby river hosting migratory geese? What is the status of the airport Chili’s (TM)? Bad regulation increases the entropy of the system, destabilizing it: if air traffic control doesn’t model the geese, they might not be able to account for bird strikes. Organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations. Melvin E. Conway</a></li> </ul> </blockquote> Systems reflect the communication structure of their respective organizations because of the positive pressure of the good regulator theorem. Systems which don’t reflect their organizations tend not to be stable, even over a short run. Either the system produces incentive to align the organization around it, or the system fails spectacularly. In either case: software engineering is the capability to translate the desired state of a product system into reality, integrating it with the current theory of that system. Information doesn’t flow one-way: engineering and product management are always in productive conflict, pushing each other back and forth on the effect of changes to the system. And in the middle is the theory of the program: this conflict is only ever as productive as their models of the system are accurate. ^{3 I say alignment is impossible, but take this in the sense that there’s no such thing as perfect communication. A listener always receives something different than what a speaker has in their head: the act of communicating quantizes the information to be transferred. </div> I want to note that, as humans, we live and die by our ability to abstract. We cannot hold perfect models of the world. We are always navigating through assumptions and missing information. “Making a mental model” doesn’t mean making a perfect model. We’re constantly updating those models as we find out new information (or as we find out we don’t need to remember specifics.)}And this is true of how we think about other organizational roles: we hold a cartooned, simplified version of other jobs in our head. A manager’s job is to manage, but what does that look like from the point of view of an engineer4</a>? A product manager works to define what the product needs. A designer builds how the product feels – visually, primarily, but on other axes as well. These are tiny definitions for whole careers worth of details! ^{4 As an engineer who has bounced through management: it looks like a completely different set of (complementary) skills, and a lot less control over specific outcomes than you’d think. </div>}Every role carries its own frame with respect to the system the organization manages. The frame the role carries determines the problems and opportunities it can see – what is legible</a> to it. And so, when I say that product and engineering have a productive conflict: it is because engineering can see things that product can’t, and product can see things that engineering can’t. And there is a career’s worth</a> of details involved in building either of those frames. Software engineering is the function of an organization that assimilates change into the existing software system. Historically, it’s been the function responsible for translating that change into specifics, and communicating out the cost of those specifics. But we’re in a mechanized world</a> now. Roles are bleeding over</a>. The cost of bespoke, one-off software has dropped to zero. The genie is here to grant your wishes. The genie warps the cost of everything in software engineering. It’s as if space-time itself</a> has warped. Think of a map: there is a coffee shop 20 minutes away by foot. Now imagine the genie installs a pneumatic tube that transports you 100 miles away to another coffee shop and back in 10 minutes. That 100-mile-distant coffee shop is now closer to you, temporally, than the one down the block. What’s more, everyone gets a lamp. Anyone can generate a program now. Bespoke software is commoditized. Generating or re-generating an entire working program is as simple as prompting. The less you care about the implementation – or keeping that implementation working in the face of change – the broader you can be about the prompt. That said, what you say to the genie matters. A lot. If you don’t know to avoid certain problems, the genie will not tell you that you should think about including a solution for them. The genie uses your frame, for better or worse. And the genie understands the world anew every time you wish. It reads documentation it left behind, sure, but as we established, that is not the same thing as the theory of the program. So it’s still on the person who rubbed the lamp to understand the implications of their wishes; even more so on the compounding results of those wishes. So the problem remains, but the means have changed. Programming is no longer the foundational practice of software engineering. That is: the work of software engineering has fled from the act of programming both to the left side of the process – definitional work (“how do we define this problem and how do we wish to solve it”) – and to the right side – verification work and risk assessment (“is this implementation correct and what will happen if we’re wrong?”) The middle of the problem is where a lot of us engineers have spent most of our careers. It’s where we developed our understanding of software systems in the first place and where we hone our ability to change them. Former colleague Laurie Voss calls this kind of engineer a “craftsperson”</a>: focused on the means instead of the outcome. I think of this work-in-the-middle in terms of “feedback from the medium”: I understand programs better by changing them; I find seams in them that suggest future changes, lines along which to cut, and the process of getting my fingerprints on a codebase makes it more real to me5</a>. ^5It is practice</a>, in other words. </div> I would venture a guess that my “craftspeople” feel similarly. Being asked to change or understand a program without programming it feels like removing one’s proprioception: like floating in a sensory deprivation tank. The impressions you get of the codebase are unmoored. You have to build up a new set of reflexes to grasp the bottom of the pool. But it is possible to build up that sense again. It feels different. The details still matter, but how you come to understand those details changes. I spend a lot more time walking the codebase with Claude and testing my assumptions there. Being able to communicate how to visualize code becomes way more important: I ask the genie for maps of the system from different angles, rendered in graphviz. Claude can put things that were tedious bookkeeping (git blame histories) more directly at hand6</a>. ^{6 And Claude can steer my editor while I walk this path, which is possible because I knew that neovim supports a socket interface and that Claude can self-discover how to control it to write a skill. This is trivia!) </div> So the details still matter. In fact, the trivia is more powerful, because the genie only grants wishes as well as you’re able to frame the wish. If you can think of novel ways to combine existing concepts, you’re getting a different value from the genie than someone else would.}From a language perspective, it’s a lot of fun to hack with: I’ve noticed that my plans are executed more successfully when I run them through a grug</a> style filter, for example7</a>. ^{7 Semantic quantization, probably: “I dislike ternaries” becomes “grug hate ternaries. hit with club”. </div> But this is a manic “the sky is falling” sort of fun. I return to the question. Is software engineering a viable career if you take out the programming? The frame you spent years developing is more valuable than ever: you are capable of regulating these tools safely because you are a model of how they operate. The new software engineering looks a lot like the old. It values most of the same things in the large: exploration, trivia, model-building. It amplifies your ability to build the theory of the program. So, does hand-written code represent a failure? Well, … no. At least, I wouldn’t call it a failure. But treating it as such is a useful forcing function. It’s just not required anymore. Where it helps you develop a better model of the program, it’s arguably more worthwhile than ever.}Thanks to @ceejbot</a> for reviewing a draft of this post!



testing, one two
Unknown — Fri, 20 Feb 2026 00:00:00 +0000
testing your patience</h1>
Welcome back to the last blog at the end of software engineering. Pull up a
stool. I’m here to give you my take on testing.</p>
Wait, wait, don’t go. I promise, it’s worth reading. I typed these words by
hand.</p>
Why would testing be an important topic now, of all times, especially
considering how many of us are likely using tools to generate</em> our test suites
and those same tools to generate</em> fixes to said test suites?</p>
Well, strawperson, I would argue that tests are more important now than they
have been for a long time.</p>
You have two feedback loops available to you. You can get feedback from your
users: they can tell you your program is broken explicitly, if you’re lucky. If
you’re unlucky, they’ll tell you this implicitly by not using your product.</p>
Tests are your other feedback loop. They’re the cheap one. Good tests will tell
you something about your system as it changes. They give you confidence that a
change to one part of your system won’t set the other parts on fire. Tests keep
your generative coding tools on track.</p>

Let’s back up for a second to talk about what a test suite actually is</em>: I find
that most developers I talk to think of a test suite as an ancillary component
of their program. Tests are a property of the important code in the code base:
you might talk about “a module and its tests</em>”. The codebase is mostly about
their shipped program, application, or service. Tests say whether that program
is “Of Kwalitee”. I mean, “Of Quality”.</p>
To which I say: look again! Each test case you build is a program</em> unto itself!</p>
This is less apparent in VM languages like Python, Ruby, JS, or Java. Compiled
languages make this explicit: Rust and C++ tests are compiled into their own
binaries and run via a harness. So you’re not shipping one program, you’re
shipping (potentially) hundreds of programs. And the codebase really contains a
system whose internal interfaces can be used to assemble these programs, one of
which actually happens to be your production application.</p>
The goal of the other 100 programs is to tell you what happens as a result of
your changes to another part of that system before a user tells you. If this is
designed well, failures in your test suite help you track down which</em> invariant
your changes violated. When designed poorly, test suites tell you only that
something in your system changed</em>.</p>
You usually see this when the tests over-specify</em> behavior based on internal
interfaces. This falls out of treating tests as an afterthought: new components
each get a suite of tests. As new components are added suites of tests for new
components overlap the old: in order to pass both sets of tests the components
must behave exactly</em> as originally implemented, including their internal
structure</em>. The overlap violates modularity, in other words.</p>

Instead…</p>
Think of your implemented system as a directed graph. A step abstracted from a
control flow graph, but more concrete than a UML diagram. The nodes represent
some business logic happening, and the edges between those nodes are interfaces
for calling and returning values. Something like this:</p>
</p>
When you test, you pick some entry and exit interfaces to test, and in doing so
you end up executing a sub-graph of the system under test. I am going to refer
to this sub-graph as the “gamut” of your test, in its sense of “complete range
or scope of a thing.”</p>
</p>
Writing an effective test suite means thinking about how the gamut of each test
relates to those around it.</p>

Write tests. Not too many. Mostly integration.</p>
~ Guillermo
Rauch</a></p>
</blockquote>
You might be accustomed to thinking of tests that have a narrow gamut as “unit
tests”, and those with a wider gamut as “integration tests.” To this, I’d say:
“yes, and!”</p>
Yes, and</em> these specific terms imply a frame of reference. From the frame of
reference of your system, you might be running a unit test of a single free
function. But if your system is a Python application, from the perspective of
the Python interpreter, you’re running a pretty broad integration</em> test of the
language runtime. (I don’t love these terms for this reason: the definition of
“unit” depends on whose ruler you’re using.)</p>
That being said.</p>
When designing tests, you’re looking to identify interfaces which are unlikely
to change frequently</em> and which cover meaningful swaths of application
behavior.</p>
Dominance in a graph describes the property of “flow” through a node with
respect to an entry point1</a></sup>. If all paths from an entry point node must pass</em>
through a given node to reach downstream nodes, that node is said to dominate</em>
the downstream nodes.</p>
^1</sup>Dominance originated in a paper on the analysis of flow diagrams</a>.
It has been a useful concept in compiler research, in particular static single
assignment</a> form.</p>
</div>
This is the property you are looking to build tests around. This lets you
“cover” many internal interfaces while starting all of your tests at the same
point. If you’re building a REST API, you’ll usually see this along the
HTTP endpoints you define. If you’re building a React app, you’ll find this
along pages and in commonly used components</em>.</p>
These dominators help design your test suite: you can take that and walk from
the smallest sub-graph with the most overlap – like authentication in an API –
out to tests that depend on larger swaths of the system working in concert. My
druthers is to number these, like I’m building a proof of the system: if
01-authentication</code> fails, I know I can safely ignore errors from
08-frobnicate</code> until I’ve resolved them.</p>
I’ve talked about picking which interface to start</em> a test’s gamut. Mocks are a
complementary, inverse tool for determining a test’s gamut. They artificially
limit</em> the test’s gamut. Mocks require</em> knowledge of the internal interfaces.</p>
This can be fine: mocking implies knowledge external to the program about
whether the interface is likely to change: you might know that the interface to
an external API you’re using will not change. You might know that contract-based
tests are in place. You might know where the person providing that interface
sits in the office and you might be within throwing distance of them.</p>
But you’ve got to be careful: the goal of picking complementary dominators and
testing them in order is to build a proof of the system that avoids leaning on
internal interfaces. The act of using mocks pins internal interfaces.</p>
Be deliberate about the interfaces you pick to use in your test suite. It’s
worth sweating the details. You will produce a crisp definition of what it means
for the system under test to fail. These failures will have better context: why
is the failure important, what does it affect, and where did it happen first,
approximately</em>? And more context will help your agent to avoid spinning itself
in circles trying to fix over-specified tests: you can trust that it won’t
resolve the issue by rewriting the test.</p>
And that’s the end of my spiel. Aren’t you glad you sat down to read this while
your agent finished writing your tests? Go tell them to pull up a stool and read
this article, too.</p>


"The Economics of Programming Languages - Evan Czaplicki"
Unknown — Mon, 09 Oct 2023 00:00:00 +0000
Evan Czaplicki, creator of Elm, lays out a compelling explanation for the
economics of programming languages: what work goes into creating a new language
and ecosystem? How is that paid for? By whom and for what purposes?</p>
You should watch it.</p>
I want to add something about the corporate languages he lists under “rewarding talent.” In particular, Evan
points out both Dart and Golang as the result of rewarding engineering effort.
While I there’s some truth in Evan’s conclusion that the engineers behind these
languages were rewarded for their contributions, I don’t think that’s the whole
picture.</p>
Google was (and maybe still is!) deeply invested in the Java ecosystem. Oracle
acquired Sun in 2010 and filed suit against Google shortly after, but the
writing was on the wall for Sun years earlier. I suspect Golang was a hedge
against Java use at Google. Google needed a language which could fill a similar
niche as Java: similar (or better) performance characteristics, garbage
collected, and straightforward to deploy. It also needed to be easy to teach &
relatively opinionated: it would be a shift from the Java taught at
universities (and from where Google sources a lot of its incoming talent), so
it needed to scale well to newcomers. Google announced Golang in 2009, a year
ahead of the Oracle Sun acquisition and subsequent lawsuit. Rob Pike was a
known quantity who had, as Evan points out, already meaningfully contributed to
the bottom line of the company and had proven language design and documentation
chops.</p>
Less obvious, I think, is where Dart fits into all of this. I still wonder at
it myself a little bit! In 2011, it looked as if Google had put together an
all-star team to speed up JavaScript and — once they had accomplished
that goal — turned around and gave up</em> on JS. However, backing up from
that: Dart made sense as a hedged bet against JS, whose spec had only just</em>
started progressing in 2009 after ten years of stagnation. While Node.js had
introduced a working module system for the language in 2010, bundlers that
utilized that module format wouldn’t arrive until 2011 with Browserify.
Finally, as part of the Sun acquisition, Oracle had also</em> acquired the
JavaScript trademark. During the time Dart was developed, it was by no means
certain that JavaScript would rise to its contemporary adoption levels.</p>
All of that is to say: while there was an element of “reward” behind both Golang
and Dart, both language efforts also</em> made strategic sense for Google.</p>


Understanding Wasm, Part 3: You Are Here
Unknown — Mon, 04 Sep 2023 00:00:00 +0000
Understanding Wasm</h1>
Part 3: You Are Here</h2>
(This is part 3 of a series. See part 1, “Virtualization”</a> and part 2, “Whence Wasm”</a>.)</em></p>

If WASM+WASI existed in 2008, we wouldn’t have needed to create Docker.
That’s how important it is. WebAssembly on the server is the future of
computing. A standardized system interface was the missing link. Let’s hope
WASI is up to the task!</p>

Solomon Hykes via twitter</a></li>
</ul>
</blockquote>
When we left off last time, we were just about to dig into what “Operating
System Support”</strong> might mean for a virtual instruction set architecture. Solomon
Hykes claimed that if WASI —the WebAssembly System Interface— had
existed in 2008, he and his companions at dotCloud1</a></sup> wouldn’t have had
to invent Docker. Given the accomplishments of Docker over the last decade,
that’s pretty wild!</p>
To understand what a “standardized system interface” might do for us, I wanted
to first understand system interfaces</em>— how we virtualize them, how they
came to be, and how they help us work together.</p>
Does WASI represent a bridge to the future for existing applications, or a
destination?</p>
To understand the future of computing, let’s look at its past.</p>

Processes and Virtual Machines</h2>
Processes are a universal abstraction in modern operating systems. They
comprise three capabilities:</p>

A continuous view of processing resources</strong> that initially includes a
single advancing program state with the ability to spawn additional,
concurrent states against the same program within the process
(“threads”</strong>.)</li>
A contiguous address namespace</strong>, into which memory may be allocated by
the operating system upon request. This initially contains the program
instructions (or “text”</strong>), environment variables and arguments, read-only
(or “static”</strong>) data, space reserved for structures to be initialized at
startup (“block starting symbol”, or “bss”</strong>2</a></sup>), and
room for a “stack” of activation frames. A number of system libraries may
also be loaded by the operating system into this address space.</li>
A system interface</strong>, made available through a set of supervisory calls
(“system calls”</strong>.) Typically this is implemented through use of dedicated
syscall</code> or system interrupt instructions (e.g., int 0x80</code>, in x86 assembly)
available through the instruction set architecture. The use of supervisory
calls is dictated through calling conventions, including what registers must
be saved, where parameters to the syscall are placed—in memory or on registers— and how results are
retrieved after control returns to the process thread. These conventions
describe an “application binary interface”, or “ABI”</strong>.</li>
</ol>
This is a virtual</em> view of the physical machine’s resources: an “extended
machine”3</a></sup>. The operating system is responsible for enforcing the illusion
that each process operates with independent, full access to the system’s
resources. This illusion is constructed through the operating system’s careful
orchestration of the machine’s processor features — the
processor’s memory mapping hardware, interrupts and traps, mode and protection
rings, and use of privileged instructions.</p>
As widespread as the concept of a process is today, it wasn’t always so:
processes and operating systems emerged alongside</em> virtual machines in the
’60s.</p>
Hardware and software design were, at the start of the 1960s, intertwined. It
was common practice to design parallel computers —those with a number of
processors— by assigning a program to each processor, along with a range
of physical memory to be accessed by</em> that processor. If, during development,
programmers discovered that one</em> process on the system needed additional
space, all other processes had to be reprogrammed to accommodate the new memory
partitions. Similarly, if one process went idle, no other processes could make
use of that idle processor hardware. Programs had to be submitted in batches in
order to maximize utilization of the hardware. Time-sharing systems would
change that.</p>
</iframe>
In 1961, DARPA, newly flush with cash4</a></sup>, funded research on time-sharing operating
systems through “Project MAC”5</a></sup>. MIT, a participating member, brought their
“Compatible Time Sharing System” (“CTSS”</strong>6</a></sup>) with them, and along with
Bell Laboratories and General Electric, began work on a successor system called
“MULTICS”.</p>
That term we introduced earlier, “extended machine”, originated from this work.
It described a machine that was “easier to program” than the underlying
physical machine. The role of a “monitor” (or “nucleus”, or “kernel”) was to
support these extended machines by using the “bare machine interface”, or
hardware, directly. The monitor scheduled user programs to minimize costly idle
time.</p>
The extended machine typically supported virtual memory, supervisory calls, and
protection rings. Virtual memory was developed so that programs could be
written in isolation, without advance knowledge of other programs running on
the system.</p>
This was accomplished by introducing indirection: the address namespace of a
program no longer mapped directly to the physical address space of memory.
Three techniques were used to accomplish this indirection: paging,
segmentation, or the combination of the two.</p>

Expand for more on virtual memory…</summary>
With paging, both the process and physical address space are subdivided into
“pages” of memory. Each page represented several hundred “words” of
memory7</a></sup>. Accesses to “non-resident” pages in process address space
trigger hardware faults. The kernel sets traps for those faults: the fault
transfers control to the kernel, which takes action to load the missing page
into physical memory to complete the mapping. Control transfers back to the
process at this point8</a></sup>.</p>
</a>
^{Illustration from}“Segmentation and the Design of Multiprogrammed Systems”</a>, Jack B. Dennis, 1965. “N” represents the process address namespace, “M” the physical namespace. Note how the contiguous “N” namespace maps to a discontiguous namespace in “M”.</sup></p>
Memory segmentation was also used to support virtual memory. Segmentation has
the advantage of allowing larger address namespaces than the native computer
word size would otherwise be capable of representing. Consider that a 16-bit
word can only represent values from 0 to 65535. One popular segmentation scheme
addresses 20-bit values. It accomplishes this by holding another 16-bit value
in a “segment register”, shifting it left by 4 bits (multiplying it by 16) and
adding the resulting value to the base offset held in the operand register.
This allows addressing up to 1MiB of memory. When the segment register changes,
the entire segment of memory is made resident at once — which could be a
single word of memory or a significant subset of physical memory. Some hardware
supports transferring control to the kernel when the segment register changes,
which allows the implementation of virtual memory purely through segmentation.</p>
</a>
^{Illustration from}“Segmentation and the Design of Multiprogrammed Systems”</a>, Jack B. Dennis, 1965. The left diagram illustrates using a word address to index into the segment. The right diagram illustrates the namespace of segments.</sup></p>
The variable size of segments could lead to conflict —or thrash—
between processes if the two segments happened to overlap. Given the much
expanded range available to them, desktop 64-bit processors tend to use paging
by itself to implement virtual memory. Processors of the era we’re discussing
used both techniques in conjunction to combine the advantages of paging with the
ability to increase the addressable memory space.</p>
Finally, protection rings change the instruction set architecture available to
the program, trapping or disallowing use of privileged instructions having to
do with memory mapping, input/output (“I/O”</strong>) devices, or the manipulation
of software timer interrupts. The operating system runs in a protection ring
with higher privileges. Processes run in a lower ring. Processes may request
services from the operating system, like memory allocation or I/O, by making
supervisory calls. System requests are implemented using interrupt instructions
or dedicated syscall</code> instructions available in the instruction set
architecture of the processor.</p>
</details>
(For more on all of this, check out cpu.land</a> and Phil Opp’s “Writing an OS in Rust” series</a>)</em>.</p>
Project MAC’s contemporaries included Project GENIE at UC Berkeley and the IBM
System/360 and 370. These technologies were built in competition with MULTICS;
IBM found that they their university bids were being out-competed by
time-sharing systems spun out from Project MAC. This led them to develop early
whole-machine virtualization software.</p>
While extended machines made it possible to safely create and run relocatable
programs, much research revolved around the development of new operating
systems. This was more difficult than developing a program for an extended
machine, as only a single kernel could run at a time. This lead to the
development of “pseudo-machines” or “virtual machine monitors”, which used the
bare machine interface to provide many copies of that same</em> bare machine
interface for the purposes of kernel development. Further, these virtual
machine monitors could be nested</em>, so long as the appropriate resource
mappings were set up in hardware ahead of time. There was some
debate</a> around “pure” virtual
machine monitors as opposed to “impure” VMMs at this time! “Impure” VMMs
presented an extended bare machine interface to the guest; the guest machine
was aware of the virtualization.</p>
IBM implemented virtual machine monitors on the System/370 using a feature of
the processors’ protection rings. Whenever a non-privileged program performed a
privileged operation, the processor hardware would fault. The VMM trapped these
privilege faults; mapping incoming requests for resources and operations to
appropriate backing resources without the knowledge of the guest operating
system. In his July 1974 article “A Survey of Virtual Machine
Research”</a>, Robert P. Goldberg noted that the primary
difficulty in implementing efficient virtualization of machines lies in the
lack of comprehensive hardware support for trapping “privilege-sensitive”
instructions. Indeed, the month prior, he and Gerald J. Popek proposed a
definition for “virtualizable architecture” in “Formal Requirements for
Virtualizable Third Generation Architectures”</a>9</a></sup>.</p>
</a>
^{Illustration of VMM, extended machine, and interfaces from}“Survey of Virtual Machine Research”</a>, Robert P. Goldberg, COMPUTE June 1974</sup></p>
So why did virtual machine monitor research halt for so many years?</p>

The Modern Operating System</h2>
The Mansfield amendments, passed 1969 and 1973, narrowed the scope of
Department of Defense funds to projects with direct military applications. This
cut public funding for operating systems research. The General Electric 635
used to build MULTICS at MIT cost as much as a passenger jet. MULTICS had been
over-budget and behind schedule for years; Bell Labs pulled out of the MULTICS
project in 1969. The aftermath of the OS research era left many useful ideas
floating in the ether, while virtual machine monitor research would freeze
until the late 1990s10</a></sup>.</p>
Many researchers from the Project MAC days joined up at Xerox PARC. There they
developed the Xerox Alto and Smalltalk, which prefigured the modern personal
computer. By the late 1970s, commercial personal computers were available to
consumers: the Apple II, Tandy TRS-80, and others. The IBM PC launched in 1981,
the Apple Lisa in 1983, and the Macintosh in 1984. The earliest versions of
these computers only ran a single program at a time, typically in conjunction
with a disk operating system (“DOS”</strong>); later they would run many processes
cooperatively. There was only one user and thus, no need to worry about
time-sharing — these systems could be written to assume cooperation
between all programs running on the machine11</a></sup>. In 1982, Intel
released the first commercial chips capable of protected mode operation &
on-die memory mapping in the form of the 80286 processor12</a></sup>.
Microsoft, Intel, and IBM’s personal computers rapidly chipped away at the
market for time-shared minicomputers from the consumer side, while
high-performance workstations made inroads on the commercial side.</p>
For the most part, those workstations ran a variant of UNIX.</p>
Dennis Ritchie, Ken Thompson, Douglas McIlroy, and Joe Ossanna developed UNIX
at Bell Laboratories in the aftermath of Bell’s strategic retreat from the
MULTICS project. UNIX benefited from its circumstances: a small
team13</a></sup> with little supervision working on a comparatively cheap computer
(the PDP-7) proved able to incorporate the best ideas in operating systems
research rapidly. UNIX did not start out as a time-sharing system, nor was it a
preemptive operating system. It wasn’t much more than a filesystem supporting a
game at first</a>.</p>

Back around 1970-71, Unix on the PDP-11/20 ran on hardware that not only did
not support virtual memory, but didn’t support any kind of hardware memory
mapping or protection, for example against writing over the kernel. This was
a pain, because we were using the machine for multiple users. When anyone
was working on a program, it was considered a courtesy to yell “A.OUT?”
before trying it, to warn others to save whatever they were editing.</p>

“A hardware story”</a>, Dennis Ritchie, 2002</li>
</ul>
</blockquote>
The initial system was multiprogrammed — that is, there were two, time-shared
shell processes running, one for each terminal connected to the machine. When
the shell executed another program, it would read the file in over the top of
the shell code and start executing it. The exit()</code> syscall would reload the
shell program over the top of the shell and restart execution. Support for a
tree</em> of processes was added rapidly, however.</p>
UNIX inherited Conway’s fork/join</code> semantics</a> from Project GENIE
as fork/exec</code>: one command to make a duplicate of the current process as a
child of the forking process, and a second command run in that child process to
replace the duplicate with the target code. This model of copying processes
using prototypal inheritance directly enabled the container models we’ll talk
about shortly.</p>
By 1977 work was underway to port UNIX from the PDP-11 to the Interdata 8/32,
as we noted way back in the first post in this series</a>. (The work had
been validated by porting a copy of UNIX to an IBM System/370 virtual machine.)
With the support of Sun Microsystems, IBM, HP, and others, UNIX and C had swept
through the industry by the late eighties. Consumer operating systems were
converging on preemptive, protected time-sharing, buoyed by advances in Intel’s
commercial hardware. Hardware virtualization features, if supported, primarily
existed to support older DOS programs. Entering the 90’s, “virtual machine”
came to refer primarily to language virtual machines, like Smalltalk and
Erlang.</p>

The Internet</h2>
The internet changed things (he said, blandly.)</p>
The internet meant unbound connectivity, which meant servers had to support
unprecedented numbers: of processes, of users, of connections. The added
pressure was not tenable with contemporary operating systems approaches, which
were too rigid. Both scaling and consolidation were difficult and relocating
processes was nearly impossible.</p>
Going into the 90’s, the ever-problematic, ever-popular x86 ISA was thought
impossible to virtualize efficiently, but pressure was building behind that
cork.</p>
These problems spurred a renaissance of virtualization research.</p>

Modern Virtual Machines</h2>
Scaling a website to meet traffic demands meant procuring and provisioning
hardware. This could not be done quickly. Internet companies were required to
expend considerable money well in advance of expected traffic and were largely
unable to recoup costs if that traffic did not materialize. The industry needed
a way to commoditize hardware.</p>
x86 was the most popular processor architecture with the best economies of
scale and broadest software support. However, it could not be virtualized
directly: privileged instructions and memory operations would silently fail
without triggering traps.</p>

Unfortunately, many current architectures are not strictly virtualizeable.
This may be because either their instructions are non-virtualizeable, or they
have segmented architectures that are non-virtualizeable, or both.</p>
Unfortunately, the all-but-ubiquitous Intel x86 processor family has both of
these problematic properties, that is, both non-virtualizeable instructions
and non-reversible segmentation. Consequently, no [Virtual Machine Monitor]
based exclusively on direct execution can completely virtualize the x86
architecture.</p>

“Virtualization System Including a Virtual Machine Monitor For a Computer
With Segmented Architecture”</a>, United States Patent #6,397,232</li>
</ul>
</blockquote>
The late 90’s saw a growing interest in direct binary translation (“DBT”</strong>.)
The hegemony of the x86 processor was predicted to crumble. Efforts such as
DAISY and Transmeta’s Crusoe bet against the continued popularity of x86.
Meanwhile, in the language virtual machine space, just-in-time compilation
research blossomed. VMWare was founded in this context, submitting a
patent</a> early on for a direct-binary-translation method of implementing
virtual machine monitors for the x86 architecture.</p>
VMWare’s virtual machine monitor triggered a translator whenever control
entered or exited a protection ring. This translator monitored for execution of
untrapped privileged instructions in order to inject manual VMM trap calls. This
enabled virtualization but added overhead to every system call.</p>
Because DBT added uneven overhead to kernel execution, it was difficult to
associate system use with virtual machine execution. This prevented accurate
billing systems from being constructed around such virtual machines.</p>
“Xen and the Art of Virtualization”</a> changed all of that.</p>

By allowing 100 operating systems to run on a single server, we reduce the
associated costs by two orders of magnitude. Furthermore, by turning the
setup and configuration of each OS into a software concern, we facilitate
much smaller-granularity timescales of hosting.</p>

“Xen and the Art of Virtualization”</a>, 2003</li>
</ul>
</blockquote>
Xen achieved this through paravirtualization: instead of pure virtualization,
it achieved “impure”, cooperative virtualization by modifying the guest operating
systems. In particular, Xen moved the operating system out</em> of the most
protected ring of the processor: from ring 0 to ring 1. Processes continued to
run in ring 3. This gave the operating system its own “extended machine”: while
it lost access to privilege instructions, it gained access to its own
supervisory call system (“hypervisor calls”.)</p>
</a>
^{Comparison of native Linux, Xen, VMWare Workstation, and User-mode Linux on various benchmarks, from}“Xen and the Art of Virtualization”</a>. Note that OLTP represents relational database workloads and WEB99 represents web-serving.</sup></p>
In particular, Xen’s performance on web application workloads was a breakthrough.</p>
Cloud virtual machines commoditized hardware, severing the connection between
“procuring and provisioning hardware” and “scaling a web service.”</p>
As with memory mapping and protection rings in the 80s, consumer hardware
lagged behind the market’s needs. AMD and Intel introduced hardware support
for virtualization of x86(_64) architectures in 2005 through SVM and VT-X,
respectively. ARM added hardware virtualization support in Cortex-A in 201114</a></sup>.</p>

Hardware is really just software crystallized early. It is there to make
program schemes run as efficiently as possible. But far too often the
hardware has been presented as a given and it is up to software designers to
make it appear reasonable. […]</p>
As Bob Barton used to say: “Systems programmers are high priests of a low
cult.”</p>

“An Early History of Smalltalk”</a>, Alan Kay</li>
</ul>
</blockquote>
Hardware virtualization support allowed operating systems to integrate
hypervisor capabilities: Windows Server added Hyper-V (2008); Linux, Kernel
Virtual Machines (“KVM”, 2006); and macOS, Hypervisor.framework (2020). (EC2
started moving from Xen to KVM-based “Nitro” virtual machines in 2017. Brendan
Gregg wrote more about the path from the first version of Xen to Nitro
here</a>.) In lieu of running Xen or VMWare, consumer operating systems
themselves became capable of running guest operating systems, accelerating the
development of new hypervisor software.</p>

A few notes on timing: by 2003, the dotcom bubble had burst. The servers that
internet startups had loaded up on flooded the market with cheap used hardware,
causing Sun Microsystems to hemorrhage money. At Amazon, Benjamin
Black</a> circulated a document describing a standardized
infrastructure; Bezos tasked Chris Pinkham with developing this in 2004. EC2
launched publicly in 2006, powered by Xen</a>. According to Steve
Yegge</a>, sometime in 2002-2003 or so, Bezos issued his famous “services
edict”, stipulating that all Amazon engineering teams deliver their work in the
form of networked services. By 2009, Netflix had moved their video encoding
operations to AWS. Netflix finished transitioning to the cloud in 2011. And
Docker launched in 2013.</p>

Docker</h2>
The high-level metaphor of Docker is that of shipping containers</strong>.
Standardized containers revolutionized the shipping industry: a standard form
factor meant a standard way to load, unload, and transport material. The
technology made shipping everything</em> faster, cheaper, and more reliable. This
is Docker’s raison d’être: to do for computer applications and operations teams
what standard containers did for the shipping industry.</p>
The metaphor is pretty durable: the concrete problem that Docker solved was
that of shared resources. Consider: given a large company, two developers may
develop two different web serving applications (“services”</strong>) that expect to
listen on the same network port15</a></sup>. Before containers, these two developers
would have to sync up: who gets to listen on port 80? They’d have to agree,
up-front, before being able to deploy their software. Worse, if they couldn’t
come to agreement about the use of resources, they’d have to involve a
third-party — someone from the operations department — who might
solve the problem by introducing a third</em> server, a reverse proxy layer, to
resolve the conflict.</p>
Docker solves this by giving each service a virtualized</strong> view of the
operating system and its resources. Both developers may remain agnostic of each
other AND the operations team. The operations team can map network ports as
necessary – or rely on orchestration software, like Nomad or Kubernetes, to do
this for them. Standard containers yield standard tools.</p>
Docker achieves16</a></sup> virtualization through a combination of abstract
programming interfaces (“APIs”</strong>) supported by the kernel of the host
operating system: secure computing mode (“seccomp”</strong>), namespaces, and
control groups — on which, more later. Each of these APIs must be
properly configured to make durable the illusion that the process is running on
a virtualized system.</p>
Windows supports containers natively through a hcsshim</code> runtime that provides
namespace and cgroup support through Windows’ native Host Compute Service.
Apple’s macOS, however, runs docker containers by spinning up a virtualized
Linux host to run a Docker daemon and container processes.</p>
While the Docker/OCI17</a></sup> container model is ubiquitous today, the
capabilities underpinning it have their origins in the virtual machine
renaissance of the late 90s.</p>

Let’s walk back towards the year 2000.</p>
Docker was originally implemented using an earlier container model called “LXC”
(“Linux Containers”), introduced in 2008. LXC, like many of its contemporary
container models, was built to act like a virtual machine. Rather than Docker’s
model of taking a single process and giving it a virtual system environment,
LXC containers typically virtualized an init</code> process (like systemd</code>,
upstart</code>, or SysV init</code>18</a></sup>.) This made them “feel” much more like a
virtual machine, with their own long-lived daemons, periodic tasks, and system
logs. But, akin to Docker, LXC was built from cgroups</code> and namespaces.</p>
Google contributed cgroups</code> (or “control groups”) to the Linux kernel in 2007.
Control groups allowed userland to communicate resource quotas for process
subtrees to the kernel: effectively allowing container runtimes to dictate the
maximum CPU time, parallelism, network, memory, and disk usage a set of
sub-processes should be able to use. This doesn’t affect the ability of those
processes to see</em> certain subsystems: control groups don’t affect what the
process has access to, only the available quality of service for that
access19</a></sup>.</p>

I consulted the local gurus about the security of a chroot</code> environment. Their
conclusion: it is not perfectly secure, but if compilers and certain programs
are missing, it is very difficult to escape.</p>

“An Evening with Berferd”</a>, Bill Cheswick, 1990</li>
</ul>
</blockquote>
Namespaces are related to earlier jail</code> and chroot</code> capabilities20</a></sup>.
This family of APIs control what operating system subresources are visible to
jailed processes. Linux began adding namespace support in 2002. The mount
namespace for filesystems came first, inspired by Plan 921</a></sup>. Eric W.
Biederman enumerated the necessary support in “Multiple Instances of
the Global Linux Namespaces”</a>:</p>

mnt</code>: The filesystem namespace.</li>
uts</code>: The UNIX Time-sharing namespace (controlling what hostname is visible to the process.)</li>
ipc</code>: Inter-process communication namespace.</li>
net</code>: The network namespace.</li>
pid</code>: The process identifier namespace.</li>
user</code>: The user and group namespace.</li>
time</code>: The time namespace.</li>
</ul>
Linux added the user</code> namespace in 2013 — the same year Docker was first
released publicly. (This was later followed up with the cgroup</code> namespace in
2016.)</p>
Namespaces and cgroups form the basis of what we think of as containers
today22</a></sup> — and they exist to work around the lack of virtualization
support in turn-of-the-century processors.</p>

From the hardware perspective, processes and guest operating systems look
nearly the same. Virtualizing the system at the process layer means sharing
more code between the virtualized systems with a finer granularity of
abstractions, but has the advantage of being achievable purely through
software: hardware can’t really tell the difference between a container and any
other process.</p>
Because virtualizing the system at the hardware interface layer is more
coarse-grained with less shared code, VMMs are generally considered safer
targets for multi-tenancy —that is, running untrusted code from third
parties collocated on the same hardware23</a></sup>. On the other hand,
VMMs introduce more startup overhead. However, in recent years, containers
and VMMs have converged: Kata containers, AWS’ Firecracker, and others use the
container user interface but run the containers inside lightweight virtual
machines, achieving remarkable performance, security, and
density24</a></sup>.</p>

Density</h2>
The more efficiently tasks can be collocated, the better the margins on
equipment; this is a competitive edge for a hosting company. Processes, whether
virtual machines or containers, have overhead. Switching between processes
takes time and memory; Tyler McMullen, CTO of Fastly, noted that were limits on
the number of processes running on a single box in his 2018 “Software Fault
Isolation”</a>. This was a natural place to look for improvements, and
so content delivery networks — which handle some of the highest volume of
traffic on the internet — started digging into the problem. Fastly and
Cloudflare both landed on web technologies, launching products in 2018.</p>
Cloudflare launched Workers, which co-locates many user tasks in a single
process using V8 isolates. V8 is, as we discussed previously, the JIT
JavaScript engine open-sourced by Google as part of their Chrome browser.
Workers customers upload JavaScript applications that implement the
ServiceWorker spec; Cloudflare deploys the application to their edge network, with points
of presence across the globe. As a bonus, since WebAssembly support is
available through the web platform API, users that really needed to could
target Wasm. (Though they’d be responsible for writing their own bindings to
the ServiceWorker API.)</p>
Fastly launched Compute@Edge, skipping JavaScript in favor of WebAssembly. But
this posed a problem. While JS could rely on the Web platform as a system
interface, Wasm does not automatically provide one. In order to hook Wasm up to
their edge compute, Fastly had to define</em> an interface: one that was stable
and versioned.</p>

WASI</h2>
(Finally.)</em></p>
Fastly, Intel, and Mozilla teamed up in 2019 to form the Bytecode Alliance.
Their goal was to define a specification for a standard system interface for
WebAssembly25</a></sup>.</p>
The WebAssembly ISA allows imports and exports: functions that the host can
pass in</em> to the Wasm module for internal use, and functions that the Wasm
module can hand back to the host to be called by</em> the host on demand. The host
may also provide a chunk of memory for the Wasm module to operate on. Imported
and exported functions may only take primitive values — integer and floating
point values of various widths. The host and module may cooperate to transfer
more complicated types. For example, the host and module might agree that a string
is represented as an integer pointer into the module’s memory along with an
integer length.</p>
This is a lot</em> like the sort of ABI</strong> a process System Interface</strong> uses.</p>
Tools like emscripten and wasm-bindgen treat the interface between the host and
Wasm module as an internal detail. Those tools were designed to take an
existing application and get it running in a browser with minimal changes,
generating both Wasm, HTML, and the JavaScript to integrate the Wasm with the
Web Platform. If one were to recompile an application with a newer version of
emscripten and try to drop the Wasm onto an older version of the HTML and JS,
it may not work. Likewise, the Wasm from an application compiled with
wasm-bindgen could not be dropped onto emscripten HTML and JS or vice versa.
They are not ABI</strong> compatible.</p>

The first preview of WASI launched in 2019. For the most part, it closely
resembles POSIX with some light editorialization: for example, POSIX APIs whose
behavior changed based on parameters, like unlink</code>, were split into separate
functions. However, WASI preview 1 deferred implementation of several key
capabilities: C++ stack unwinding/exception support, full network socket
support, and fork</code>/exec</code> were omitted.</p>
fork</code> and exec</code> form the bedrock of the POSIX process model. Their
descendants, clone</code> and unshare</code>, form the basis for working with Linux
cgroups and namespaces. In short, it is hard to support fork</code> without also
taking the traditional process model on board. And WebAssembly may not want to
do that</a>.</p>
System interface ABIs don’t just define the boundary between a process and the
kernel. They’re also used to enable software linking</em> — the reuse of
compiled artifacts: shared objects (.dylib</code>, .so</code>, or .dll</code>.) Linking can be
performed at compile time (static linking) or at process start (dynamic
linking.) While shared object linking wasn’t defined as part of the first WASI
preview26</a></sup>, neither did the specifiers wish to preclude it in
future versions of the specification.</p>
The standard approach to sockets would have exposed too much power between
linked modules. Sockets are typically represented as a “descriptor” or “handle”
in the form of an integer the operating system hands to the process on request.
Operations on the socket are performed by making a system call referencing that
integer, and all linked modules are assumed to have access to the same system
interface. This model is difficult to secure: malicious linked modules can
“guess at” active descriptors by picking random integers27</a></sup>. The system
can’t differentiate between linked modules and the core application logic when
receiving calls referencing the descriptors.</p>
The problem with the POSIX model is that the system interface forms a
monolithic wall between the kernel and the user process, leaving
undifferentiated space on either side. The ABI emphasizes the importance of the
system interface over</em> the importance of the interface between modules.</p>

A Component Model</h2>
Through the Wasm Component Model</strong>, WASI preview 2 fundamentally rethinks the
process model.</p>

One of WebAssembly’s unique attributes is the ability to run sandboxed without
relying on OS process boundaries. Requiring a 1-to-1 correspondence between
wasm instances and heavyweight OS processes would take away this key advantage
for many use cases. Fork/exec are the obvious example of an API that’s
difficult to implement well if you don’t have POSIX-style processes, but a lot
of other things in POSIX are tied to processes too. So it isn’t a simple matter
to take POSIX, or even a simple subset of it, to WebAssembly.</p>

“Why not a more traditional set of POSIX-like syscalls?”, WASI Rationale</a> - Dan Gohman</li>
</ul>
</blockquote>
The Wasm Component Model</strong> defines a static linking approach for combining
many Wasm core modules into a single file, a “component”. The Component
Model</strong> also a new, modular interface definition language called “WebAssembly
Interface Types” (“WIT”</strong>.) WIT allows for the definition of named functions,
groups of functions, and high-level types like strings and structs. WIT also
allows for groupings-of-groupings with requirements around exports and imports,
called “worlds”. WIT worlds can be used to generate host code or as a target
for compiling shared code — it is a language for defining contracts
between modules.</p>
Instead of one monolithic wall of functions representing all system services,
the Wasm Component Model proposes a system of smaller fences placed between
every module. Each interface only specifies the functionality it needs, and may
be fulfilled by any other module — or the host. Instead of exposing
entire namespaces of functionality, like mnt</code>, uts</code>, or net</code>, the interface
can be described in a fine-grained way: “module A requires a function for
reading input data.”</p>
This allows sockets to be represented as higher level objects whose full
capabilities aren’t transferred between linked modules, as opposed to an
integer descriptor tracked by the host. (See Dan Gohman’s excellent “No
Ghosts!”</a> for more on this.)</p>
WASI preview 2 aims to stabilize wasi-cli</code></a> and
wasi-http-proxy</code></a> worlds first. However, at the time of writing the
Component Model proposal is only in stage 1</a>, so many popular tools
and runtimes do not support Wasm components or WASI preview 2. And many
platform-as-a-service (“PaaS”</strong>) startups have appeared since Solomon Hyke’s
tweet.</p>
Companies have proposed alternative system interfaces.</p>
Deis labs introduced WAGI in 2020. WAGI sidesteps the issue of missing POSIX
support by using WASI preview 1’s standard input and output instead of sockets.
This is reminiscent of the venerable “common gateway interface” (“CGI”</strong>)
from the 1990s. Fermyon, a Wasm PaaS company, supports WAGI through its Spin
framework.</p>
In 2023, Wasmer introduced WASIX. WASIX, a superset of WASI, aims to target
more of the POSIX API, including support for fork</code>. This is akin to an
Emscripten for server applications, with the goal of lowering the barrier
between existing web services and PaaS Wasm companies. Like Emscripten, WASIX
values easy portability of existing applications. As a result, it doesn’t
editorialize too much on the POSIX API. However, at the time of writing, WASIX
doesn’t support shared object linking. While WASIX doesn’t preclude linking
shared objects in the future, it seems likely that it would follow the existing
linking model, whether for good or for ill.</p>

Every technology we’ve talked about made it easier for developers to
collaborate by reducing their need to coordinate ahead of time. Interfaces are
a contract, a minimum viable agreement, between two parties. Interfaces build
two-sided network effects: implementing the contract lets your program work
with an unbound number of other programs which consume the contract and vice
versa. Interfaces are living systems: they have internal motion. Inertia.</p>
The inertia of an interface may even overcome deep flaws with that interface.</p>
In this series, we’ve talked about how the inertia of the x86 ISA spurred an
unexpected breakthrough in hardware design — despite the bet that we’d
all be using VLIW RISC processors today. How the inertia of JavaScript, through
the web platform, precluded competing web technologies, like Java, PNaCL, and
Dart, and eventually gave rise to WebAssembly. This is the heart of “Worse is
Better”</a> and</strong> Gall’s Law</a>: “A complex system that
works is invariably found to have evolved from a simple system that worked.”</em>
It’s the human side of computing: it is better to be in a room full of people
gathered around the warmth of an imperfectly useful interface than it is to be
out, alone in the cold, with a perfect interface.</p>
Good interfaces are grown, not invented: coaxed out of usefully-bad interfaces
by redirecting some of their inertia. Borrowing the energy of the inertia to
achieve a new orbit, so to speak.</p>
Processes and virtual machines are some of our oldest, most widely-used
usefully-bad interfaces. Their use at scale — and our ability to reason
about their capabilities — have begun to fail us, but they are the mass
around which all of modern computing orbits. This is what’s interesting about
WASI and the WebAssembly Component model: it is a moonshot. The initial
trajectory won’t get WASI to the moon: WASI preview 2, if successful, still
looks like embedded Wasm in web proxies and command lines. But that might be a high
enough apogee to slingshot to a new, higher orbit.</p>
On the other hand, moonshots are not a sure thing: like Java before it, WASI
doesn’t represent a bridge, but a destination. That is the fundamental difference
between the component approach and the WASIX approach.</p>
For my part, I think this WASI moonshot warrants energy: Docker succeeded, in
part, because it described a vision of the future that anyone could take part
in building. Through npm, JavaScript had a similar trajectory. I think we’re on the
cusp of the same moment with WebAssembly.</p>

Epilogue</h2>
So, whew. That’s a lot.</p>
I’m excited to announce that I’ll be joining Dylibso</a> as of this
month to work on Wasm materials, tools, and smooth out friction in the
ecosystem wherever I find it.</p>
Working on these posts has been hugely educational for me on a number of
levels, and I’d like to thank everyone who reviewed these posts (C J Silverio,
Eric Sampson, and Aria Stewart), advised and helped source research (Ron Gee,
Dan Gohman) and encouraged me. In particular I’d like to thank my family for
their support: my wife, Krysten, and my parents, Mark and Sue. They dealt with
an entire summer of me talking non-stop about Wasm, writing, and computing
history.</p>

Bibliography and Timeline</h2>
So many PDFs this time around!</p>
I’d like to call out “The Ideal Versus the Real: Revisiting the History of
Virtual Machines and Containers”</a> by Allison Randal, which dives
more deeply into the history and interrelations between these technologies than
my effort here. Give it a read!</p>

1961. “Dynamic Storage Allocation in the Atlas Computer”</a>, John Fotheringham,</li>
1961. “One-level storage system”</a>, Kilburn, Edwards, Lanigan, Sumner</li>
1961. Compatible Time-Sharing System first operational</li>
1962. Atlas Computer operational</li>
1963. “A multiprocessor system design”</a>, Melvin E. Conway</li>
1964. Compatible Time-Sharing System in operational use</li>
1964. IBM System/360</li>
1965. GE 645 hardware simulated at MIT for MULTICS</li>
1965. “Segmentation and the Design of Multiprogrammed Computer Systems”</a> - Jack B. Dennis

cites 1961 “one-level storage system”</li>
</ul>
</li>
1966. “Programming Semantics for Multiprogrammed Computations”</a> - Jack B. Dennis, Earl C. Van Horn

cites 1963 “A multiprocessor system design”</li>
</ul>
</li>
1967. GE 645 hardware received at MIT</li>
1968. “Virtual Memory, Processes,and Sharing in MULTICS”</a> - Robert C. Daley, Jack B. Dennis</li>
1969. UNIX v1 running on a pdp-7</a></li>
1969. MULTICS replaces GECOS on GE 645</li>
1970. “Virtual Memory”</a>, Peter Denning

cites 1961 “one-level storage”</li>
cites 1961 “dynamic storage allocation”</li>
cites 1966 “Programming Semantics for Multiprogrammed Computations” Jack B. Dennis, Earl C. Van Horn</li>
</ul>
</li>
1970. IBM System/370 released</li>
1971. UNIX moved to the PDP-11</a></li>
1973. “Virtual machine or virtual operating system”</a>, Bellino, J, C Hans

this seems a LOT like paravirtualization and/or containers — could not achieve full performance
virtualizing an entire IBM 360 machine, so the VMM provided “higher level” primitives</li>
</ul>
</li>
1974. “Survey of Virtual Machine Research”</a>, Robert P. Goldberg

cites bellino/hans 1973 re: “pure” vs “impure” virtual machines</li>
</ul>
</li>
1979. UNIX v7, introduction of chroot</code></li>
1979. “Software Engineering in 1968”</a> - B. Randell</li>
1981. “The Origin of the VM/370 Time-Sharing System”</a> - R. J. Creasy</li>
1982. chroot</code> added to BSD</li>
1998. immunix apparmor introduced</li>
1998. VMWare x86</li>
1999. Resource Containers</a></li>
2000. FreeBSD jails</a></li>
2000. SELinux first released OSS</li>
2001. linux vserver</li>
2001. VMWare releases gsx</a>, first x86 server virtualization product</li>
2002. VMWare patent - “Virtualization System Including a Virtual Machine Monitor For a Computer With Segmented Architecture”</a>, United States Patent #6,397,232</li>
2002. linux namespaces (2.4.19); mount namespace</li>
2002. linux security modules</li>
2003. “Xen and the Art of Virtualization”</a></li>
2003. SELinux merged into linux kernel (2.6.0-test3)</li>
2003. Work on AWS EC2 kicks off</a></li>
2004. Solaris Zones</a></li>
2005. OpenVZ (open virtuzzo)</a></li>
2005. intel vt-x, amd-v</a></li>
2006. Slicehost launched</li>
2006. “Multiple instances of the global linux namespaces”</a> - Eric W. Biederman</li>
2006. AWS EC2 first released - built on xen virtualization</li>
2006. “Virtualization in Linux”</a> - Kirill Kolyshkin</li>
2007. “Adding Generic Process Containers to the Linux Kernel”</a> - Paul B. Menage, Google</li>
2007. cgroups lands in linux kernel (2.6.24); final form of google process containers</li>
2007. KVM lands in the linux kernel (forked QEMU added KVM support)</li>
2008. LXC released</li>
2008. Microsoft launches Hyper-V for windows server</li>
2009. AWS VPC launch, ELB, Autoscaling, Cloudwatch</li>
2009. Netflix moves movie encoding to the cloud</li>
2009. Canonical adopts apparmor</li>
2010. Netflix moves account signup, etc to the cloud</li>
2010. Hyper-V support merged into libvirt</a>



In 2010, Bolte et al. incorporated support for Hyper-V into
libvirt, so it could be managed through a standardized interface,
together with Xen, QEMU+KVM, and VMware ESX.</p>
</blockquote>
</li>
</ul>
</li>
2011. Netflix’s entire operation is on the cloud</li>
2012. KVM support merged into mainline QEMU</a></li>
2013. user namespaces land in the linux kernel (3.8); this completes support for containers.</li>
2013. Docker released (based on LXC)</li>
2013. LMCTFY (“Let me contain that for you”)</li>
2014. Kubernetes released</li>
2014. Docker Swarm released</li>
2014. docker-compose v1</li>
2014. Docker replaces LXC with libcontainer</code></li>
2014. Canonical begins work on lxd</code>, orchestration for LXC containers</li>
2015. Terraform released</li>
2015. Consul released</li>
2015. Hashicorp releases first version of vault, nomad</li>
2015. Docker spins runc</code> out of its container runtime</li>
2016. dirty cow container CVE</a></li>
2016. CRI-O launched</li>
2016. docker-compose v2</li>
2016. LXD released</li>
2016. “AWS re:Invent 2016: Netflix: Container Scheduling, Execution, and Integration with AWS (CON313)”</a></li>
2017. docker-compose v3</li>
2017. “The Evolution of Container Usage at Netflix”</a></li>
2018. Kata, gVisor, Nabla</li>
2018. “Software Fault Isolation and Edge Computing | Altitude San Francisco 2018”</a> - Tyler McMullen</li>
2019. CRI-O handed over to cncf</li>
2019. Docker Swarm start of 2-year EOL</li>
2019. “A fork() in the road”</a> - Andrew Baumann, Jonathan Appavoo, Orran Krieger, and Timothy Roscoe</li>
2020. “The Ideal Versus the Real: Revisiting the History of Virtual Machines and Containers”</a> - Allison Randal</li>
2023. “WASI Command and Reactor Modules”</a> - Rob Wong, Zach Shipko, Gavin Hayes</li>
</ul>

^{1</sup>
Which would later be rebranded as “Docker”, after their most famous
product.</p>
</div>}^{2</sup>
“BSS” is also known as “better save space”.</p>
</div>}^{3</sup>
Hey, there’s that “machine” word again.</p>
</div>}^{4</sup>
Which had nothing at all</em> to do with Yuri Gagarin’s recent orbital trip, I’m sure</em>.</p>
</div>}^{5</sup>
“MAC” expanded to “Mathematics and Computation” early on, later
expanding to “Multiple Access Computer”, “Machine Aided Cognitions”, and
“Man and Computer”. It also funded the MIT AI lab.</p>
</div>}^{6</sup>
MIT also originated the “Incompatible Time Sharing”, which would give
us EMACS.</p>
</div>}^{7</sup>
A “word” in this case refers to the number of bits the hardware was optimized to process.
You’re probably reading this on a device with a native word size of 64-bits, but in the past 32-
and 16-bit word sizes were common. In the distant past, you might see 36-bit words!</p>
</div>}^{8</sup>
There are are all sorts of neat tricks that paging enables, including “copy on
write” pages – mapping the same memory to different places in the same
address space, and only creating copies of them when they’re mutated.</p>
</div>

^{9</sup>
“Third Generation Architecture” refers to the generation of
computers designed in the 1960’s using early integrated circuits; these are
typically called “minicomputers”. They were succeeded by fourth generation
architecture in the early 1970’s which began to use microprocessors.</p>
</div>
Even today, virtualizable architectures can be set to meet “Popek and Goldberg
virtualization requirements.”</p>

^{10</sup>
IBM continued to ship virtual machine monitor systems throughout this
period. However, they were primarily focused on virtualizing earlier
technologies, like mainframes and minicomputers, on top of newer hardware.</p>
</div>

^{11</sup>
Now, that said, that assumption was frequently (and spectacularly) invalidated.</p>
</div>

^{12</sup>
Gates called these processors “brain dead”. At the time,
IBM and Microsoft were co-developing OS/2 for the IBM PC, and IBM wanted to
target the 286. However, the protection modes of the 286 were “one-way” –
once the processor entered protected mode, re-entering real mode required
restarting the system.</p>
</div>

^{13</sup>
The benefits of a small team are made evident through this quote, which
is a prime example of Conway’s Law</a>:</p>
</div>

Where under Unix one might say</p>
ls >xx</span></span></code></pre>
to get a listing of the names of one’s files in xx, on Multics the notation was</p>
iocall attach user_output file xx</span></span>
list</span></span>
iocall attach user_output syn user_i/o</span></span></code></pre>
Even though this very clumsy sequence was used often during the Multics days,
and would have been utterly straightforward to integrate into the Multics
shell, the idea did not occur to us or anyone else at the time. I speculate
that the reason it did not was the sheer size of the Multics project: the
implementors of the IO system were at Bell Labs in Murray Hill, while the
shell was done at MIT. We didn’t consider making changes to the shell (it was
their program); correspondingly, the keepers of the shell may not even have
known of the usefulness, albeit clumsiness, of iocall. […]</p>
Because both the Unix IO system and its shell were under the
exclusive control of Thompson, when the right idea finally surfaced, it was a
matter of an hour or so to implement it.</p>
“The Evolution of the Unix Time-sharing System”</a>, Dennis M. Ritchie, 1996</p>
</blockquote>

^{14</sup>
Indeed, ARM Cortex’s virtualization support is specifically marketed as
meeting “Goldberg and Popek virtualizability requirements”!</p>
</div>}^{15</sup>
A port is a numbered resource, managed by an operating system,
representing a stream of incoming or outgoing network requests.</p>
</div>}^{16</sup>
Docker comprises a long-lived process (or “daemon”, in this case
called containerd</code>) for running these virtualized containers (via runc</code>
or crun</code>), a user interface for controlling</em> those containers, a registry
protocol for sharing</em> container images, and a file format describing how
to build</em> those container images. The file format for building docker
images, a Dockerfile</code>, specifies a series of commands that construct a
container image; each command forms a distinct, content-addressable
“layer”. Dockerfile</code>s may source other Dockerfile</code>s during the build
process, common layers are reused between builds.</p>
</div>}^{17</sup>
In 2013, dotCloud rebranded themselves as Docker in an attempt to
capture some of the value of this ecosystem. To ensure trust that the
ecosystem would outlive the company, Docker and CoreOS formed the “Open
Container Initiative” in 2015. As a result, there are alternative
OCI/Docker runtimes: crio</code> and podman</code>, for example.</p>
</div>}^{18</sup>
Init processes are responsible for setting up the userland system:
they are the root of the tree of userland processes; setting up daemons for
resolving DNS, networking, devices, filesystem mounts, and more.</p>
</div>}^19</sup>This control allowed Google to more efficiently allocate shared
resources using their internal orchestration software, AKA “Borg”</a>.</p>
</div>

^20</sup>The earliest jail-like capability, chroot</code>, was added to AT&T
Unix in 1979. It was also added to the Berkeley Software Distribution
(“BSD</strong>”28</a></sup>) in 1982.  chroot</code> allowed a process to “pivot” the root
directory to a subdirectory, effectively hiding parent directories from a
process. This offered incomplete protection</a>, thus BSD
introduced the jail</code> system call in 2000.</p>
</div>

In the case of the chroot(2)</code> call, a process’s visibility of the file
system name-space is limited to a single subtree. However, the
compartmentalisation does not extend to the process or networking spaces and
therefore both observation of and interference with processes outside their
compartment is possible.</p>
To this end, we describe the new FreeBSD ‘Jail’ facility, which provides a
strong partitioning solution, leveraging existing mechanisms, such as
chroot(2)</code>, to what effectively amounts to a virtual machine environment</em>.</p>

“Jails: Confining the omnipotent root.”</a>, Poul-Henning Kamp, Robert N. M. Watson, 2000</li>
</ul>
</blockquote>
Jails did not address resource management or scheduling concerns. Sun addressed
this in 2004 with Solaris’s “Zones”, which provided an Docker-like experience
years in advance. However, as we mentioned, Sun fell on hard times during the
2000’s, eventually meeting its demise in 2010 after being acquired by Oracle.
As a result, Solaris didn’t experience the widespread adoption that various
Linux distributions enjoyed during this time.</p>

We have been gratified when casual users mistake the technology for a virtual machine.</p>

“Solaris Zones: Operating System Support for Consolidating Commercial Workloads”</a></li>
</ul>
</blockquote>

^{21</sup>
Plan 9! Which, as you’ll recall from the last article, Java
killed! In the 80s and 90s, “virtual machine” in the sense of “system
emulation” had withered so far as to be supplanted by “virtual machine”
meaning “language model runtime.”</p>
</div>}^22</sup>Ok, I’m adding this one as a footnote because this is a long post
already. I’m only going to gesture at linux-vserver, which was focused on scaling
webservers through containerized networking. However, Virtuozzo (and its successor project,
OpenVZ) approached containers for an entirely different reason: to enable
checkpoint/restore of work on high-performance batch clusters. This would allow
relocation of processes between computers (“nodes”) in a cluster by namespacing
all of their system resources. This required maintaining patches against the linux
kernel at the time, so as far as I can tell it never really took off, but it did
spawn the “Checkpoint and Restore in Userspace” (“CRIU”</strong>) project</a>.</p>
</div>

^23</sup>This is notwithstanding the Meltdown and Spectre</a> vulnerabilities.
Meltdown exploits a race condition between memory access and privilege checking and affects operating
systems and hypervisors. Exploits allow processes and VMs to read memory across security boundaries,
effectively breaking the illusion of virtual memory. Spectre exploits speculative execution —
a property of modern superscalar processors. Speculative execution executes every code path leading
out from a branch point, throwing away the results from the paths not taken. However, this speculative
execution can affect caches, so the path not taken may be observed by measuring operation timings after
the fact.</p>
</div>

^24</sup>For an in-depth look at how container and virtualization
approaches compare in terms of performance on various axes, check out “A
Fresh Look at the Architecture and Performance of Contemporary Isolation
Platforms”</a>.</p>
</div>

^{25</sup>
And to develop the Cranelift, Wasmtime, and Wasm Micro
Runtime (“WAMR”</strong>) projects.</p>
</div>}^26</sup>WASI preview 1 instead classes modules as “reactors” or
“commands”. You can read more on this on Dylibso’s blog</a>.</p>
</div>

^{27</sup>
This is known as “forging” a descriptor.</p>
</div>

^{28</sup>
It’s out of scope for this post, but suffice it to say that UNIX
split in the 80’s and 90’s: roughly, Linux, BSD, and SysV. Linux provides
the kernel of popular distributions like Redhat, CentOS, Ubuntu, Debian,
and Android. BSD provides a specification for a kernel for operating
systems: most conspicuously, Apple’s modern operating systems and SunOS.
SunOS’s successor, Solaris, was based on AT&T’s UNIX System V, along with
HP’s HP-UX and IBM’s AIX.</p>
</div>}}}}}}}}


Understanding Wasm, Part 3: A System Interface
Unknown — Thu, 31 Aug 2023 00:00:00 +0000
Understanding WASM</h1>
Part 3: A Virtual System</h2>
(This is part 3 of a series. See part 1, “Virtualization”</a> and part 2, “Whence WASM”</a>.)</em></p>
It’s not enough for a virtual architecture, like WebAssembly, to be portable
across physical processors: after all, if that’s all it could do we’d be
limited to calculating numbers. Chris Lattner’s “Low-level Virtual
Architecture”</a> established a handy rubric that we’ve been using to judge
WebAssembly as a Virtual Instruction Set Architecture. WebAssembly has:</p>

✅ Simple, low-level operations.</strong></li>
✅ No execution-oriented features.</strong></li>
✅ Portability across processor designs.</strong></li>
✅ High-level information to support optimization.</strong></li>
✅ Language independence.</strong></li>
❓ Operating system support.</strong></li>
</ol>
Solomon Hykes claimed that if WASI had existed in 2008, he and his companions
at dotCloud1</a></sup> wouldn’t have had to invent Docker. This is a pretty bold
claim! Docker not only solved a lot of issues for developers over the course of
the last decade, it changed the course of the “platform as a service” market.</p>
This quote sets up WebAssembly (and WASI) as an alternative to Docker; not only
than that, it might be better</em> than Docker. So, uh, what gives? What is WASI?
What is Docker? What does “the future of computing” even mean in this context?</p>

Contain your Enthusiasm</h2>
Let’s start with Docker. Docker was launched in 2013 by dotCloud. The
high-level metaphor of Docker is that of shipping containers</strong>. Standardized
containers revolutionized the shipping industry: a standard form factor meant a
standard way to load, unload, and transport material. The technology made
shipping everything</em> faster, cheaper, and more reliable. This is Docker’s
raison d’être: to do for computer applications and operations teams what
standard containers did for the shipping industry.</p>
The metaphor is pretty durable: the concrete problem that Docker solved was
that of shared resources. Consider: given a large company, two developers may
develop two different web serving applications (“services”</strong>) that expect to
listen on the same network port2</a></sup>. Before containerization, these two
developers would have to sync up: who gets to listen on port 80? They’d have to
agree, up-front, before being able to deploy their software. Worse, if they
couldn’t come to agreement about the use of resources, they’d have to involve a
third-party — someone from the operations department — who might solve the
problem by introducing a third</em> server, a reverse proxy layer, to resolve
the conflict.</p>
Docker solves this by giving each service a virtualized</strong> view of the
operating system’s resources. Both developers may remain agnostic of each other
AND the operations team. The operations team can map network ports as necessary
– or rely on orchestration software, like Nomad or Kubernetes, to do this for
them. Standard containers yield standard tools.</p>
So far, so good. So what is</em> Docker?</p>
Docker is a virtualization technology, just like all of the other
virtualization technologies we’ve talked about in previous posts. However, it
achieves virtualization through a combination of technologies supported by the
kernel of the host operating system: seccomp, namespaces, and control groups.
Namespaces allow for partitioning of operating system concepts across
processes. Seccomp (“secure computing”) allows or disallows the set of system
requests a process may make. Control groups partition use of hardware
resources: the amount of CPU time, number of processors, and amount of memory
processes in the group can use. Each namespace, control group, and seccomp
profile has to be layered properly to make durable the illusion that the
process is running on a virtualized system.</p>
Docker comprises a long-lived process (or “daemon”, in this case called
containerd</code>) for running these virtualized containers (via runc</code> or crun</code>),
a user interface for controlling</em> those containers, a registry protocol for
sharing</em> container images, and a file format describing how to build</em> those
container images. The long-lived process, container builder, and registry
protocol is usually called the “runtime”, while the user interface for
starting, stopping, and managing container state is called a “frontend”. The
file format for building docker images, a Dockerfile</code>, specifies a series of
commands that construct a container image; each command forms a distinct,
content-addressable “layer”. Dockerfile</code>s may source other Dockerfile</code>s
during the build process, common layers are reused between builds.</p>
dotCloud rebranded themselves as Docker in an attempt to capture some of the
value of this ecosystem in 2013; to ensure trust that the ecosystem would
outlive the company the “Open Container Initiative” was formed in 2015. As a
result, there are alternative runtimes (crio</code> from Redhat), frontends
(Rancher), or combinations of the two (podman</code>.)</p>
Containers rely on kernel support: on operating systems that don’t support
namespaces and cgroups containers are typically run in a virtual machine
process. Windows supports containers natively through a hcsshim</code> runtime that
provides namespace and cgroup support through Window’s native Host Compute
Service. Apple’s macOS, however, runs docker containers by spinning up a
virtualized Linux host to run a Docker daemon and container processes.</p>
While the Docker/OCI containerization model is ubiquitous today, the
capabilities underpinning it have a much longer history.</p>


I never intended to use these tools to simulate a system in real-time. I wanted
to watch the cracker’s keystrokes, to trace him, learn his techniques, and warn
his victims. The best solution was to lure him to a sacrificial machine and tap
the connection. The Ethernet is easy to tap, and modified tcpdump software can
separate and store the sessions. But I didn’t have a spare machine handy, so I
took the software route. (Steve Bellovin did construct such a machine. We never
managed to lure anyone interesting to it.)</p>
I consulted the local gurus about the security of a chroot</code> environment. Their
conclusion: it is not perfectly secure, but if compilers and certain programs
are missing, it is very difficult to escape.</p>

“An Evening with Berferd”</a>, Bill Cheswick, 1990</li>
</ul>
</blockquote>
The need to consolidate servers, decouple hardware provisioning from service
scaling, and migrate batch processes between high performance compute nodes
created a renaissance of virtualization technologies in the late 90’s and early
2000’s. The rapid adoption of the internet drove these use cases, thawing a
nearly twenty year long freeze in virtual machine research.</p>
Let’s walk backwards through time for a second.</p>
Docker was originally implemented using an earlier containerization model
called “LXC” (“Linux Containers”), introduced in 2008. LXC, like many of its
contemporary containerization models, was built on the model of a virtual
machine. Rather than Docker’s model of taking a single process and giving it a
virtual system environment, LXC containers typically virtualized an init</code>
process (like systemd</code>, upstart</code>, or SysV init</code>3</a></sup>.) This made them
“feel” much more like a virtual machine, with their own long-lived daemons,
periodic tasks, and system logs.</p>
Linux Containers were made possible in part by Google’s contribution of control
groups (cgroups</code>) to the Linux kernel in 2007. Control groups allowed userland
to communicate resource quotas for process subtrees to the kernel: effectively
allowing container runtimes to dictate the maximum CPU time, parallelism,
network, memory, and disk usage a set of sub-processes should be able to use.
This doesn’t affect the ability of those processes to see</em> certain subsystems:
control groups don’t affect what the process has access to, only the available
quality of service for that access. This control allowed Google to more
efficiently allocate shared resources using their internal orchestration
software4</a></sup>.</p>
Namespaces are related to earlier jail</code> capabilities. They control what
operating system subresources are visible to jailed processes. The earliest
jail-like capability, chroot</code>, was added to AT&T Unix in 1979. It was also
added to the Berkeley Software Distribution (“BSD</strong>”5</a></sup>) in 1982. chroot</code>
allowed a process to “pivot” the root directory to a subdirectory, effectively
hiding parent directories from a process. This offered incomplete
protection</a>, thus BSD introduced the jail</code> system call in 2000.</p>

In the case of the chroot(2)</code> call, a process’s visibility of the file
system name-space is limited to a single subtree. However, the
compartmentalisation does not extend to the process or networking spaces and
therefore both observation of and interference with processes outside their
compartment is possible.</p>
To this end, we describe the new FreeBSD ‘Jail’ facility, which provides a
strong partitioning solution, leveraging existing mechanisms, such as
chroot(2)</code>, to what effectively amounts to a virtual machine environment</em>.</p>

“Jails: Confining the omnipotent root.”</a>, Poul-Henning Kamp, Robert N. M. Watson, 2000</li>
</ul>
</blockquote>
Jails were effective but did not address resource management or scheduling
concerns. Sun addressed this in 2004 with Solaris’s “Zones”, which provided an
LXC- or Docker-like experience. However, Sun fell on hard times during the
2000’s, eventually meeting its demise in 2010 after being acquired by Oracle.
As a result, Solaris didn’t experience the widespread adoption that various
Linux distributions enjoyed during this time. (We’ll touch on the source of
Sun’s woes a little later.)</p>
Starting in 2002, Linux began adding namespace support. The mount namespace for
filesystems came first, inspired by Plan 96</a></sup>. Eric W. Biederman
outlined the necessary namespace support in “Multiple Instances of the
Global Linux Namespaces”</a>:</p>

mnt</code>: The filesystem namespace.</li>
uts</code>: The UNIX Time-sharing namespace (controlling what hostname is visible to the process.)</li>
ipc</code>: Inter-process communication namespace.</li>
net</code>: The network namespace.</li>
pid</code>: The process identifier namespace.</li>
user</code>: The user and group namespace.</li>
time</code>: The time namespace.</li>
</ul>
Linux added the user</code> namespace in 2013 — the same year Docker was first
released publicly. (This was later followed up with the cgroup</code> namespace in
2016.) Namespaces and cgroups form the basis of what we think of as containers
today7</a></sup>.</p>
As we noted previously, virtualizing the system at the process/system interface
required careful virtualization of each kind</em> of system resource visible to
processes, both in terms of which subresources are visible (files, other
processes, users, etc) and</em> in terms of their quality of service (available
memory or bandwidth.) Each of these resources can be virtualized separately</em>,
but the Docker containerization model generally virtualizes them
together8</a></sup>.</p>

We have been gratified when casual users mistake the technology for a virtual machine.</p>

“Solaris Zones: Operating System Support for Consolidating Commercial Workloads”</a></li>
</ul>
</blockquote>
There’s a fork in the road, here. It is important to understand both paths: why
they diverged, how they’re starting to merge once again, and what that means
for WebAssembly. Containerized processes virtualize at the system interface;
virtual machines virtualize at the hardware interface. However, processes and
virtual machines are two expressions of the same concept: time-sharing</strong>.</p>

Processes</h2>
Processes are a near-universal concept in modern operating systems. They
comprise three capabilities:</p>

A continuous view of processing resources</strong> that initially includes a
single advancing program state with the ability to spawn additional,
concurrent states against the same program within the process
(“threads”</strong>.)</li>
A contiguous address namespace</strong>, into which memory may be allocated by
the operating system upon request. This initially contains the program
instructions (or “text”</strong>), environment variables and arguments, read-only
(or “static”</strong>) data, space reserved for structures to be initialized at
startup (“block starting symbol”, or “bss”</strong>9</a></sup>), and
room for a “stack” of activation frames. A number of system libraries may
also be loaded by the operating system into this address space.</li>
A system interface</strong>, made available through a set of supervisory calls
(“system calls”</strong>.) Typically this is implemented through use of dedicated
syscall</code> or system interrupt instructions (e.g., int 0x80</code>, in x86 assembly)
available through the instruction set architecture. The use of supervisory
calls is dictated through calling conventions, including what registers must
be saved, where parameters to the syscall are placed—in memory or on registers— and how results are
retrieved after control returns to the process thread. These conventions
describe an “application binary interface”, or “ABI”</strong>.</li>
</ol>
This is a virtual</em> view of the physical machine’s resources: an “extended
machine”10</a></sup>. The operating system is responsible for enforcing the illusion
that each process operates with independent, full access to the system’s
resources. This illusion is constructed through the operating system’s careful
orchestration of the machine’s processor features — the
processor’s memory mapping hardware, interrupts and traps, mode and protection
rings, and use of privileged instructions.</p>
As widespread as the concept of a process is today, it wasn’t always so:
processes and operating systems emerged alongside</em> virtual machines in the
’60s.</p>
Hardware and software design were, at the start of the 1960s, intertwined. It
was common practice to design parallel computers —those with a number of
processors— by assigning a program to each processor, along with a range
of physical memory to be accessed by</em> that processor. If, during development,
programmers discovered that one</em> process on the system needed additional
space, all other processes had to be reprogrammed to accommodate the new memory
partitions. Similarly, if one process went idle, no other processes could make
use of that idle processor hardware. Programs had to be submitted in batches in
order to maximize utilization of the hardware. Time-sharing systems would
change that.</p>
</iframe>
In 1961, DARPA, newly flush with cash11</a></sup>, funded research on time-sharing operating
systems through “Project MAC”12</a></sup>. MIT, a participating member, brought their
“Compatible Time Sharing System” (“CTSS”</strong>13</a></sup>) with them, and along with
Bell Laboratories and General Electric, began work on a successor system called
“MULTICS”. Several teams coordinated on it.</p>
That term we introduced earlier, “extended machine”, originated from this era.
It described a machine that was “easier to program” than the underlying
physical machine. The role of a “monitor” (or “nucleus”, or “kernel”) was to
support these extended machines by using the “bare machine interface”, or
hardware, directly. The monitor scheduled user programs to minimize costly idle
time.</p>
The extended machine typically supported virtual memory, supervisory calls, and
protection rings. Virtual memory was developed so that programs could be
written without advance knowledge of other programs running on the system.</p>
This was accomplished by introducing indirection: the address namespace of a
program no longer mapped directly to the physical address space of memory.
Three techniques were used to accomplish this indirection: paging,
segmentation, or the combination of the two.</p>
With paging, both the process and physical address space are subdivided into
“pages” of memory. Each page represented several hundred “words” of
memory14</a></sup>. Accesses to “non-resident” pages in process address space
trigger hardware faults. The kernel sets traps for those faults: the fault
transfers control to the kernel, which takes action to load the
missing page into physical memory to complete the mapping. Control transfer
back to the process at this point15</a></sup>.</p>
</a>
^{Illustration from}“Segmentation and the Design of Multiprogrammed Systems”</a>, Jack B. Dennis, 1965. “N” represents the process address namespace, “M” the physical namespace. Note how the contiguous “N” namespace maps to a discontiguous namespace in “M”.</sup></p>
Memory segmentation was also used to support virtual memory. Segmentation has
the advantage of allowing larger address namespaces than the native computer
word size would otherwise be capable of representing. Consider that a 16-bit
word can only represent values from 0 to 65535. One popular segmentation scheme
addresses 20-bit values. It accomplishes this by holding another 16-bit value
in a “segment register”, shifting it left by 4 bits (multiplying it by 16) and
adding the resulting value to the base offset held in the operand register.
This allows addressing up to 1MiB of memory. When the segment register changes,
the entire segment of memory is made resident at once — which could be a
single word of memory or a significant subset of physical memory. Some hardware
supports transferring control to the kernel when the segment register changes,
which allows the implementation of virtual memory purely through segmentation.</p>
</a>
^{Illustration from}“Segmentation and the Design of Multiprogrammed Systems”</a>, Jack B. Dennis, 1965. The left diagram illustrates using a word address to index into the segment. The right diagram illustrates the namespace of segments.</sup></p>
The variable size of segments could lead to conflict —or thrash—
between processes if the two segments happened to overlap. Given the much
expanded range available to them, desktop 64-bit processors tend to use paging
by itself to implement virtual memory. Processors of the era we’re discussing
used both techniques in conjunction to combine the advantages of paging with the
ability to increase the addressable memory space.</p>
Finally, protection rings change the instruction set architecture available to
the program, trapping or disallowing use of privileged instructions having to
do with memory mapping, input/output (“I/O”</strong>) devices, or the manipulation
of software timer interrupts. The operating system runs in a protection ring
with higher privileges. Processes run in a lower ring. Processes may request
services from the operating system, like memory allocation or I/O, by making
supervisory calls. System requests are implemented using interrupt instructions
or dedicated syscall</code> instructions available in the instruction set
architecture of the processor.</p>
(For more on all of this, check out cpu.land</a> and Phil Opp’s “Writing an OS in Rust” series</a>)</em>.</p>
Project MAC’s contemporaries included Project GENIE at UC Berkeley and the IBM
System/360 and 370. These technologies were built in competition with MULTICS;
IBM found that they their university bids were being out-competed by
time-sharing systems spun out from Project MAC. This led them to develop early
whole-machine virtualization software.</p>
While extended machines made it possible to safely create and run relocatable
programs, much research revolved around the development of new operating
systems. This was more difficult than developing a program for an extended
machine, as only a single kernel could run at a time. This lead to the
development of “pseudo-machines” or “virtual machine monitors”, which used the
bare machine interface to provide many copies of that same</em> bare machine
interface for the purposes of kernel development. Further, these virtual
machine monitors could be nested</em>, so long as the appropriate resource
mappings were set up in hardware ahead of time. There was some debate around
“pure” virtual machine monitors as opposed to “impure” VMMs at this time!
“Impure” VMMs presented an extended bare machine interface to the guest; the
guest machine was aware of the virtualization.</p>
IBM implemented virtual machine monitors on the System/370 using a feature of
the processors’ protection rings. Whenever a non-privileged program performed a
privileged operation, the processor hardware would fault. The VMM trapped these
privilege faults; mapping incoming requests for resources and operations to
appropriate backing resources without the knowledge of the guest operating
system. In his July 1974 article “A Survey of Virtual Machine
Research”</a>, Robert P. Goldberg noted that the primary
difficulty in implementing efficient virtualization of machines lies in the
lack of comprehensive hardware support for trapping “privilege-sensitive”
instructions. Indeed, the month prior, he and Gerald J. Popek proposed a
definition for “virtualizable architecture” in “Formal Requirements for
Virtualizable Third Generation Architectures”</a>.
Even today, virtualizable architectures can be set to meet “Popek and Goldberg
virtualization requirements”16</a></sup>.</p>
</a>
^{Illustration of VMM, extended machine, and interfaces from}“Survey of Virtual Machine Research”</a>, Robert P. Goldberg, COMPUTE June 1974</sup></p>
So why did virtual machine monitor research halt for so many years?</p>
The Mansfield amendments, passed 1969 and 1973, narrowed the scope of
Department of Defense funds to projects with direct military applications. This
cut public funding for operating systems research. The General Electric 635
used to build MULTICS at MIT cost as much as a passenger jet. MULTICS had been
over-budget and behind schedule for years; Bell Labs pulled out of the MULTICS
project in 1969. The aftermath of the OS research era left many useful ideas
floating in the ether, while virtual machine monitor research would freeze
until the late 1990s17</a></sup>.</p>
</iframe>
[](https://archive.org/details/byte-magazine-1984-11/page/n229/mode/2up)
Many researchers from the Project MAC days joined up at Xerox PARC. There they
developed the Xerox Alto and Smalltalk, which prefigured the modern personal
computer. By the late 1970s, commercial personal computers were available to
consumers: the Apple II, Tandy TRS-80, and others. The IBM PC launched in 1981,
the Apple Lisa in 1983, and the Macintosh in 1984. The earliest versions of
these computers only ran a single program at a time, typically in conjunction
with a disk operating system (“DOS”</strong>); later they would run many processes
cooperatively. There was only one user and thus, no need to worry about
time-sharing — these systems could be written to assume cooperation between
all programs running on the machine18</a></sup>. In 1982, Intel released the first
commercial chips capable of protected mode operation & on-die memory mapping in
the form of the 80286 processor19</a></sup>. Taken together with the
rise of high-performance workstations, Microsoft, Intel, and IBM’s PC rapidly
chipped away at the market for time-shared minicomputers.</p>
For the most part, those workstations ran a variant of UNIX.</p>
Dennis Ritchie, Ken Thompson, Douglas McIlroy, and Joe Ossanna developed UNIX
at Bell Laboratories in the aftermath of Bell’s strategic retreat from the
MULTICS project. UNIX benefited from its circumstances: a small
team20</a></sup> with little supervision working on a comparatively cheap computer
(the PDP-7) proved able to iterate on the best ideas in operating systems
research rapidly. Douglas McIlroy managed the department. Douglas was
particularly animated</a> by the possibilities code reuse
unlocked. UNIX did not start out as a time-sharing system, nor was it a
preemptive operating system. It wasn’t much more than a filesystem supporting a
game at first</a>.</p>

Back around 1970-71, Unix on the PDP-11/20 ran on hardware that not only did
not support virtual memory, but didn’t support any kind of hardware memory
mapping or protection, for example against writing over the kernel. This was
a pain, because we were using the machine for multiple users. When anyone
was working on a program, it was considered a courtesy to yell “A.OUT?”
before trying it, to warn others to save whatever they were editing.</p>

“A hardware story”</a>, Dennis Ritchie, 2002</li>
</ul>
</blockquote>
The initial system was multiprogrammed — that is, there were two, time-shared
shell processes running, one for each terminal connected to the machine. When
the shell executed another program, it would read the file in over the top of
the shell code and start executing it. The exit()</code> syscall would reload the
shell program over the top of the shell and restart execution. Support for a
tree</em> of processes was added rapidly, however. UNIX inherited Conway’s
fork/join</code> semantics</a> from Project GENIE as fork/exec</code>: one
command to make a duplicate of the current process as a child of the forking
process, and a second command run in that child process to replace the
duplicate with the target code. This model of copying processes using
prototypal inheritance directly enabled the containerization models we talked
about earlier (Linux eventually grew clone</code> and unshare</code> to handle removing
privilege from new processes early.)</p>
By 1977 work was underway to port UNIX from the PDP-11 to the Interdata 8/32,
as we noted way back in the first post in this series</a>. (The work had
been validated by porting a copy of UNIX to an IBM System/370 virtual machine.)
With the support of Sun Microsystems, IBM, HP, and others, UNIX and C had swept
through the industry by the late eighties. Entering the 90’s, “virtual machine”
came to refer primarily to language virtual machines, like Smalltalk and
Erlang. Consumer operating systems were converging on preemptive, protected
time-sharing, buoyed by advances in Intel’s commercial hardware. Hardware
virtualization features, if supported, primarily existed to support older DOS
programs.</p>
First and foremost, UNIX succeeded as a user interface</strong>. Not necessarily in the
traditional sense, but rather as a programmer’s work bench. The system put
forward a comparatively simple, unified model of programming. UNIX inherited
code sharing and linking from MULTICS. This code sharing and linking was
powerful, but ultimately subordinate to the system’s user interface. Code
linked into a program shared that program’s resources, capabilities, and access
to the system interface.</p>

Return to the fork in the road</h2>
The rapid growth of the internet in the mid-90’s rekindled virtualization
research. Scaling a website meant procuring and provisioning hardware. This
could not be done quickly. Internet companies were required to expend
considerable money well in advance of expected traffic and were largely unable
to recoup costs if that traffic did not materialize. The industry was looking
for a way to commoditize hardware.</p>
x86 was the most popular processor architecture with the best economies of
scale and broadest software support. However, it could not be virtualized
directly: privileged instructions and memory operations would silently fail
without triggering traps.</p>

Unfortunately, many current architectures are not strictly virtualizeable.
This may be because either their instructions are non-virtualizeable, or they
have segmented architectures that are non-virtualizeable, or both.</p>
Unfortunately, the all-but-ubiquitous Intel x86 processor family has both of
these problematic properties, that is, both non-virtualizeable instructions
and non-reversible segmentation. Consequently, no [Virtual Machine Monitor]
based exclusively on direct execution can completely virtualize the x86
architecture.</p>

“Virtualization System Including a Virtual Machine Monitor For a Computer
With Segmented Architecture”</a>, United States Patent #6,397,232</li>
</ul>
</blockquote>
The late 90’s saw a growing interest in direct binary translation (“DBT”</strong>.)
The hegemony of the x86 processor was predicted to crumble. Efforts such as
DAISY and Transmeta’s Crusoe bet against the continued popularity of x86.
Meanwhile, in the language virtual machine space, just-in-time compilation
research blossomed. VMWare was founded in this context, submitting a
patent</a> early on for a direct-binary-translation method of implementing
virtual machine monitors for the x86 architecture.</p>
VMWare’s virtual machine monitor worked thusly: entering or exiting a
protection ring triggered the translator, which monitored for execution of
untrapped privileged instructions and inject manual VMM trap calls. This
enabled virtualization but added overhead to every system call.</p>
Because DBT added uneven overhead to kernel execution, it was difficult to
associate system use with virtual machine execution. This prevented accurate
billing systems from being constructed around such virtual machines.</p>
“Xen and the Art of Virtualization”</a> changed all of that.</p>

By allowing 100 operating systems to run on a single server, we reduce the
associated costs by two orders of magnitude. Furthermore, by turning the
setup and configuration of each OS into a software concern, we facilitate
much smaller-granularity timescales of hosting.</p>

“Xen and the Art of Virtualization”</a>, 2003</li>
</ul>
</blockquote>
Xen achieved this through paravirtualization: instead of pure virtualization,
it achieved “impure”, cooperative virtualization by modifying the guest operating
systems. In particular, Xen moved the operating system out</em> of the most
protected ring of the processor: from ring 0 to ring 1. Processes continued to
run in ring 3. This gave the operating system its own “extended machine”: while
it lost access to privilege instructions, it gained access to its own
supervisory call system (“hypervisor calls”.)</p>
</a>
^{Comparison of native Linux, Xen, VMWare Workstation, and User-mode Linux on various benchmarks, from}“Xen and the Art of Virtualization”</a>. Note that OLTP represents relational database workloads and WEB99 represents web-serving.</sup></p>
In particular, Xen’s performance on web application workloads was a breakthrough.</p>
A few notes on timing: by 2003, the dotcom bubble had burst. The servers that
internet startups had loaded up on flooded the market with cheap used hardware,
causing Sun Microsystems to hemorrhage money. At Amazon, Benjamin
Black</a> circulated a document describing a standardized
infrastructure; Bezos tasked Chris Pinkham with developing this in 2004. EC2
launched publicly in 2006, powered by Xen</a>. According to Steve
Yegge</a>, sometime in 2002-2003 or so, Bezos issued his famous “services
edict”, stipulating that all Amazon engineering teams deliver their work in the
form of networked services. By 2009, Netflix had moved their video encoding
operations to AWS. Netflix finished transitioning to the cloud in 2011. And
again, just as a reminder, Docker launched in 2013.</p>
Service oriented architectures —composed from hundreds of tiny web
microservices making tiny remote procedure calls to one another over
HTTP— roamed the earth. You might have wondered earlier why two engineers
would come into conflict over a network port on a single machine.</p>
Well, now you know why!</p>


Hardware is really just software crystallized early. It is there to make
program schemes run as efficiently as possible. But far too often the
hardware has been presented as a given and it is up to software designers to
make it appear reasonable. […]</p>
As Bob Barton used to say: “Systems programmers are high priests of a low
cult.”</p>

“An Early History of Smalltalk”</a>, Alan Kay</li>
</ul>
</blockquote>
Cloud virtual machines commoditized hardware, severing the connection between
“procuring and provisioning hardware” and “scaling a web service.”</p>
As with memory mapping and protection rings in the 80s, consumer hardware
lagged behind the market’s needs. AMD and Intel introduced hardware support
for virtualization of x86(_64) architectures in 2005 through SVM and VT-X,
respectively. ARM added hardware virtualization support in Cortex-A in 201121</a></sup>.</p>
Hardware virtualization support allowed operating systems to integrate
hypervisor capabilities: Windows Server added Hyper-V (2008); Linux, Kernel
Virtual Machines (“KVM”, 2006); and macOS, Hypervisor.framework (2020). (EC2
started moving from Xen to KVM-based “Nitro” virtual machines in 2017. Brendan
Gregg wrote more about the path from the first version of Xen to Nitro
here</a>.) In lieu of running Xen or VMWare, consumer operating systems
themselves became capable of running guest operating systems, accelerating the
development of new hypervisor software.</p>

The core innovation of Docker is not the underlying containerization
technology. As we’ve established: the underpinnings of the OCI format predate
Docker by many years. Docker’s innovation is in its user interface around
authoring, sharing, and running container images. This reusability led to the
creation of standardized orchestration software; virtual machine images have
yet to catch up to this level of popular reusability.</p>
From the hardware perspective, processes and guest operating systems look
nearly the same. Virtualizing the system at the process layer means sharing
more code between the virtualized systems with a finer granularity of
abstractions, but has the advantage of being achievable purely through
software: hardware can’t really tell the difference between a container and any
other process.</p>
Because virtualizing the system at the hardware interface layer is more
coarse-grained with less shared code, VMMs are generally considered safer
targets for multi-tenancy —that is, running untrusted code from third parties
collocated on the same hardware22</a></sup>. On the other hand, VMMs are
generally perceived as introducing more startup overhead. However, in recent
years, containers and VMMs have converged: Kata containers, AWS’ Firecracker,
and others use the container user interface but run the containers inside
lightweight virtual machines, achieving remarkable performance, security, and
density23</a></sup>.</p>

The more efficiently tasks can be collocated, the better the margins on
equipment; this is a competitive edge for a hosting company. Processes,
whether virtual machines or containers, have overhead. Switching between
processes takes time and memory; density experiments as of TKTK 202X have shown 10K
containers running on a single host. This was a natural place to look for
improvements, and so content delivery networks — which handle some of the
highest volume of traffic on the internet — started digging into the problem.
Fastly and Cloudflare both landed on web technologies, launching products
in 2018.</p>
Cloudflare launched Workers, which co-locates many user tasks in a single
process using V8 isolates. V8 is, as we discussed previously, the JIT
JavaScript engine open-sourced by Google as part of their Chrome browser.
Workers customers upload JavaScript applications that implement the
ServiceWorker spec; Cloudflare deploys the application to their edge network, with points
of presence across the globe. As a bonus, since WebAssembly support is
available through the web platform API, users that really needed to could
target WASM. (Though they’d be responsible for writing their own bindings to
the ServiceWorker API.)</p>
Fastly launched Compute@Edge, skipping JavaScript in favor of WebAssembly. But
this posed a problem. While JS could rely on the Web platform as a system
interface, WASM does not automatically provide one. In order to hook WASM up to
their edge compute, Fastly had to define</em> an interface: one that was stable
and versioned.</p>

Enter WASI (finally)</h2>
Fastly, Intel, and Mozilla teamed up in 2019 to form the Bytecode Alliance.
Their goal was to define a specification for a standard system interface for
WebAssembly24</a></sup>.</p>
The WebAssembly ISA allows imports and exports: functions that the host can
pass in</em> to the WASM module for internal use, and functions that the WASM
module can hand back to the host to be called by</em> the host on demand. The host
may also provide a chunk of memory for the WASM module to operate on. Imported
and exported functions may only take primitive values — integer and floating
point values of various widths. The host and module may cooperate to transfer
more complicated types. For example, the host and module might agree that a string
is represented as an integer pointer into the module’s memory along with an
integer length.</p>
This is a lot</em> like the sort of ABI</strong> a process System Interface</strong> uses.</p>
In fact, the first preview release of WASI explicitly mimicked the POSIX system
interface — roughly speaking, the system interface common to all variations of
UNIX systems. This differed from contemporary tools, like emscripten and
wasm-bindgen, which treat the interface between the host and WASM module as an
internal detail. Those tools were designed to take an existing application and
get it running in a browser with minimal changes, generating both WASM, HTML,
and the JavaScript to integrate the WASM with the Web Platform. If one were to
recompile an application with a newer version of emscripten and try to drop the
WASM onto an older version of the HTML and JS, it may not work. Likewise, the
WASM from an application compiled with wasm-bindgen could not be dropped onto
emscripten HTML and JS or vice versa. They are not ABI</strong> compatible.</p>
WASI existed to enable multiple compilers and hosts. It would be a [thin
waist][thin-waist] for WebAssembly25</a></sup>.</p>

The first preview of WASI launched in 2019. While it included many POSIX
concepts, it omitted full support for network sockets and process forking.
These features are key to supporting high-performance network servers. The
omission was intentional: system interface ABIs don’t just define the boundary
between a process and kernel. They’re also used to enable software linking</em> –
the reuse of compiled artifacts: shared objects (.dylib</code>, .so</code>, or .dll</code>.)
Linking can be performed at compile time (static linking) or at process start
(dynamic linking.) While shared object linking wasn’t defined as part of the
first WASI preview, neither did the specifiers wish to preclude it in future
versions of the specification.</p>
The standard approach to sockets would have exposed too much power between
linked modules. Sockets are typically represented as a “descriptor” or “handle”
in the form of an integer the operating system hands to the process on request.
Operations on the socket are performed by making a system call referencing that
integer, and all linked modules are assumed to have access to the same system
interface. This model is difficult to secure: malicious linked modules can
“sniff” for active descriptors. The system can’t differentiate between linked
modules and the core application logic when receiving calls referencing the
descriptors.</p>
The problem with the POSIX model is that the system interface forms a
monolithic wall between the kernel and the user process, leaving
undifferentiated space on either side. The ABI emphasizes the importance of the
system interface over</em> the importance of the interface between libraries.</p>
Finding a solution to this problem took time.</p>
The second preview of WASI proposes a solution: the WASM component model. This
defines a static linking approach for combining many WASM core modules in a
single file. This proposal is in stage 1, which means that many popular WASM
tools and runtimes do not yet implement it. Despite this, WASI preview 2 hinges
on this proposal, which defines a new interface definition language called
“WebAssembly interface types” (“WIT”.) WIT allows for the definition of named
functions, groups of functions, and high-level types like strings and structs.
WIT also allows for groupings-of-groupings with requirements around exports and
imports, called “worlds”. WIT worlds can be used to generate host code or as a
target for compiling shared code.</p>
This allows sockets to be represented as higher level objects whose full
capabilities aren’t transferred between linked modules, as opposed to an
integer descriptor tracked by the host. (See Dan Gohman’s excellent “No
Ghosts!”</a> for more on this.)</p>
WASI preview 2 appears to be targeting a release this year. At the time of
writing the component model proposal is only in stage 1, so many popular tools
and runtimes do not support WASM components (or WASI preview 2.) And many
platform-as-a-service (“PaaS”</strong>) startups have appeared since Solomon Hyke’s tweet.</p>
In the meantime, companies have proposed alternative system interfaces: Deis
labs introduced WAGI in 2020. WAGI sidesteps the issue of missing POSIX support
by using WASI preview 1’s standard input and output instead of sockets. This is
reminiscent of the venerable “common gateway interface” (“CGI”</strong>) from the
1990s. Fermyon, a WASM PaaS company, supports WAGI through its Spin framework.</p>
In 2023, Wasmer introduced WASIX. WASIX, a superset of WASI, aims to target
more of the POSIX API. This is akin to an Emscripten for server applications,
with the goal of lowering the barrier between existing web services and PaaS
WASM companies. Like Emscripten, WASIX values easy portability of existing
applications. As a result, it doesn’t editorialize too much on the POSIX API.
At the time of writing, WASIX doesn’t support shared object linking. While WASIX
doesn’t preclude linking shared objects in the future, it seems likely that it
would follow the existing linking model, for good or ill26</a></sup>.</p>

The common thread through every technology we’ve talked about is that they were
all created to make it easier for people to work together. To lower the barrier
of cooperation. Processes made it possible to write programs separately,
containers made it possible to use OS resources without up-front coordination,
VMMs made it possible to decouple infrastructure scaling from procuring
hardware. Each of these take a direct conversation between two components and
move it to a third party, an interface. They form a minimal agreement — the
smallest contract that one must agree to in order to work seamlessly with any
other party that also agrees.</p>
WASI, through the web component model, revises the UNIX system interface,
narrowing it from a programmer’s workbench to a secure computing environment.
Taken to the extreme, the WASM component model reinvents the operating system
process model to take modern needs into account. It promises secure
multi-tenancy at runtime while lowering the barrier to cooperation between
library authors and application programmers. From my (maybe optimistic?)
perspective, it’s on the cusp of a Docker moment. The technologies that
underpin Docker predated Docker by years. Docker bundled them together and made
it easy to share one’s work with others. What does WASM’s moment look like?</p>

Epilogue</h2>
I’m excited to announce that I’ll be joining Dylibso as of this month to work
on WASM materials, tools, and smooth out friction in the ecosystem wherever I
find it.</p>
Working on these posts has been hugely educational for me on a number of
levels, and I’d like to thank everyone who reviewed these posts — C J
Silverio, Eric Sampson, and Aria Stewart — helped source research — Ron Gee,
– and encouraged me. In particular I’d like to thank my family: my wife,
Krysten, and my parents, Mark and Sue, for dealing with an entire summer of me
talking non-stop about writing these posts and reviewed the most radioactive,
unfit-for-human-consumption drafts of these posts (“WASM good? why?”)</p>

Bibliography and Timeline</h2>
So many PDFs this time around!</p>
I’d like to call out “The Ideal Versus the Real: Revisiting the History of
Virtual Machines and Containers”</a> by Allison Randal, which dives
more deeply into the history and interrelations between these technologies than
my effort here. Give it a read!</p>

1961. “Dynamic Storage Allocation in the Atlas Computer”</a>, John Fotheringham,</li>
1961. “One-level storage system”</a>, Kilburn, Edwards, Lanigan, Sumner</li>
1961. Compatible Time-Sharing System first operational</li>
1962. Atlas Computer operational</li>
1963. “A multiprocessor system design”</a>, Melvin E. Conway</li>
1964. Compatible Time-Sharing System in operational use</li>
1964. IBM System/360</li>
1965. GE 645 hardware simulated at MIT for MULTICS</li>
1965. “Segmentation and the Design of Multiprogrammed Computer Systems”</a> - Jack B. Dennis

cites 1961 “one-level storage system”</li>
</ul>
</li>
1966. “Programming Semantics for Multiprogrammed Computations”</a> - Jack B. Dennis, Earl C. Van Horn

cites 1963 “A multiprocessor system design”</li>
</ul>
</li>
1967. GE 645 hardware received at MIT</li>
1968. [“Virtual Memory, Processes,and Sharing in MULTICS”][multics-virtual-memory] - Robert C. Daley, Jack B. Dennis</li>
1969. UNIX v1 running on a pdp-7</a></li>
1969. MULTICS replaces GECOS on GE 645</li>
1970. “Virtual Memory”, Peter Denning

cites 1961 “one-level storage”</li>
cites 1961 “dynamic storage allocation”</li>
cites 1966 “Programming Semantics for Multiprogrammed Computations” Jack B. Dennis, Earl C. Van Horn</li>
</ul>
</li>
1970. IBM System/370 released</li>
1971. UNIX moved to the PDP-11</a></li>
1973. “Virtual machine or virtual operating system”</a>, Bellino, J, C Hans

this seems a LOT like paravirtualization and/or containers — could not achieve full performance
virtualizing an entire IBM 360 machine, so the VMM provided “higher level” primitives</li>
</ul>
</li>
1974. “Survey of Virtual Machine Research”</a>, Robert P. Goldberg

cites bellino/hans 1973 re: “pure” vs “impure” virtual machines</li>
</ul>
</li>
1979. UNIX v7, introduction of chroot</code></li>
1979. “Software Engineering in 1968”</a> - B. Randell</li>
1981. “The Origin of the VM/370 Time-Sharing System”</a> - R. J. Creasy</li>
1982. chroot</code> added to BSD</li>
1998. immunix apparmor introduced</li>
1998. VMWare x86</li>
1999. Resource Containers</a></li>
2000. FreeBSD jails</a></li>
2000. SELinux first released OSS</li>
2001. linux vserver</li>
2001. VMWare releases gsx</a>, first x86 server virtualization product</li>
2002. VMWare patent - “Virtualization System Including a Virtual Machine Monitor For a Computer With Segmented Architecture”</a>, United States Patent #6,397,232</li>
2002. linux namespaces (2.4.19); mount namespace</li>
2002. linux security modules</li>
2003. “Xen and the Art of Virtualization”</a></li>
2003. SELinux merged into linux kernel (2.6.0-test3)</li>
2003. Work on AWS EC2 kicks off</a></li>
2004. Solaris Zones</a></li>
2005. OpenVZ (open virtuzzo)</a></li>
2005. intel vt-x, amd-v</a></li>
2006. Slicehost launched</li>
2006. “Multiple instances of the global linux namespaces”</a> - Eric W. Biederman</li>
2006. AWS EC2 first released - built on xen virtualization</li>
2007. “Adding Generic Process Containers to the Linux Kernel”</a> - Paul B. Menage, Google</li>
2007. cgroups lands in linux kernel (2.6.24); final form of google process containers</li>
2007. KVM lands in the linux kernel (forked QEMU added KVM support)</li>
2008. LXC released</li>
2008. Microsoft launches Hyper-V for windows server</li>
2009. AWS VPC launch, ELB, Autoscaling, Cloudwatch</li>
2009. Netflix moves movie encoding to the cloud</li>
2009. Canonical adopts apparmor</li>
2010. Netflix moves account signup, etc to the cloud</li>
2010. Hyper-V support merged into libvirt</a>



In 2010, Bolte et al. incorporated support for Hyper-V into
libvirt, so it could be managed through a standardized interface,
together with Xen, QEMU+KVM, and VMware ESX.</p>
</blockquote>
</li>
</ul>
</li>
2011. Netflix’s entire operation is on the cloud</li>
2012. KVM support merged into mainline QEMU</a></li>
2013. user namespaces land in the linux kernel (3.8); this completes support for containers [1]</li>
2013. Docker released (based on LXC)</li>
2013. LMCTFY (“Let me contain that for you”)</li>
2014. Kubernetes released</li>
2014. Docker Swarm released</li>
2014. docker-compose v1</li>
2014. Docker replaces LXC with libcontainer</code></li>
2014. Canonical begins work on lxd</code>, orchestration for LXC containers</li>
2015. Terraform released</li>
2015. Consul released</li>
2015. Hashicorp releases first version of vault, nomad</li>
2015. Docker spins runc</code> out of its container runtime</li>
2016. dirty cow container CVE (https://blog.paranoidsoftware.com/dirty-cow-cve-2016-5195-docker-container-escape/)</li>
2016. CRI-O launched</li>
2016. docker-compose v2</li>
2016. LXD released</li>
2017. docker-compose v3</li>
2018. Kata, gVisor, Nabla</li>
2019. CRI-O handed over to cncf</li>
2019. Docker Swarm start of 2-year EOL</li>
2020. “The Ideal Versus the Real: Revisiting the History of Virtual Machines and Containers”</a> - Allison Randal</li>
</ul>

^{1</sup>
Which would later be rebranded as “Docker”, after their most famous
product.</p>
</div>}^{2</sup>
A port is a numbered resource, managed by an operating system,
representing a stream of incoming or outgoing network requests.</p>
</div>}^{3</sup>
Init processes are responsible for setting up the userland system:
they are the root of the tree of userland processes; setting up daemons for
resolving DNS, networking, devices, filesystem mounts, and more.</p>
</div>}^4</sup>AKA “Borg”</a>.</p>
</div>

^{5</sup>
It’s out of scope for this post, but suffice it to say that UNIX
split in the 80’s and 90’s: roughly, Linux, BSD, and SysV. Linux provides
the kernel of popular distributions like Redhat, CentOS, Ubuntu, Debian,
and Android. BSD provides a specification for a kernel for operating
systems: most conspicuously, Apple’s modern operating systems and SunOS.
SunOS’s successor, Solaris, was based on AT&T’s UNIX System V, along with
HP’s HP-UX and IBM’s AIX.</p>
</div>}^{6</sup>
Plan 9! Which, as you’ll recall from the last article, Java
killed! In the 80s and 90s, “virtual machine” in the sense of “system
emulation” had withered so far as to be supplanted by “virtual machine”
meaning “language model runtime.”</p>
</div>}^7</sup>Ok, I’m adding this one as a footnote because this is a long post
already. I’m only going to gesture at linux-vserver, which was focused on scaling
webservers through containerized networking. However, Virtuozzo (and its successor project,
OpenVZ) approached containerization for an entirely different reason: to enable
checkpoint/restore of work on high-performance batch clusters. This would allow
relocation of processes between computers (“nodes”) in a cluster by namespacing
all of their system resources. This required maintaining patches against the linux
kernel at the time, so as far as I can tell it never really took off, but it did
spawn the “Checkpoint and Restore in Userspace” (“CRIU”</strong>) project</a>.</p>
</div>

^{8</sup>
And I didn’t touch on SELinux, which the NSA contributed to the Linux kernel
for, um, reasons</em></p>
</div>}^{9</sup>
“BSS” is also known as “better save space”.</p>
</div>}^{10</sup>
Hey, there’s that “machine” word again.</p>
</div>}^{11</sup>
Which had nothing at all</em> to do with Yuri Gagarin’s recent orbital trip, I’m sure</em>.</p>
</div>}^{12</sup>
“MAC” expanded to “Mathematics and Computation” early on, later
expanding to “Multiple Access Computer”, “Machine Aided Cognitions”, and
“Man and Computer”. It also funded the MIT AI lab.</p>
</div>}^{13</sup>
MIT also originated the “Incompatible Time Sharing”, which would give
us EMACS.</p>
</div>}^{14</sup>
A “word” in this case refers to the number of bits the hardware was optimized to process.
You’re probably reading this on a device with a native word size of 64-bits, but in the past 32-
and 16-bit word sizes were common. In the distant past, you might see 36-bit words!</p>
</div>}^{15</sup>
There are are all sorts of neat tricks that paging enables, including “copy on
write” pages — mapping the same memory to different places in the same
address space, and only creating copies of them when they’re mutated.</p>
</div>

^{16</sup>
“Third Generation Architecture” refers to the generation of
computers designed in the 1960’s using early integrated circuits; these are
typically called “minicomputers”. They were succeeded by fourth generation
architecture in the early 1970’s which began to use microprocessors.</p>
</div>

^{17</sup>
IBM continued to ship virtual machine monitor systems throughout this
period. However, they were primarily focused on virtualizing earlier
technologies, like mainframes and minicomputers, on top of newer hardware.</p>
</div>

^{18</sup>
Now, that said, that assumption was frequently (and spectacularly) invalidated.</p>
</div>

^{19</sup>
Gates called these processors “brain dead”. At the time,
IBM and Microsoft were co-developing OS/2 for the IBM PC, and IBM wanted to
target the 286. However, the protection modes of the 286 were “one-way” –
once the processor entered protected mode, re-entering real mode required
restarting the system.</p>
</div>

^{20</sup>
The benefits of a small team are made evident through this quote, which
is a prime example of [Conway’s Law][conway]:</p>
</div>

Where under Unix one might say</p>
ls >xx</span></span></code></pre>
to get a listing of the names of one’s files in xx, on Multics the notation was</p>
iocall attach user_output file xx</span></span>
list</span></span>
iocall attach user_output syn user_i/o</span></span></code></pre>
Even though this very clumsy sequence was used often during the Multics days,
and would have been utterly straightforward to integrate into the Multics
shell, the idea did not occur to us or anyone else at the time. I speculate
that the reason it did not was the sheer size of the Multics project: the
implementors of the IO system were at Bell Labs in Murray Hill, while the
shell was done at MIT. We didn’t consider making changes to the shell (it was
their program); correspondingly, the keepers of the shell may not even have
known of the usefulness, albeit clumsiness, of iocall. […]</p>
Because both the Unix IO system and its shell were under the
exclusive control of Thompson, when the right idea finally surfaced, it was a
matter of an hour or so to implement it.</p>
“The Evolution of the Unix Time-sharing System”</a>, Dennis M. Ritchie, 1996</p>
</blockquote>

^{21</sup>
Indeed, ARM Cortex’s virtualization support is specifically marketed as
meeting “Goldberg and Popek virtualizability requirements”!</p>
</div>}^22</sup>This is notwithstanding the Meltdown and Spectre</a> vulnerabilities.
Meltdown exploits a race condition between memory access and privilege checking and affects operating
systems and hypervisors. Exploits allow processes and VMs to read memory across security boundaries,
effectively breaking the illusion of virtual memory. Spectre exploits speculative execution —
a property of modern superscalar processors. Speculative execution executes every code path leading
out from a branch point, throwing away the results from the paths not taken. However, this speculative
execution can affect caches, so the path not taken may be observed by measuring operation timings after
the fact.</p>
</div>

^23</sup>For an in-depth look at how containerization and virtualization
approaches compare in terms of performance on various axes, check out “A
Fresh Look at the Architecture and Performance of Contemporary Isolation
Platforms”</a>.</p>
</div>

^{24</sup>
And to develop the Cranelift, Wasmtime, and WASM Micro
Runtime (“WAMR”</strong>) projects.</p>
</div>}^{25</sup>
This is where we get the wasm32-wasi</code> target triple (and why it’s
different from wasm32-unknown-unknown</code>.</p>
</div>}^26</sup>The Bytecode Alliance has taken a dim view</a> of WASIX.</p>
</div>}}}}}}


Understanding Wasm, Part 2: Whence Wasm
Unknown — Fri, 30 Jun 2023 00:00:00 +0000
Understanding Wasm</h1>
Part 2: Whence Wasm</h2>
“Write once, run anywhere” is a great sales pitch. It grabs your attention.
It’s pithy! It invites the reader to fill in the blanks of “who is writing
what, where is it going, and how does it get there” with the answer that most
appeals to them. This was Java’s sales pitch. WebAssembly seems to have the
same goal. Given that Java still exists, why do we need WebAssembly? What makes
them different?</p>


GUARDA COME DONDOLO / Edouardo Vianello
Unknown — Fri, 30 Jun 2023 00:00:00 +0000
A little 1960’s Italian pop music to kick off the weekend.</p>


understanding wasm pt 2: whence wasm
Unknown — Thu, 29 Jun 2023 00:00:00 +0000
understanding wasm</h1>
part 2: whence wasm</h2>
“Write once, run anywhere” is a great sales pitch. It grabs your attention.
It’s pithy! It invites the reader to fill in the blanks of “who is writing
what, where is it going, and how does it get there” with the answer that most
appeals to them. This was Java’s sales pitch. WebAssembly seems to have the
same goal. Given that Java still exists, why do we need WebAssembly? What makes
them different?</p>


Anxiety Groove | Louie Zong
Unknown — Fri, 23 Jun 2023 00:00:00 +0000
Louie Zong is a heckin’ treasure.
There’s a huge list of his stuff I want to link you to but I’m going
to hold off on that.
He makes music, cartoons, 3d art, videos, teaching content, the list
goes on.
Having just got back from being social for several days in a row,
including a (lovely) wedding reception near Mt. Hood, it perfectly
captures how I feel. “Glad you guys are having fun! I’m gonna
bounce!” Thankfully, I have a relatively quiet weekend coming up. But
I did find myself scooting around town singing along to this on
repeat.
OK, I lied about holding off because I have to link</em> to his Mario
italodisco album</a>. I
can’t not!</p>


Plain Text Journaling · peppe.rs
Unknown — Tue, 20 Jun 2023 00:00:00 +0000
This is really quite similar to my neovim notetaking setup</a>, with
a bit more structure.
I particularly enjoyed their vim-specific improvements, like using iabbrev</code> to shorten common words (like todo</code> and done</code> to
·</code> and ×</code>, respectively) and formatprg</code> for automatic todo sorting. And using syntax highlighting was a beautiful detail.
(I’m filing these away as todo list items for myself, to be honest.)
The post also comes with a #nix</strong></a> flake too. As an aside, I intend to start gathering info about nix,
since it looks quite promising as a system setup tool. I’ve never quite</em> been happy with ansible for those purposes, especially when it comes
to local dev.</p>


Understanding Wasm, Part 1: Virtualization
Unknown — Wed, 10 May 2023 00:00:00 +0000
Understanding Wasm</h1>
Part 1: Virtualization</h2>
So, this all started when I tried to answer a colleague’s question about “what
makes WebAssembly (Wasm) different from the Java Virtual Machine (JVM)?” They
were particularly interested in parsing out Wasm’s goals as they overlapped and
differed from the historical goals of the JVM (“write once, run
anywhere”</a>). Further, they were curious about the differences in approach
that lead folks to believe Wasm might succeed at achieving these goals. You
might say:</p>

Wasm is a Web platform technology, developed and supported in the open
by multiple vendors. The JVM was primarily driven by a single vendor.</li>
</ul>
This happens to be perfectly correct! But still. There was something there that
itched at the back of my mind: I knew</em> these things were different, but I
wasn’t able to articulate my mental framework to explain</em> those differences.
There’s a lot going on in that pat answer that bears unpacking:</p>

Where did Wasm come from – what problems was it created to solve?</li>
How did Wasm come to be developed and supported by multiple vendors, while
the JVM remained single-vendor?</li>
Are Wasm and the JVM really the same type of thing?</li>
What does “virtual machine” mean</em>?</strong></li>
</ul>
In particular, when I went to talk about Wasm – which bills itself as a
“virtual instruction set architecture” 1</a></sup> in specification – I found
that I had a shallow understanding of the word “virtual”. I knew the term for
its effect, by example, but not deeply</em>. So. Having found myself with some
time on my hands recently, I split this post in three (at least, to start
with.) Today, we’re going to talk about virtualization, instruction set
architectures, and machines. This post isn’t meant to get into all</em> of the
details, but should give you a good mental framework with which to place these
concepts, and serve as a nice jumping-off point for further research.</p>

Let’s start by forming some rough categories using examples.</p>
There’s the “virtual machines” that Amazon or Google are happy to sell us:
virtualized computers running on their hardware in the cloud. Alternatively:
VMWare, VirtualBox, and QEMU</a> let me run virtualized computers on my</em>
hardware. So, that’s one definition of “virtual machine”: “software simulating
hardware.” (You might even lump “video game emulators” into this category!)</p>
On the other hand, we have “virtual machines” in the “JVM” sense: things like
the .NET Common Language Runtime (“CLR”), Erlang’s BEAM, JavaScript’s
SpiderMonkey or V8 engines, or even Python’s interpreter. These are all
programs running on an operating system for the purposes of taking an input
language and executing it. They do this by translating the input language into
the input language of the current computer. The Python virtual machine consumes
a Python operation code (opcode) and reacts to it by executing some number of
native machine operations. Let’s call this “software evaluating a language” for
now.</p>
On the gripping</a> hand: the Low Level Virtual Machine. I’m calling this one out
separately because it’s not really “software simulating hardware”, nor is it
“software evaluating a language”: it’s something else entirely. It’s a
waystation for languages on their way toward native machine code. C, C++, Rust,
and other languages can be translated from their source to this intermediate
representation, on which a number of optimizations can be made by the LLVM
toolchain. LLVM is a “machine which abstracts over other machines”, an
“abstract machine.”</p>

There are three things that these three rough piles have in common:</p>

They all accept some sort of input language</em>.</li>
They all refer to the term “virtual.”</li>
They all refer to the term “machine.”</li>
</ol>
Let’s approach these concepts in reverse order.</p>
You might already be familiar with the term “Turing machine” or the phrase
“Turing complete”, but in case you’re not: a Turing machine is a mathematical
model of computation for which some useful properties hold. It describes an
abstract machine – hey, there’s that term again! – that operates on a finite
set of symbols printed on an infinite spool of tape. The classic Turing machine
uses a “head” to write to a cell to a discrete cell on that tape, shifting the
tape left or right based on its current state. Turing completeness</em> describes
a property of a language, expressed as a discrete syntax: a Turing complete
language can express any task that can be accomplished by computers.</p>
There’s an interesting relationship between “input language” and
“machine”. A sufficiently detailed description of a language also describes a
machine for processing that language, at least in the abstract. Much like how
an IKEA instruction manual describes the ideal Billy bookshelf. Many Billy
bookshelves can be assembled based on one instruction manual; all in various
degrees of “conversation” with the physical constraints encountered during
assembly.</p>

So what does it mean to “virtualize” a machine?</p>

A good way to distinguish virtual and transparent is this: Virtual means it’s
not there but you can see it; transparent means that it is there but you
can’t see it.</p>
Peter J. Denning - IEEE Annals of Computing History</a></p>
</blockquote>
“Virtual” comes to computer science by way of the field of optics, where it
describes the perceived image of an object reflected by a mirror. There are two
frames of reference: an observer and the mirror. The observer sees a virtual
world behind the plane of the mirror. The mirror only “sees” the world in front
of it.</p>
So, abstract machines are the platonic idea of a machine capable of processing
a given language. Virtual machines are one of two ways to realize these
machines: by writing them as programs to be executed by another machine</em>. A
virtual machine acts like a mirror for the input language: the language “sees”
a machine designed just for it, but cannot see any of the implementation
details. For all the input language knows, it might be running directly on
hardware!</p>
(Our vocabulary gets a little muddled in practice: the “AM” in Erlang’s “BEAM”
runtime stands for “abstract machine” 2</a></sup>, while the “VM” in both the JVM
spec and</em> in LLVM’s name stand for “virtual machine.” In other words, the
boundary between “abstract” and “virtual” is a little blurry. Note also:
specific aspects</em> of a machine might be virtualized while others are left
transparent. For example, most programs assume a single contiguous view of
memory, which may exceed the actual memory available to the system. The
operating system virtualizes</em> this view for processes, working behind the
scenes to maintain the illusion.)</p>
The other way to realize an abstract machine is to reify it in hardware: to
build a processor to carry out the physical task of computing. A processor
doesn’t have to be electronic, that just happens to be the most common and
efficient computing substrate available to us today. You could, in theory, act
as a processor with a pencil, eraser, and a notebook if you wanted to –
assuming you followed some strict set of rules that described what you do</em>
with those items. That is, a processor still requires an input language with
rules. This input language takes the form of a contract between processor and
program: an instruction set architecture</strong> (“ISA”.) Examples of ISAs include
x86_64</code> and arm64</code> – it’s frequently abbreviated as arch</code>, or
“architecture” 3</a></sup>.</p>
The ISA acts as a “narrow waist</a>” (as in an hourglass): it contains just
enough</em> specification that many approaches to implementation are valid on both
sides of the contract. Software may expect that ADD</code> or JNE</code> exhibits the
expected behavior across implementations, while hardware is free to implement
that behavior as it sees fit. (AMD and Intel both</em> implement the x86_64</code>
ISA!) This is all, of course, subject to physical constraints – speed, power
draw, and thermal envelopes all come into play when implementing</em> an ISA 4</a></sup>.
So, while it’s possible to define an ISA that cannot be efficiently implemented
as hardware, people generally don’t do so on purpose</a>.</p>

Now, “Air Bud” 5</a></sup> rules apply here: software running against an ISA doesn’t know
what implements that ISA. It’s perfectly possible to virtualize</em> an ISA such
that the hardware is simulated by software 6</a></sup>. Software running on
virtualized hardware doesn’t know</em> the hardware has been virtualized: it can’t
see the mirror. This is how we get those shiny cloud VMs, VirtualBox, VMWare,
and QEMU “virtual machines”! Because we don’t make any promises about the
implementation, the virtualizer might take a number of different approaches. It might:</p>

interpret machine code as bytecode in software (“emulation”</strong>),</li>
pipe it directly through to its underlying ISA if those two happen to match
(“hardware virtualization”</strong>),</li>
or recompile the ISA at runtime if it doesn’t (“dynamic binary
translation”</a></strong> or “just in time” / “JIT”</strong> compilation)</li>
</ul>
Emulation is the most resource-intensive approach 7</a></sup>, but is the
most flexible, theoretically: you can emulate a powerpc</code> architecture on top
of your x86_64</code> processor in software. This comes at a cost! Emulation
introduces a relatively low performance ceiling compared to the hardware running
the emulator.</p>
Hardware virtualization is less flexible but more efficient: these are
processor instructions that aid in the creation of hypervisor software.
Hypervisors typically 8</a></sup> run in a lower protection
mode</a> than operating systems; they can be thought of as an operating
system for operating systems. Hypervisor software provides (near-)transparent
processor access for virtualized operating systems, while virtualizing other
system resources (like persistent disks, memory, and peripherals.) On x86
platforms</a>, this is provided by AMD’s AMD-V (AKA “Secure Virtual
Machine” or “SVM”) and Intel’s VT-x (AKA “VMX”), ARM has an
equivalent</a>. AWS, GCP, VMWare, the Windows Subsystem for Linux 9</a></sup>, Apple’s
Hypervisor framework: all of these use hardware virtualization. As mentioned,
this is generally</em> less flexible than other ISA virtualization strategies, in
that the ISA of the system virtualized must match the host ISA10</a></sup> – you can’t
virtualize an ARM machine on top of a physical x86 machine.</p>
That is, unless you’re running a dynamic binary translator (“DBT”) of some
sort. The most widely-deployed DBT ISA VM that I’m aware of is Apple’s Rosetta.
Rosetta 1 11</a></sup> eased the transition from PowerPC architecture to
Intel/x86 before its removal in OSX Lion, and Rosetta 2 is doing the same for
the Intel/x86 transition to Apple’s own ARM-architecture hardware. Notable
mentions include Intel’s IA-32 Execution Layer</a>, the Dolphin emulator’s JIT
recompilation of Gamecube/Wii PowerPC code to x86/ARM 12</a></sup>, and, of
course, the Transmeta Crusoe</a>. There are caveats with this approach: some
virtual instructions may have no hardware equivalent, and so must use (much
slower!) emulation in software. Most importantly, this is the most complex
solution to program: it requires a working emulator</em> to fall back on for
virtualized operations that have no native equivalent.</p>
DBTs are probably the most expensive virtualization technology to develop –
they are JIT VMs, which we’ll talk about again soon. Companies turn to them
when both the language to be virtualized and the target platform are fixed
points: IBM had</em> to support Power architecture, e.g. However, because the
addressable base of x86 machines is much larger, individual developers weren’t
likely to compile their software specially for IBM – there’s little incentive
for them to do so.</p>
Whew, okay, let’s take a breath and recap briefly. A “machine” is any
Turing-capable computer. An “abstract machine” is the conceptual definition of
such a machine over an input language. An instruction set architecture is a
contract language between the hardware of a machine and the input language of
that machine – software. To “virtualize” a system means to present an
interface to that system “as if” it existed using some mechanism. The more
transparent that mechanism is, the less performance will be lost in
virtualizing: that is, hardware virtualization support makes the processor
partially “transparent” to guest operating systems. On the opposite side of the
gradient: software emulation is fully virtual, straightforward to implement,
but slow compared to native execution. In the middle we have Dynamic Binary
Translation: translating a binary in one format “just in time” to execute it as
a native format. DBT JITs are expensive to write but potentially very
effective!</p>

Now, finally, input languages. Let’s revisit that “language virtual machine”
category of virtual machines we piled up earlier.</p>
As you might suspect, hardware only gets one</em> input language – an ISA
13</a></sup>. However, approximately no one</em> wants to write all
of their software</em> directly in</em> that one input language. For one, it’s a pain,
but more importantly, it’s intimately tied to the hardware. If, say, you have
to support another ISA, that means a ground-up rewrite (or hinging your
software’s value proposition squarely in the hands of a system virtual machine
vendor. Is your software really worth making your users simulate an entire
other machine and operating system just to run it? For my part, no
14</a></sup>.) Leaving that aside, domain specific languages are powerful for a
reason: they help us model the problem space of our software more directly.</p>
So, we invented programming languages 15</a></sup>. Each programming language
defines an abstract machine whether explicitly as part of a specification, or
implicitly as part of an implementation. That abstract machine can be implemented
in (roughly) one of three ways.</p>
INCOMING FOOLISH CONSISTENCY</a>. BRACE FOR IMPACT.</strong></p>
These three methods correspond to what we’ve already talked about</em> above: we
can emulate the language abstract machine in software, we can attempt to
translate it to native code at runtime, or we can map the input language
transparently to the underlying hardware. In other words: we can write an
interpreter (like Ruby, Python; an “emulator”), a JIT VM (like the JVM or
popular JavaScript engines; a “dynamic binary translator” of sorts), or an
ahead-of-time (“AOT”) compiler (as we have for C, C++, Rust, and others;
“hardware virtualization,” after a fashion, possible because the two machines
are so similar.) Interpreters are generally slow but portable and flexible, JIT
VMs are complex, potentially efficient, and generally require an interpreter to
fall back to, and compilers produce efficient translations at the cost of
flexibility.</p>
They are all</em> ways to virtualize a machine.</p>
If you’re like me, you’re probably accustomed to thinking of Erlang, C#, or
JavaScript as running atop a virtual machine. You might</em> even think “yeah,
Python’s interpreter constitutes a virtual machine.16</a></sup>” But Go? C?
C++? Rust? Compiled languages run on the</em> machine, not on a virtual machine,
right?</p>
Well, sorta. C’s specification, in fact, specifies an abstract
machine17</a></sup>. It is the job of compiler software to statically
map operations in that abstract machine to the corresponding operations in the
hardware – the machine as defined by the ISA, in other words. Rust, Go, Java,
and C# work similarly, in this fashion: a compiler rephrases the input language
into a form that can be easily translated to a target machine
(“lowering”</a> it). (In C# and Java’s cases, the target machine
happens to be a virtual machine: the CLR18</a></sup> in the former case, and the
JVM in the latter.)</p>
This is where that last category of examples comes in: LLVM and friends. These
“VMs” are intermediate</em> languages (or “intermediate representations”, or
“IR”), meant to facilitate the translation of one abstract language machine
into another. Rust itself has two (HIR and MIR</a>), many JS
engines</a> have their</a> own internal</a> IRs, and the JVM
bytecode specification is itself an IR. IRs narrow semantics, provide a place
to collect extra information about the program, and act as a single point at
which to apply useful optimizations 19</a></sup>. AOT compiled languages, like C, Go,
or Rust, contain sufficient information to be “lowered” to low level
intermediate languages, which themselves can be compiled to an ISA after
optimizations have been applied. Interpreted languages can be JIT compiled,
assuming the runtime can collect sufficient information to generate native code
20</a></sup>.</p>
I’m going to turn now to a paper from 1978 that I definitely did not read</em>
ahead of drafting this post:</p>

Thus a language definition implies a model of the machine on which programs
in the language will run. If a real machine conforms well to the model, then
an implementation on that machine is likely to be
efficient and easily written; if not, the implementation will be painful to
provide and costly to use.</p>
Via “Portability of C Programs and the Unix System”</a>, S.C. Johnson and D.M. Ritchie</p>
</blockquote>
I urge you to read this paper in full. When I read it, I spun around in my
office chair, arms raised in celebration for a full minute.</p>
Ahem.</em></p>
It’s a story about porting C & UNIX from the PDP-11 to the Interdata 8/32. I’ll
briefly summarize it here: it includes early automated regression testing, the
first tool called “lint”, the origin of several</em> concepts in C (including
unsigned integers, before which folks just had to type pun char*</code> pointers
because they happened to behave the same way</em> on the PDP-11), the above quote,
early virtualization, a description of using an intermediate language in a
compiler toolchain to port Fortran</em> to new machines in addition to C, and a
whole lot of new working systems spun up from existing, simpler working
systems</a>. It is just so</em> good.</p>
But the bit of it that is germane to this post is this: C started with an
implicit abstract machine, that mapped nearly 1:1 with the PDP-11. They
specifically picked</em> the Interdata 8/32 because it was</em> so different from the
PDP-11. In porting C, they modified the definition of C, using the large corpus
of existing software to validate the port and their changes. They built an
intermediate representation (though they didn’t call it that) and an abstract
model of the machine that the C language represented by extracting</em> them from
existing, working systems.</p>
It wasn’t just</em> luck that C’s machine model mapped so closely to the machines
they were trying to straddle: the authors explicitly call out that they had
previous experience porting languages between machines with C’s predecessor, B,
and the large software project, Altran (a Fortran library.) In other words, they
made good bets and were willing to coax the language the rest of the way. And in
shaving off those rough edges, they made an eventual port to a third machine that
much easier: C’s abstract machine became a self-reinforcing system.</p>

It is true that moving a program from one machine to another does not
guarantee that it can be moved to a third. There are many issues in
portability about which we worried in a theoretical way without having to
face them in fact. […] Nevertheless, moving UNIX
to a third new machine, or a fourth, will be easier than it was to the
second. The operating system and the software have been carefully
parameterized, and this will not have to be done again. We have also learned
a great deal about the critical issues (the “hard parts”).</p>
</blockquote>
That is: because C was so portable and had so much software already available
for it, ISAs were incentivized to map closely to C’s abstract machine. Because
hardware based on those ISAs kept advancing, there was more incentive to write
C (or languages that targeted C’s abstract machine.) ISA’s and C’s abstract
machine evolved symbiotically.</p>
And that draws me to the close of this post. We spent a lot of time defining
terms here: what a machine is, what it means to virtualize it, what an
instruction set architecture is, the differences and similarities between
virtualizing a system versus a language, implementation approaches and their
costs, and finally how to use a set of existing machines to create and refine
an abstract machine. WebAssembly, a self-billed “virtual ISA”, is following in
C’s footsteps here: extracting a reusable abstract machine from a set of
existing virtual machines. Next time, we’ll talk about the context Wasm was
developed in and how it aims to achieve Java’s goal of “Write once, run
anywhere.”</p>

(Thanks to Eric Sampson</a> for reviewing drafts of this blog post.)</em></p>
^1</sup>See “scope” under the w3 Wasm specification</a>.</p>
</div>
^2</sup>BEAM was a replacement for the original Erlang Abstract Machine:
“JAM</a>”, or “Joe’s</a> Abstract Machine”.</p>
</div>
^4</sup>You might have heard of “complete instruction set computers”
(“CISC”) and “reduced instruction set computers” before
(“RISC”</a>): this refers
to the approach taken in defining the ISA. Reduced instruction set computers
generally specify fewer</em>, more popular instruction codes in a more regular
fashion. For example, as opposed to a CISC architecture, a RISC machine may
not be able to combine memory access and computation in a single instruction.
The idea is that the reduced set of instructions narrows the focus of the
implementation, potentially making it faster. These days, ARM is considered a
RISC machine, while x86 is considered a CISC machine. In practice, ISAs
implementers may decompose operations into microcode invisible to software,
so the line is a little blurry. For an interesting read on the subject
from one of the folks behind the Transmeta Crusoe, see this paper</a>.</p>
</div>
^3</sup>You might have seen “arch” in various software releases
before, in the form x86_64-pc-win32</code> or arm-apple-darwin-eabi</code>. This is
known as a “target triple</a>” and is meant to communicate the
target processor, hardware vendor, and operating system.</p>
</div>
^{5</sup>
“There’s no rule that says dogs can’t play basketball!”</p>
</div>}^6</sup>Or indeed, using hardware to virtualize other hardware: these days the
x86 ISA is implemented as a frontend for other, vendor-specific processor microcode.
See “C Is Not A Low Level Language”</a>.</p>
</div>
^7</sup>Analogue’s Super nt</a> famously turned to Field Programmable Gate Arrays
(FPGAs) in order to fully replicate the behavior of the Super Nintendo
Entertainment System. There was a video that interviewed the person who implemented all of that
FPGA work, but it is sadly lost to me. Happily, you can find out more about the technical capabilities
over on the Digital Foundry video</a>.</p>
</div>
^8</sup>Type 1 hypervisors (“native”) run on the hardware beneath all
other operating systems: Oracle VM, VMWare’s ESXi</a>, Microsoft’s Hyper-V</a>,
and Amazon’s AWS Nitro</a>. Type 2 (“hosted”) hypervisors run as a process on
top of an operating system. Type 2 includes VMWare, Virtualbox, and Virtual
PC. It’s not clear to me at the time of writing whether Apple’s
Hypervisor.framework is a Type 1 or Type 2 hypervisor. I get the sense that
user-oriented operating systems are beginning to embed type 1 hypervisors
(e.g., Microsoft supports WSL on Windows using a subset of Hyper-V), exposing
the functionality out to the main guest OS as a framework.</p>
</div>
^{9</sup>
WSL2</em> uses hardware virtualization. WSL1</em> was implemented entirely
differently, and we’ll talk more about that approach in a coming post.</p>
</div>}^10</sup>For a really fun example of the inflexibility of hardware virtualization,
see the Wii U’s virtual Wii</a>
mode. It works because the Wii U is (roughly speaking) three Wii processors in a trenchcoat;
so to become a Wii, it hides two of those processors and a bunch of RAM.</p>
</div>
^11</sup>Rosetta 1, at least, is an implementation of Transitive
Corporation’s QuickTransit</a>
software, which Apple licensed. IBM acquired the company in 2009.</p>
</div>
^12</sup>Seriously. The Dolphin dev blog is a treasure trove of interesting
posts about various ISA JITs. See this one</a>
on adding an M1 backend to their JIT!</p>
</div>
^13</sup>Yes, yes, even if the hardware is the Transmeta
Crusoe. It has one input language, which happens to be a “very large
instruction word” (“VLIW”) set that can handle the union of a couple</em> of
other hardware ISAs. If you’re interested in this processor, read more about
it here</a>.</p>
</div>
^{14</sup>
Video games being the BIG exception to this. And even then, that
took some pretty anticompetitive practices to put in place.</p>
</div>}^{15</sup>
I’m being exceptionally</em> glib about this. The history of programming languages
is large and fascinating. It has a lot</em> to do with linguistics but very little</em>
to do with the point of this (increasingly long) post.</p>
</div>}^16</sup>See this wiki page</a> for an overview of
“ways to implement a language virtual machine.” Also, go read “Crafting
Interpreters”</a>!</p>
</div>
^18</sup>The “CLR” referred to here is Microsoft’s “Common Language
Runtime”, which implements the “Common Language Infrastructure” (“CLI”)
abstract machine. “CLI” is miserable to search for, though, so I’m committing
the sin of metonomy</a> by referring to the abstract machine language by its
implementation.</p>
</div>
^19</sup>I’m probably so out of date at this point, but see Sea of Nodes</a>,
“Static Single Assignment” (“SSA”)</a>, “Loop Invariant Code Motion” (“LICM”)</a>,
inline expansion</a>, and the “optimizing compiler”</a> page on wikipedia. It’s a
very interesting space!</p>
</div>
^20</sup>I’m leaving a lot out here, but to gesture at it: at least for
JavaScript, the job of a JIT VM is to run the language in multiple stages. In
the initial stage, it’s running as an interpreter, collecting type
signatures; if a function is called frequently enough, with consistent enough
type information, the interpreter code replaces that function with one in the
native machine’s ISA. This compiled code includes type checks that can cause
the function to “bail out” if unexpected types are encountered at runtime,
falling back to the interpreter. As I understand it, the JVM and CLR operate
in a similar fashion, though with more initial type information available.
The creation of JIT VMs owes a lot to the Smalltalk & Self lineage of
languages; Strongtalk, in particular, informed the design</a> of many</em> vms.</p>
</div>
^21</sup>Hardware virtualization involves the use of a program called a
“control program”, “hypervisor”, or “virtual machine manager” (“VMM”),
usually running in a lower protection
mode</a>
than the kernel itself.</p>
</div>
^{22</sup>
An aside here. I’ve just described two more</em> subtypes of VMs here:
process</em> VMs and system</em> VMs. System VMs run outside of the context of an
operating system: a hypervisor, loaded as part of the hardware boot process.
Process VMs, however, include QEMU, VMWare, Python, V8, SpiderMonkey: any VM
that runs as a process managed by an operating system.</p>
</div>}^23</sup>See “Abstract Machines”</a> on
Wikipedia. Also, go read “Abstract Machine Models”</a>.</p>
</div>
^{24</sup>
Languages without specifications still</em> have abstract
machines, but they’re not written down anywhere: they live in the heads of
the maintainers of the canonical implementation. The Ruby and Python machines
are “that feeling you get when you read the entire codebase”, the wind in
your hair on a pleasant Spring day.
Of course, that means accurate, alternate implementations are that much more
difficult to write.</p>
</div>}^17</sup>See the freely available
copy</a>
of the C standard, §5.1.2.3, which states:
“The semantic descriptions in this International Standard describe the
behavior of an abstract machine in which issues of optimization are irrelevant.”</p>
</div>
^25</sup>Via “C Is Not a Low-level Language”</a>.</p>
</div>


A Mirror for Rust: Compile-Time Reflection Report - Shepherd's Oasis
Unknown — Mon, 08 May 2023 00:00:00 +0000
A long read, but a good read. Covers the use of syn</code> and reflect</code> in the Rust ecosystem,
what those tools are in reaction to, and the limits of that pattern. Proposes a compile-time
reflection system with low- and mid-level APIs that would (in theory) allow Rust to get away
from the widespread peer dependency on serde</code>.
Linked terms include:</p>

SFINAE</a></li>
dtolnay’s reflect</code> crate</a></li>
dtolnay’s syn</code> crate</a></li>
bevy’s bevy_reflect</code> crate</a></li>
C’s P99 macro library</a></li>
the Rust offset_of!</code> macro PR</a></li>
</ul>


In defense of blub studies | benkuhn.net
Unknown — Mon, 08 May 2023 00:00:00 +0000
This post talks about the importance of schlepping through the details of the tools you’re working with. As you can probably
guess, I love it. It really rings true for me: the times I have learned the most as a programmer are the times I explicitly
said “heck it, I’m going to write my own ORM / template language / implementation of the deflate algorithm.” Or “I’m going to
learn how Django works by reading the source</em> for the area I’m interested in.”
It’s important on multiple levels – “blub studies” make you engage deeply with your tools and build a theory of how they work,
which inspire you to build new tools, save you time debugging (as the author notes) or help you put the tool in context.
This link comes to me via simplermachines</a>, which is another great blog I urge you to follow.
Linked terms include:</p>

“Blub”, via “Beating the Averages”</a>, a Paul Graham essay. (So, uh, take that for what it’s worth.)</li>
“An Introduction to Computer Networks”</a>, which might be just the thing</strong> I’ve been looking for, re: network engineering.</li>
</ul>


6.6
Unknown — Fri, 16 Dec 2022 00:00:00 +0000
This used to be my go-to “I am entirely burnt out and I give up”
song. (It’s maybe alarming that I had a “go-to” song for that? Hm,
hm.)
I’m happy to report that I play it a LOT LESS these days than I used
to, but whoo.</p>


Dead 80's | Mrs. Magician
Unknown — Fri, 16 Dec 2022 00:00:00 +0000
“On new year’s eve, I met a girl / she scared all my friends and then she showed me a whole new world”
Love this track just for that line. Oh, and the easy nihilism and jangly guitars.</p>


Don't Try | Radioactivity | Dirtnap Records
Unknown — Fri, 16 Dec 2022 00:00:00 +0000
I didn’t come by this track honestly! This popped up on Shanti (or
Kevin?)’s Spotify playlist while they were driving Krysten and I
around Palm Springs in a rental car. I’ve been in a strange state of
mind for a while: I’m starting to physically feel older, my job (at
the time) wasn’t really working out, and, to be frank, folks: my
brain just no good no more. Years of pandemic and years of being
glued to Twitter waiting to see what new outrage Trump would drum up
and years of working from home and being separated from friends and
family have done a number on my good brain juice.
But this song popped me right out of that torpor by collecting all of
the melancholy-angst that had been building up inside of me and just
wailing it out. I leaned to read the name of the song and artist from
the back seat and mumbled it to myself until I could get home so I could
write it down in some kind of computer. And now I’m writing it into this
computer!</p>


JAPANTHER - FIRST OF ALL | JAPANTHER | Seayou Records
Unknown — Fri, 16 Dec 2022 00:00:00 +0000
Man. What a great “I’m incredibly frustrated” song. And at the same time,
I wish it wasn’t on repeat for me? At least it dispaced 6.6</a> for me.</p>


Never Again | Thin Lips
Unknown — Fri, 16 Dec 2022 00:00:00 +0000
“Raw and powerful”, unrequited love, alliteration. It just sounds
like being in your early twenties. At least, it sounds like being in
my</em> early twenties. Or how I wanted</em> my early twenties to sound
when I was in my early twenties? I don’t know. I’m in my late
thirties as I write this. I can barely believe I used to stay out
past midnight.</p>


Nostalgia of Island | Tatsuro Yamashita
Unknown — Fri, 16 Dec 2022 00:00:00 +0000
Got this from one action button video or another. I feel like I should have more
to say about this, but… yanno, some music just takes you somewhere. I’m happy
I went on this trip.</p>


Simulation Swarm | Big Thief
Unknown — Fri, 16 Dec 2022 00:00:00 +0000
What a big, warm, sad sound.</p>


Something About Lemons | Chumped
Unknown — Fri, 16 Dec 2022 00:00:00 +0000
I am an ABSOLUTE SUCKER for any song about it being hard to get out of
a town that folks stay in. I think about Lawrence KS in those terms a lot,
though in practice most folks I know had already moved out of there before
me.</p>


x-post: LARPing a Gossip Protocol about Reality
Unknown — Fri, 04 Nov 2022 00:00:00 +0000
This was originally posted on my Cohost account</a>; I’ve reposted it here
for posterity.</em></p>
LARPing a Gossip Protocol about Reality</h1>

I’ve edited and re-edited this and I don’t feel super comfortable sharing it
but I need it out of my head sooner rather than later. I need that brain space
back. And besides, I want to move on to post things that aren’t</em> about other
social network platforms on this social network platform. So. Here goes.</p>
</small>
Twitter changed how I write, for certain. Twitter also changed how I think.</p>

I spent more than a decade squeezing every stray thought into Twitter. Every
evening, morning, and spare moment scrolling to see what folks were talking
about. Minutes in the shower thinking about my</em> position on the topic of the
day, then how best to cram that into 140, then 280 characters. “Hold on a
moment,” I’d say to myself mid-anything-at-all-really, “Has anyone made
the ‘Node dot michael jackson’ joke yet?” 1</a></sup></p>
For a decade we talked to each other in tabloid headlines. It was a place to
make dumb jokes, then watch likes pour in. To make even dumber jokes, and then
watch nothing at all happen. To collectively gawk at the news of the day. To
stay up late at night in fear of a war with North Korea. Twitter was the place
I went to feel</em> social without having to actually be social</em>.</p>
Twitter made me realize that hundreds if not thousands</em> of people would,
given the same relative prompts, produce the same dumb joke. Myself included.</p>

You know what was new to me in the early-to-mid 00’s?</p>
The idea that I could quantify</em> the approval of my peers. Used to be, you’d
write something, put it out on the internet, and you might never even know if
anyone cared about what you made. Sure, you could stare at Google Analytics,
but all that told you was that “they” saw</em> it. And who are “they”, anyway? I
had no way to know. You might get a comment or two on a link aggregator site,
but largely… nothing.</p>
I graduated into the 2008 recession. At that same time, I lost my first
programming job. I was unemployed for the first time in my life (for what would
be, to date, the longest stretch of time in my life.)</p>
I was scared</em>. I watched the savings I had put together to move to Portland
dwindle perilously close to empty. This stretch of unemployment was a formative
experience (even despite the number of safety nets available to me!)</p>
The ability to quantify approval came to serve a completely new purpose for me
at this point, one apart from feeding an annoyingly deep-seated personal need
for external validation. I began thinking that “the more people know and
approve of my work online, the less likely I am to be unemployed in the
future.” This wasn’t a unique thought, really, lots of other people who had
similar context came to the same conclusion. But Twitter hadn’t risen to
popularity yet, so how was I to know?</p>
I resolved to try to do a lot more of my work in public, and in particular, to
make things for platforms where I could measure the return on my effort</em>. This
former was a conscious decision, the latter, less so.</p>
At the time, it generally seemed like the things I wanted to do anyway were the
sorts of things that were getting approval online. I wasn’t concerned with the
thought that I would end up tailoring my creative output to fit any particular
platform. The internet was still pretty weird in the mid-to-late 00’s! (It
continues to be  weird now, but, like, in the sense that you might see Arby’s
flirt with Steak’ums.)</p>

When I joined Twitter in 2009-ish, it was the exact opposite</strong> of compelling:
it felt like being able to group text people I could already group text</em>. My
digital social network perfectly overlapped my real, physical social network.
It felt forced; a parody of a social network, good only for dodging phone
carrier SMS fees.</p>
More to the point, it felt like a fad. At one point, my boss at the two (2!)
person local startup I had mercifully landed at introduced a twitter-alike
microblog system to work – so we could micro-post our current working status.
Our desks literally faced each other in an office the size of a closet. Such
was the power of the fad at the time: we dutifully microblogged status updates
we could have literally just said out loud to each other, face to face. We
didn’t just work on the cutting edge – we lived</em> on the cutting edge. And it
hurt</em>.</p>
We eventually ditched that microblog. There wasn’t enough of a social network
on that social network. And so it was with Twitter at the time.</p>

The inflection point came shortly after I moved to Portland in 2012. I still
remember: buoyed by a digital social network I had grown around Node.js via
IRC, I started to weigh in on the then-nascent ECMAScript modules debate on
Twitter. Not only that, I was weighing in on it from karaoke. Over a beer</em>.
And I was getting responses from the guy who designed JavaScript</em> 2</a></sup>. And
another guy, whose code I idolized</em>. And he liked my tweets</em>.</p>
Reader. Let me tell you. Through a combination of this ill-conceived platform
and the mundane ignorance of the privilege I benefit from as a straight white
cis guy, I had gained entry to a conversation I had absolutely</em> not paid the
table stakes to participate in.</p>
I was hooked</em>.</p>

Twitter was a free copy editor: the constraint of 140 characters encouraged
pithiness, impact, and comedic pacing. I stop short of saying it made me a
better, more thoughtful writer, but the medium certainly</em> shaped the messages
I wrote.</p>
This helpful editing advice wasn’t free</em>, to be clear: Twitter got something
out of it. The constraints made it easier for people to create content; they
also put an upper limit on wordiness which made it easier to feel okay with
posting something short</em>. The medium itself helped create an environment where
content flowed infinitely; it helped us build our own skinner box. More tweets,
more news, more jokes, more opportunities for validation.</p>
I (and others) often complained about conversations on Twitter losing “nuance.”
Why were we all so okay sacrificing nuance so readily? Well, nuance doesn’t
count for much if you don’t feel like you can be heard in the first place, so.</p>
To be honest, it felt good</em> to tweet through it, sometimes. It was cathartic.
But it also frequently felt less like interacting with other people</em> and more
like feeding a machine: a machine hungry for attention, eyeballs, tweets,
retweets, stories, feelings, hot takes</em>.</p>
I often found myself editing my thoughts down to tweet size so I could convey
them on Twitter. The medium became the measure of my thoughts. I’d find myself
thinking something insipid about the tactical branding of men’s soap while
shopping at Target (now with Mil-Spec (TM) paracord soap holder) and think,
“I’d like to tweet about that later.”</p>

When a measure becomes a target, it ceases to be a good measure.</a></p>
</blockquote>

I don’t actually</em> want to tweet about that later. I don’t think I want
to tweet anymore, at all. I’d like some of my brain back, please.</p>
</small>

Four things.</p>
One:</strong> Twitter’s best feature was its biggest liability. Anyone can talk to
anyone at any time about anything. Anything you say has the potential to reach
audiences both friendly and unfriendly at any point over time. You are close to
everyone at the same time that you are speaking to no one in particular. You
get to choose the sound bite, but you can never give it context. You are
LARPing a gossip protocol about reality. Most of the time this is fine. Some of
the time this is fun.</p>
Two:</strong> Some of the time, this is dangerous. There are people who know this and
want to exploit split brain situations on purpose. Or create them. Who looked
at a thousand people making the same joke and thought “okay, how do we control
the punchline?”</p>
Three:</strong> worryingly, there’s some
evidence</a> that nonlocal
interactions, rather than echo chambers, increase polarization and violent
partisanship. Being able to quantify validation doesn’t just apply to artists
and writers, it applies to people who crave violence against “the other”. It
gives them a mechanism to receive encouragement and a built-in, captive
audience of targets to lash out at.</p>
Four:</strong> ad-funded social network platforms optimize for engagement. Content that
prompts anger draws shares and eyeballs faster than anything else. Thus,
platforms are incentivized to optimize for as little moderation as possible
while still being able to sell ads. Meanwhile, trolls optimize for creating as
much anger as they can while dodging the (scant) moderation in place.</p>
Twitter doesn’t bear the nickname “hellsite” for nothing. Over time, the format
and the incentives of the platform make it a factory for fascism.</p>
And if you couldn’t help but insert “nothing wrong with me” after
reading each number in the prior paragraphs, a la Drowning Pool’s “Let the
Bodies Hit the Floor”: condolences, we think alike.</small></p>

And yet. Something’s got to give, now.</p>
Twitter was a digital third place</a>.
I am sad at the propsect of seeing it fade into the digital entropy that
consumed IRC, Digg, Xanga, Livejournal, Starsiege: Tribes, and Tripod before
it.</p>
At its height, I could go to Twitter to ask questions, look for jobs 3</a></sup>, goof
around, expand my worldview, find out about trends in my profession, find
interesting artists, find new indie games, and keep up to date with world news
(through perspectives completely different than mine!) And, yanno, read dril
tweets.</p>
But it really wasn’t healthy</em>, at least not for me. And I don’t know if I want
to rush to replace it with something similar.</p>

Thanks to Jeff Lembeck</a> and Krysten for reviewing this</p>
</small>

^{1</sup>
because ES modules were getting node support and they were going to
change the file extension for those modules to “.mjs”, get it, and oh my
god what have I done with the last ten years of my free time</p>
</div>
^{2</sup>
who would later be revealed to be a bit of a turd</p>
</div>
^{3</sup>
isn’t it strange how much better Twitter was at this than LinkedIn?</p>
</div>}}}


"The UNIX Programming Environment", reviewed
Unknown — Sun, 21 Aug 2022 00:00:00 +0000
Why should you read 1984’s “The UNIX Programming Environment”</a>
thirty-eight years after it was first published? Why did I</em> read this book?
(Well, I am an unrepentant fan of bash.) Is it even relevant anymore? (Yes.)
Why should you read it?</p>
In order to understand where we’re going as an art, it’s important to know
where we came from. It’s 2022 and I still SSH to boxes, edit files using vim,
and write bash scripts. To be honest, sometimes it feels like the only that’s
changed in the fifteen years is the amount of YAML I have to wrap my bash in.</p>
So yeah, as the Palmolive commercial says, “You’re soaking in it!”</a>
This 1984 book remains relevant: most tools we use are either from this era,
reactions to these tools, or reactions to the reactions. Tools written outside
of this conversation exist and can be effective, but generally they feel
slightly off. They don’t integrate with other tools like they should.</p>
Writing tools that coöperate fluently with each other gives you an opportunity
to leverage network effects; to make all existing tools more useful with your
tool.</p>

So. That all being said, let’s talk about the book itself. “The UNIX
Programming Environment” is short and sweet, clocking in at a little over 250
pages. Each chapter introduces a new topic, explores how it interacts with
topics from prior chapters, then raises issues for the next chapter to resolve.
The book starts with the key innovations of the UNIX operating system and
builds from there to illustrate a “blessed path” for program development:</p>

The filesystem</li>
Using the shell</li>
Filters</li>
Shell programming</li>
The process environment: using stdio</li>
System calls</li>
Program development</li>
Document preparation</li>
</ul>
Some chapters are more relevant to contemporary readers than others. In
particular, while the section on document preparation is interesting from a
historical perspective, unless you’re writing man</code> pages by hand, you probably
won’t have occasion to use the tools described there.</p>
It’s worth noting at this point that this book wasn’t written during the early
development of UNIX. Rather, this book was written 11-15 years later, when UNIX
had already escaped containment at Bell Labs and was rapidly growing in
popularity.</p>
Quoth the book, in an example given in a late chaper:</p>

John Lions taught the 6th Edition UNIX kernel in an undergraduate OS course
at UNSW. In notes prepared for the class, he wrote, “the whole documentation
is not unreasonably transportable in a student’s briefcase.” (This has been
fixed in recent versions.)</p>
</blockquote>
(You can expect this level of self-effacing, dry humor throughout.)</p>
The section on program development gleefully sings the praises of yacc</code>,
sharing the fascination with parsing so common in texts from this era. The C
code listings are somewhat less directly applicable today than the shell
listings, but the progression from “using the shell” to “using filters” to
“programming the shell” to “writing the program in C with system calls</a>” does an
excellent job of illustrating the third point from the oft-quoted,
frequently-truncated UNIX philosophy. I include the full treatment of the
philosophy from the book itself:</p>

First, let the machine do the work. Use programs like grep and wc and
awk to mechanize tasks that you might do by hand on other systems.</p>
Second, let other people do the work. Use programs that already exist as
building blocks in your programs, with the shell and the programmable filters
to glue them together. Write a small program to interface to an existing one
that does the real work, as we did with idiff. The UNIX environment is rich
in tools that can be combined in myriad ways; your job is often just to think of
the right combination.</p>
Third, do the job in stages.</strong> Build the simplest thing that will be useful, and
let your experience with that determine what (if anything) is worth doing next.
Don’t add features and options until usage patterns tell you which ones are
needed.</p>
Fourth, build tools. Write programs that mesh with the existing environ-
ment, enhancing it rather than merely adding to it. Built well, such programs
themselves become a part of everyone’s toolkit.</p>
</blockquote>
I invite you, dear reader, to make a comparison to pithier, widely-known Doug
McIllroy version</a>. The book’s version of the philosophy resonates with my
experiences as a professional programmer for the last sixteen years. These are
words to live by!</p>

My goal in writing this review —other than to defend an otherwise
indefensibly wanton habit of purchasing classic Computer Science textbooks from
my local used bookstore— is to nudge</em> you, dear reader, into leafing
through this book, perhaps on a lazy day off. The UNIX Programming Environment
conveys the enthusiasm and laughing regret of folks who’ve worked within a
particular moment in computer science. It’s a great jumping-off point for
future research, while connecting the dots between many of the disparate tools
you use every day.</p>
So, I leave you as the book does, with an aside about ed</code>. One of the most
interesting things I learned from this book (aside from the fact that stdin</code>
is the result of open("/dev/tty")</code>, and stdout</code> is the result of dup(1)</code>, so
you can write output to stdin) is the relationship between ed</code>, grep</code>, and
sed</code>. You may be aware, as I was, that grep</code> was derived from ed</code>; but I was
surprised to find that grep</code> had wayward siblings: gred</code> and gres</code> – all
derived from the ed</code> incantation, g/re/*</code>.</p>
Eventually all of the variant commands were wrapped into a successor program,
sed</code>. Where this becomes useful for me: my editor, neovim</code>, descends from
vim</code>, which descends from vi</code>, which descends from ex</code>, which descends from
ed</code>. My muscle memory for my editor translates directly into using sed</code>! And
likewise, everything I learn about sed</code> carries back into my editor.</p>
Network effects are powerful!</p>


(2019) Keynote: The Path to Components - Luke Wagner, Distinguished Engineer, Fastly - YouTube
Unknown — Tue, 02 Aug 2022 00:00:00 +0000
One of the best explainers on the state of WASI/WIT/WASM components I’ve seen.
This is part of an interesting trend: the industry is starting to revisit bedrock abstractions. Systemd started doing this
with single-machine process management, Kubernetes did it with cluster management, Aurae is starting to look into what it looks
like for larger clusters. At the same time that we’re having this renaissance in service orchestration, WASM components are relitigating
inter-process communications and resource sharing (“What if programs didn’t call each other by file system location, but instead by calling
directly into each other? What if they didn’t have to serialize state into text?”)
I am much more hopeful about the hypothetical future of orchestrated WASM binaries than I am about the container landscape, to be honest. Docker
left behind quite a crater.</p>


IPv6 Internet Is Broken
Unknown — Tue, 02 Aug 2022 00:00:00 +0000
Fascinating stuff. I’ve spent half of this year picking up the rudiments of network engineering,
and it’s absolutely fascinating to me that IPv6 attempts to return to the “end to end” principle
we lost with IPv4 NAT. (Peering disputes around IPv6 are only tangentially related to this, of course,
but two neurons grazed past each other when I read this.)
Of note (for me) are the other links on this post – the “looking
glass” sites for those providers are new to me!
Also, the “cheap VPS” and “subnetting cheatsheets” seem handy.</p>


Refactoring Is Not Just Clickbait - Kevlin Henney - NDC Oslo 2022
Unknown — Tue, 02 Aug 2022 00:00:00 +0000
Great talk; I especially enjoyed his “three axes” approach to describing changes to programs.
He also mentions Herbert Simon, whose writing on systems I am now enjoying.</p>


Running LXD Containers in Macpine
Unknown — Tue, 02 Aug 2022 00:00:00 +0000
This is pretty rad.
I ended up wanting to ssh</code> directly into my containers; in order to
do this I had to edit /etc/ssh/sshd_config</code> on the macpine box
(using alpine ssh lxd</code>.)</p>
AllowTcpForwarding yes # change this from "no" to "yes" in /etc/ssh/sshd_config</span></span></code></pre>
After running /etc/init.d/ssh reload</code>, I was able to use the macpine LXD VM as
an ssh jumpbox (via ssh’s handy -J</code> option.)</p>


So you're Staff+ ... now what?
Unknown — Tue, 02 Aug 2022 00:00:00 +0000
This is one of the most useful talks I’ve seen. I acted as a principal
engineer at Eaze for a little over a year; early on I struggled with the
role because it wasn’t clear how involved I should get with the various
teams I floated between.
Sabrina Leandro outlines a great way to build accountability for yourself
that empowers you to work directly in different codebases while also giving
yourself headroom to do the planning, research, and coordination that come
with the role.</p>


The Continuity of Splines - YouTube
Unknown — Tue, 02 Aug 2022 00:00:00 +0000
What a presentation! I only took calculus up to “engineering calc 2” level in college,
which leaves off approximately where this picks up – that is, calculus for parametric equations. That is to say, just when math starts
to get really interesting, if you’re into gamedev.</p>


The Rust Programming Language for Game Tooling
Unknown — Tue, 02 Aug 2022 00:00:00 +0000
It’s great to see Rust adoption at game studios. I appreciated the
link to gimli</code> and iced</code>. His recommendation to start building rust
in a contained fashion – in a corner of the codebase – matches my
experience at Eaze. I definitely did the same there: starting small,
nerd-sniping a colleague with a standalone CLI tool, then rolling it out
to more and more developers.
Also, he mentions that his favorite web framework is Tide! (I love to see
it, even if I’m not using Tide since Tokio won the async wars.)</p>


toast - crates.io: Rust Package Registry
Unknown — Tue, 02 Aug 2022 00:00:00 +0000
Another super-useful tool from Kevin’s menagerie of good Rust links.
This allows easy cross-compilation using containers.</p>


Watchexec
Unknown — Tue, 02 Aug 2022 00:00:00 +0000
Wow, this is great. I’m using it already to power my link dump parser.</p>


Color Studies
Unknown — Sun, 20 Feb 2022 00:00:00 +0000
Color Studies</h1>
TKTK</p>
After the last post, I want to back away from tech for a bit, so I’m writing about color palettes today.</p>
I used Colormind</a> to derive palettes from images that
resonated with me.</p>

sonic cd: stardust speedway zone</h2>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>

I’m kicking things off with a bunch of 16-bit Sonic screenshots. It’s probably
no surprise that I love the art direction of this series; it’s one of the first
things I remember identifying as “futuristic” in the early nineties.</p>
This shot, in particular, is from Sonic CD, a game on a console I only managed
to rent once. The color palette is doing a TON of work here: it feels frenetic
and cosmopolitan. Depictions like this really formed my perception of what a
modern metropolis should look like!</p>
(It bears mentioning: I love a dark palette with pops of purple and green.)</p>

sf sunset</h2>
</div>
</div>
</div>
</div>
</div></div>

I found this via this tweet</a>.</p>
A strong blue-orange composition, sticking towards the midtones!</p>
I’m tickled that Colormind detected the yellow street sign and threw it into the
palette as a little burst of green.</p>
I find these blue-orange palettes captivating: they can tilt warm or cold with
only a little nudge, melancholy or energetic, even futuristic (thinking back to
LEGO’s “Ice Planet
2002”</a> line from
the nineties.)</p>

sonic 2: chemical plant zone act 1</h2>
</div>
</div>
</div>
</div>
</div>
</div></div>

Increasingly displaced by blue-white LED lighting, sodium lamps used to throw
their golden hue lazily over city blocks. Chemical plant zone captures this;
it’s kind of an oddball arrangement with the warmer tones in the background
getting covered up by higher-energy bright colors, grays, and blues.</p>
A near-black red sky is at once ominous and comforting.</p>

sonic 2: chemical plant zone act 2</h2>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>

As the aforementioned level plays forward, you end up getting closer to the
eponymous pink goo. The colors skew cooler here; even the pink tends a little
towards the blue side.</p>

doom eternal</h2>
</div>
</div>
</div>
</div>
</div>
</div></div>

Another strong blue-orange composition; it’s fascinating how little one has to
lean towards the blues to drop the temperature sharply.</p>

metroid by mostly ghostly</h2>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>

A great
illustration</a>
by LowBudgetGhost. (I love this illustrator’s work!) I picked this piece because it’s indicative
of their other work: strong mid-to-light tone colors vibrating against each other.</p>
Go check out their work!</p>

chicago winter storm tweet</h2>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>

A really lovely
photo</a>
of a chicago winter storm.</p>
This just popped across my feed and reminded me of an ill-fated winter visit to
Chicago from years past (one of many, to be honest!) In particular, I love how
this palette captures the juxtaposition of those snowy whites and grays against
the warm, again ominous sky.</p>

lawrence ks sunset</h2>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>

Home sweet home. Lawrence is a small Kansas college town where I lived from
2004-2012. This shot is from Summer 2009; a view from the hill, where the
college is located, down to the main street of the town.</p>

lift in palm springs</h2>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>


weeping alaskan close up</h2>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>


blue sky</h2>
</div>
</div>
</div>
</div>
</div></div>


halloween jack o lantern</h2>
</div>
</div>
</div>
</div>
</div>
</div></div>


neon blinds</h2>
</div>
</div>
</div>
</div>
</div></div>


Tampa</h2>
</div>
</div>
</div>
</div>
</div></div>

Can I just say that I’m really chuffed that Colormind picked four beautiful
colors and “swamp green” for this picture?</p>
(This picture actually got me back into liquid inks – mostly to play with
masqueing fluid!)</p>

Budapest Alley, at Night</h2>
</div>
</div>
</div>
</div>
</div></div>

I took a trip to Budapest a few years back – my first overseas flight, with
Krysten and some friends – and took this photo looing down from the back of
our hotel on our first night.</p>
This picture has a pretty sedate palette: purple and yellow bounce off each
other, but since they’re not super-saturated they end up feeling pretty chill.</p>

Budapest Sunset One</h2>
</div>
</div>
</div>
</div>
</div></div>


Budapest Sunset Two</h2>
</div>
</div>
</div>
</div>
</div></div>


1208 Room, Markers</h2>
</div>
</div>
</div>
</div>
</div></div>

An illustration I did of my old room from 2007. The room was a mess and I think
the colors faithfully reflect that. The room in question had a bright green
carpet set against burgundy walls, and the only source of light at night was a
cheap Target paper lamp, giving off warm incandescent light. As a result, you
get light purples, oranges, olives, and greens. The red of the wall isn’t
present, thankfully.</p>
I particularly like the pastel-violet to mustard yellow transition.</p>

Captain Yajima Splash</h2>
</div>
</div>
</div>
</div>
</div></div>

From Worthikid’s “Captain Yajima”</a> short animation.</p>
Another pulp-comics-inspired palette. I love the poster colors – the red,
green, and eggshell white are out in front, grabbing your attention. The black is
tinged a sickly green, like the black ink on neon green you’d see for a band flier.</p>
I like that this palette gives you three different ways to grab attention.</p>

F-Zero: Big Blue</h2>
</div>
</div>
</div>
</div>
</div></div>

The nineties aesthetic loves a beach. We hit a bunch of loud, hyper-saturated colors
here: deep blues, palm tree greens, and neon oranges. The rambunctious colors are grounded
by a staid, almost wine-like, magenta.</p>

Demon’s Crest Town</h2>
</div>
</div>
</div>
</div>
</div></div>

A game I really loved as a kid.</p>
Demon’s Crest is a SNES Capcom game set in a strange, ruined medieval world
ruled by demons. The environment design is top-notch; it really shines in the
early game when you’re wandering through ruined towns and overgrown cemeteries.</p>
Demon’s Crest strikes a balance between eeriness and melancholy. It’s probably
no surprise they’re using a saturated purple-green palette to play up the pulp
horror vibes.</p>


Portland Springtime Cherry Blossoms</h2>
</div>
</div>
</div>
</div>
</div></div>

I found this via this Reddit post</a>.</p>

Portland Springtime Cherry Blossoms Part Two</h2>
</div>
</div>
</div>
</div>
</div></div>

Found via this Reddit post</a>.</p>

Deep dream rainbow</h2>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>

Found via this Tweet</a>.</p>



The Canvas Cares Not
Unknown — Sun, 30 Jan 2022 00:00:00 +0000
The Canvas Cares Not</h1>
Computers don’t care how programs are structured. Businesses don’t care how
systems are architected. The canvas does not care how paint is applied.
Software engineering is an art.</p>
Software engineering is also a culture of artists; despite that, we recoil at
being called “artists.” Art has no right answers: a work of art means something
different to everyone who receives it. Software engineering denies that it is
an art form, which leads to a deep-seated intellectual insecurity. It
fetishizes science.</p>
Until we come to terms with the reality of the profession, we delude ourselves.
Our output, our programs, our systems suffer for our deliberate
misunderstanding. What’s more, in our pursuit of the trappings of science, we
inflict suffering on one another needlessly.</p>

A Front-Row Seat at the “Best Art” Awards</h2>
First, art. Why do I say software engineering is art? Art is communication:
between people, between the artist’s past self and present self, between the
author and the audience. Music is art, prose is art, cooking is art,
architecture is art.</p>
Art is forever imperfect. It leans on shared context. Art is always “in
reaction to” itself: an infinite regress</a>.</p>
Software engineering culturally focuses on the utilitarian byproducts of art:
the food the cooking produced, the building the people live in, the album the
musician pressed; we are obsessed with the utility produced while we ignore the
aesthetic involved.</p>
Aesthetics are, broadly, the values we bring to art: they’re how we decide what
is “good art” and what is “bad art.” The concepts of “bad” and “good” are
personal, not universal: it is entirely reasonable for two people to disgree
about whether a book is bad, or whether an API is overengineered.</p>
The art of software engineering is the voice of the engineers behind the code.
The communication of the history of the system, the decisions made and options
weighed. Every single engineer has different tastes, different aesthetic
values, a different voice.</p>
Focusing solely on the utility that we produce ignores the fact that we have</em>
tastes — that we have a voice</strong>. We are, in fact, explicitly in the
practice of pretending we don’t</em>: we commonly defend our taste by lashing it
crudely to quantifiable, universal truths. It is not enough to simply dislike
leaving semicolons out of your JavaScript, it has to go hand in hand with the
claim that “automatic semicolon insertion leans on error-correction code in the
parser, and is thus a program that does not include semicolons is less
technically correct</em> than a program that includes semicolons.” It’s a tenuous
conclusion; one that leaves no room for people who don’t</em> like semicolons to
be right. And yet people of both tastes write programs successfully every day.</p>
There is no such thing as a universal “best tool for the job” because our job
is to communicate</em>. There exists no meaningful award for “best art” because,
as a form of communication, art is intrinsically subjective. So it is with
programming.</p>

A Zero-Sum Game</h2>
Software engineering is, as a culture, intellectually insecure.</p>
We hold technologies in our periphery with some imagined faint praise or
damnation.</p>
We hold technologies in our direct line of sight with some self-assessed
— but again, imagined — level of expertise, while we gloss over
their essential complexity.</p>
We imagine that our grasp of complex systems and topics grants us universal
insight across all complex systems. We generalize prematurely. We pattern-match
like a cheese grater, dragging complex ideas across the plane of our
understanding until they come through the other side misshapen.</p>
To some degree we do all of these things because we are human. But we make the
mistake of thinking that we are special humans. That we are on one end of the
“bell curve of intelligence”, as if such an abstract concept could ever
honestly be made concrete (don’t @ me about IQ tests, they’re bullshit.)</p>
We’re told we are the smartest and the best and that we should seek to work
only with</em> the smartest and the best, but:</p>
We don’t</strong> know everything and we’re constantly</strong> reminded of just how
average we are; just how small our understanding is. And if our understanding
is small, then maybe we aren’t the smartest and best. And if we’re not the
smartest and best, then no one wants to work with us. If no one wants to work
with us, well, then, shelter and food are going to be a problem.</p>
It is far easier to react defensively to that insecurity than to embrace it.</a></p>
What makes us so rigid? Who told us to be this way?</p>
Software engineering is art, yes, but it’s also a job. The utility of our work
attracts capital, and capitalists have have found it useful to tell us how
great we are. Probably because it makes us turn a blind eye to the
externalities of the systems we produce.</p>
Because the utility of the systems we produce is the sole focus of capital
— our sole value to business — we are systemically encouraged to
avoid</em> thinking of ourselves as artists, to recoil</em> from the thought of
having aesthetics.</p>
Developers internalize this conflict: we have druthers, preferences, etc., but
we’re told that we shouldn’t. Because we shouldn’t have anything as silly as “a
preference about semicolons”, we try to rationalize it by picking some concrete
truth and lashing our foibles to it. We’re sandbagging our opinions both from
each other and ourselves.</p>
And so, as a rule, we’re insecure.</p>
As a result, we’re constantly fighting both for and against our aesthetics and
for and against the aesthetics of others. And our desire to continue drawing a
salary prevents us from embracing the simple fact that we even have</em>
aesthetics in the first place. We seek to quantify our aesthetics, to ground
them in universal truth: “For my expression to be be valid, all alternatives
must be invalid.” A zero-sum game.</p>
We get in our own way. We get in each other’s way.</p>

A Successful Hire</h2>
It’s no wonder that the state of technical interviews is so abysmal.</p>
Technical interviews are a coarse values compatibility test between the
interviewee and representatives from the hiring team. Ideally, this catches
things like “does your preferred communication style match that of the team?”,
“how best do you learn and is the team suited to support that mode?”, among
similar questions.</p>
However.</p>
Because of the deep-seated insecurity of engineering at large, interviews are
often a form of earning cachet</strong> with the existing team. In the worst case,
this looks like answering “gotcha questions”, whiteboarding, and
regurgitating trivia. Hazing, in other words.</p>
Why do I say “worst case” when it’s so widespread? Consider: it is common
practice for most engineering managers to have some amount of programming
background; thus those most frequently responsible for screening candidates
ahead of technical interviews can already tell</em> if the candidate knows how to
program or not</em>. Any discussion of “can the candidate program up to the quality
level of the organization” can be linked directly or thematically back to “‘A’
players only want to work with ‘A’ players”</a>, which is generally
misunderstood to mean</em> “this candidate shares my aesthetics.” A gentle thump on
the offending interviewer’s head may be dispensed by the hiring manager.</p>
New hires bring in their own opinions, values, & aesthetics.</p>
A successful hire always</strong> changes the values of the team that they join; and
in turn the team always</strong> changes the values of the person that they hire.</p>
The weight of the change — whether it tilts in favor of the team or the
hire — depends on the experience of the hire and the role they are being
hired into, but it NEVER goes to zero on either side. (If either side remains
unchanged, it wasn’t a successful hire: they were most likely shuffled onto
another team or fired.)</p>
Now, keep in mind: capital wants</em> fungible engineers. They’re focused
solely on the utility we produce. From that frame, the conflict &
compromise of a team’s aesthetic values is a bug, not a feature. But they
couldn’t be more wrong: the primary value in hiring a new engineer is not
simply “increasing parallelism of work”, but “expanding the artistic scope of
the team.”</p>
Invention happens because of unexpected connections; art is improved by
resolving constructive conflict and integrating feedback. But if you leave that
conflict unresolved — if our field is so stunted as to believe the conflict
shouldn’t exist</em> — you merely end up with a team in discord.</p>

Concede the Point</h2>
When aesthetic conflict can only be resolved by quantifiable arguments grounded
in utility or universal fact, the result is tenuous rationalization of
aesthetics. In other words, a team in this state lacks the means</em> to resolve
the conflict constructively.</p>
This gives rise to bad behaviors:</p>

Argument to exhaustion: whoever argues the longest for their point wins.</li>
Oil and water: different parts of the system are built based on increasingly
different aesthetic principles. The engineers involved “agree to disagree”,
so the conflict gets pushed into the interface between their two systems.</li>
Disingenuous arguments (or sandbagging): coming up with a solution and
constructing arguments to support it instead of investigating a problem
space, eroding trust.</li>
Appeal to the wisdom of the crowd: “everyone is using
$FRAMEWORK_OR_PROCESS</code>, so it must be the right answer for us as well.” This
ignores the context of the team, who may not be in agreement with the
practice.</li>
</ul>
Teams in discord are not stable. Unresolved conflict will show up in the system
they produce. Ignoring the aesthetic of the team threatens the utility they
produce.</em></p>
The core of the problem is that, absent a means to talk about their personal
desires & values for the solution, engineers “argue to win”. The starting point
of the argument is disingenuous, & they do not feel they can admit to their
aesthetics because it would concede the point.</p>
By way of comparison, a healthy team respects the tastes of its members, which
lets them talk about them openly when faced with a problem. Conflicts can be
resolved constructively by surfacing the values behind them.</p>

Borrow These Values</h2>
Art is, as previously established, always in reaction to itself. It is subject
to fads: a technique, thought process, or organizational principle sparks the
imagination of a large number of artists at once. The fad might propose a novel
solution to long-standing problems, reframe the art in a fresh way, or simply
resurface an old technique for a new generation. Fads create a vacuum, a new
space to explore in the art.</p>
Invariably fads die out, either being subsumed into the practice completely or
being replaced by new trends. They become contextualized, contained,
understood. Penning them in extinguishes the initial enthusiasm that propelled
them to popularity. You don’t see a lot of art that centers pointillism these
days. The fad has been explored and it no longer captures the imagination.
There’s just no point in focusing on it anymore.</p>
Much like visual media, software engineering experiences fads of execution and
fads of process. Fads of execution center a technique, a tool: for the web, the
2010’s were about JavaScript frameworks: React, Angular, Ember. Fads of
process, by contrast, center an organizing principle: “we should act as Site
Reliability Engineers!”</p>
Fads are a good thing in moderation. They expand the borders of the art. Long
after their demise, the lessons learned still resonate and inform future
artists. Fads, however, can produce zealotry: it’s tempting to believe that
“we’ve finally figured out the art! It is the end of history!” The fad can live
forever, becoming synonymous with the art itself.</p>
Software engineering is fertile ground for such zeal. Fads in this field live a
charmed life: we call them “best practices.” They’re dumped on teams, with the
expectation that the team’s output will improve. Agile, scrum, devops, SRE,
React, Node, matrix orgs, the Spotify model, Rails, Web 2.0! All fads.</p>
When the team’s productivity is down and your back is to the wall, there’s a
pressure to find a magic bullet. A process or tool that worked for some other
team. Not your team – a famous team, a successful team. After all, engineers
ought to be fungible, right?</p>
But all of these tools and processes are shaped by the aesthetic values of the
teams that originated them. You can borrow these values, but they’ll always fit
a little like hand-me-downs: a little uncomfortably. Your team is not Google.
You may not be able to afford Google’s values: Google has a money printer,
after all.</p>
Your team needs to find its own voice. “Best practices” are the start of that
work, not the end.</p>

Finding One’s Voice</h2>
First, release the idea that it is possible to be right about software
engineering.</strong></p>
Second, I am going to quote John Gardner’s “The Art of Fiction”, which is as
applicable to fabulists as it is to software engineers:</p>

[…] in order to achieve mastery [they] must read widely and deeply</strong> and must
write not just carefully but continually, thoughtfully assessing and
reassessing what [they] write, because practice, for the writer as for the
concert pianist, is the heart of the matter</strong>.</p>
</blockquote>
Practice, practice, practice. There are no silver bullets. Read lots of code,
write lots of code, try out different approaches, see what speaks to you.
Discard the rest.</p>
Finally: I am going to give you the scariest bit of advice. In order to work
well with others, you must be vulnerable</em>. Admit that your tastes are your
tastes</em>. Concede the ground of universal truth; it was never yours to begin
with.</p>
Read old code together; react honestly when you find code that violates your
values, but know that what you read isn’t wrong</em>, it merely offends you. There
may be entirely valid reasons for it to be written the way it is! Build trust
with your colleagues by demonstrating that you are willing to give authors the
benefit of the doubt. Your colleagues can connect the dots: how you talk about
code when the author’s not around extends to how you talk about their</em> code
when they’re</em> not around.</p>
Let your teammates see that you value their</em> aesthetics. Invite them to
inspect and adjust your values when you start a project. If you accommodate
them, they are more likely to accommodate you. Expose your values and thinking
whenever you go to solve a problem. Accept feedback graciously, and make sure
you give credit when acting on it.</p>
This profession doesn’t need to be a zero-sum game. Treat it like an art,
respect your colleagues as fellow artists, and practice it together.</p>

Thanks!</h3>
Thanks @ceejbot</a> and
@jefflembeck</a> for reviewing drafts of this
post.</p>


SSH to Cloud Managed Servers
Unknown — Sat, 22 Jan 2022 00:00:00 +0000
Using ProxyCommand to SSH to Cloud Managed Servers</h1>
Problem Statement</h2>
Given:</p>

You have a series of server instances: something like an AWS Autoscaling Group (or “ASG”).</li>
These groups are named in a regular pattern: environment-service</code>.</li>
You don’t necessarily have a stable set of IP addresses to SSH into.</li>
Your instances are attached to a private network.</li>
You have a well-known bastion server with SSH access.</li>
You don’t want to use AWS Secure Session Manager, or their equivalent (for whatever reason!)</li>
You have jq</code> installed.</li>
</ul>
You wish to be able to SSH to an ASG-managed instance without knowing the IP address ahead of time.</p>
Solution</h2>
These steps enable you to run ssh myuser@myhosts.environment.service</code>.</p>

Use .ssh/config</code> host aliases to capture an autoscaling group name.</li>
Point that host alias at a ProxyCommand</code>.</li>
The ProxyCommand</code> script receives two arguments: the user-specified host and the port (See Fig. 1)</li>
The script removes the host alias prefix (myhosts.</code>), extracts the environment, service name, and
(optional) instance index into variables (See Fig. 2, section 1)</li>
The script uses a command line interface with your cloud provider to list instances for that
autoscaling group and select an instance by the requested index (See Fig. 2, section 2.)</li>
Then the script retrieves private IP information for that instance. (See Fig. 2, section 3.)</li>
Finally, we connect to the target instance through a jumpbox.</li>
</ul>
Figures</h2>
Figure 1:</strong> An SSH config with ProxyCommand</code> inserted:</p>
Host myhosts.*</span></span>
    ProxyCommand /path/to/proxy-command-script %h %p</span></span>
    ForwardAgent yes</span></span></code></pre>
Figure 2:</strong> An example proxy command:</p>
#!/bin/bash</span></span>
</span>
# if necessary!</span></span>
source path/to/some/credentials</span></span>
</span>
# Section 1</span></span>
target</span>=</span>${1</span>/</span>myhosts</span>\.</span>/</span>}</span></span>
environment</span>=</span>$(</span>echo</span> $target</span> |</span> cut</span> -d</span>'.'</span> -f1</span>)</span></span>
service</span>=</span>$(</span>echo</span> $target</span> |</span> cut</span> -d</span>'.'</span> -f2</span>)</span></span>
num</span>=</span>$(</span>echo</span> $target</span> |</span> cut</span> -d</span>'.'</span> -f3</span>)</span></span>
num</span>=</span>${num</span>:-</span>0}</span></span>
</span>
# Section 2</span></span>
autoScalingGroup</span>=</span>$(</span></span>
  aws</span> autoscaling describe-auto-scaling-groups --output json</span> |</span> \</span></span>
  jq</span> -r</span> '.AutoScalingGroups[] |</span></span>
  select( .AutoScalingGroupName == "'"</span>${environment}</span>-</span>${service}</span>"'" ) |</span></span>
  .Instances[].InstanceId'</span> |</span> \</span></span>
  sort</span></span>
)</span></span>
</span>
if</span> [</span> ${</span>#</span>autoScalingGroup}</span> -lt</span> 1</span> ];</span> then</span></span>
  >&2</span> echo</span> "Autoscaling group \"</span>$1</span>\" not found or empty"</span></span>
  exit</span> 1</span></span>
fi</span></span>
</span>
instance</span>=</span>$(</span>echo</span> ${autoScalingGroup}</span> |</span> head</span> -n</span>$((</span>num</span> +</span> 1</span>))</span> |</span> tail</span> -n1</span>)</span></span>
</span>
if</span> [</span> -z</span> "</span>$instance</span>"</span> ];</span> then</span></span>
  >&2</span> echo</span> "Could not find server by index \"</span>$num</span>\""</span></span>
  exit</span> 1</span></span>
fi</span></span>
</span>
# Section 3</span></span>
ip</span>=</span>$(</span></span>
  aws</span> ec2 describe-instances --instance-id</span> "</span>$instance</span>"</span> --output json</span> |</span> \</span></span>
  jq</span> -r</span> '.Reservations[].Instances[] | .PrivateIpAddress'</span></span>
)</span></span>
</span>
if</span> [</span> -z</span> "</span>$ip</span>"</span> ];</span> then</span></span>
  >&2</span> echo</span> "Could not find ip of instance \"</span>$instance</span>\""</span></span>
  exit</span> 1</span></span>
fi</span></span>
</span>
# Section 4</span></span>
ssh</span> -q ubuntu@jumpbox.</span>$environment</span>.example.com -W</span> $ip</span>:</span>$2</span></span></code></pre>


My Note-Taking Setup
Unknown — Tue, 27 Apr 2021 00:00:00 +0000
My Notetaking Setup</h1>
A quick update today because this tweet</a> reminded me that I wanted
to talk briefly about my notetaking setup. I recently streamlined
my notetaking tools after living with them for about two years(!)</p>
The problem statement I started from is:</p>

I primarily work in a terminal with a vim session inside,</li>
I have two laptops across which I wish to share notes,</li>
I experience process as friction (so Zettelkasten</a> and Bullet
Journaling</a> are right out),</li>
I find myself needing to refer back to notes via distant memory,</li>
I write notes more often than I read them,</li>
</ol>
Given this, I want to keep searchable notes about work, life, and research
with as little friction as possible.</strong></p>
The system I use is a slight refinement of a previously unregarded group of
behaviors! I used to take notes in an editor in any directory I happened to be
in. As a result, I had a pile of unordered, untitled notes files in my $HOME</code>
directory. Whenever I switched computers, I’d lose all of my notes. Even on a
single computer, finding</em> a note I had previously written was fairly
impossible.</p>
It wasn’t all bad, though: the act of writing notes was valuable in and of
itself in helping me retain thoughts, and – maybe more importantly – it was
the seed</em> of a habit which I could spin in a more useful direction. In a
sense, I had handled the first point of my problem statement already!</p>
Two years ago, I collected all of my notes, meticulously adding titles and
modified times from the files themselves, then moved them into a ~/notes</code>
directory. I symlinked this directory to DropBox on both of my laptops.
Success! I crossed off point 2</strong>. There was a little added process: note
filenames now started with a YYYYMMDD-</code> timestamp to give them an unambiguous
sort order.</p>
This, however, added process. The process was “look up the date, then run vim ~/notes/20080808-chomp-womp.md</code>” to take a note. Even that</em> amount of added
process itched, so a couple of weeks ago, I wrote a little script</a> to take
notes. (I wrote it in Bash, to no-one’s surprise</a>.) This crossed off
point 3</strong>. I integrated fzf</code></a> and ag</code></a> into the script so now I can run:</p>
$ note search 'some term'</span></span></code></pre>
To bring up a fuzzyfinder window with just the matching notes. If only one note
matches, it’ll open in $EDITOR</code> instead. Inside fuzzyfinder I can select
multiple notes to open in vim</code> tabs.</p>
I can also run note last</code> to open up the last note I took. Finally, running
note</code> with no arguments opens an $EDITOR</code> window and lets me enter freeform
text. It takes the first line, slugifies it, and places it in the ~/notes</code> dir
with an appropriate name. Combined, those features tackle points 4 and 5</strong>.</p>
I’ve been fairly happy with the results: I’ve taken more notes and been less
tempted to combine unrelated notes into a single document. The thing I’d want
you to take away from this is not that you should use my</em> note-taking system,
but that having any</em> note-taking system is useful. (Also, look for places in
your life where you’re experiencing friction and try to formulate a problem
statement! You’ll be happier with solutions that are tied to a tightly-defined
problem!)</p>


the case for bash
Unknown — Fri, 02 Apr 2021 00:00:00 +0000
The Case For Bash</h1>
esac</code></small></p>
Friends, coders, I come not to praise bash, but to contextualize
it.</p>
Bash is oft-maligned: you hear it sneered at in sentences like “our continuous
integration pipeline is just a pile of bash scripts</a>.” It is commonly
regarded as unintelligible and thus unmaintainable. The pile of bash scripts
are the toxic byproduct of an infrastructure team understaffed, pressed for
time, unwilling or unable to do it “the right way” in a blessed programming
language, such as Ruby, Python, or, uh, … YAML?</p>

some problems with bash:</p>

It’s a weird & counterintuitive language</li>
you probably don’t need to write it that often, so you don’t practice</li>
the times you DO need to use it, it’s often because something important
(like a build) broke and it needs to be fixed RIGHT NOW</li>
</ol>
</blockquote>
– Julia Evans, @b0rk</a>, via tweet</a></p>
Julia has a a great zine</a> that will get you comfortable with Bash. I’m not
going to try to do that in this post! Instead, I’m going to try to convince you
why it’s worth</em> learning Bash. To do that, I’m going to talk about the problem
space that Bash thrives in, and why other languages are a bad fit for that same
space.</p>
So, about problem spaces and languages. You might be familiar with the idea
that any language with sufficient flow-control primitives</a> can express
the same algorithm that any other language of that class can express. In other
words, given access to the same system APIs, the same approach to solving a
problem could be expressed in Java, Ruby, or Bash. By comparison to art, if you
had to render an image of the inky blackness of outer space, you could
accomplish it with pen and ink or with a flat brush and paint. One approach
will save you a lot of time (and wrist problems!)</p>
This is to state by example: certain tools are better adapted to certain
problems. You, as a programmer, are likely to run into a wide range of problems
requiring a corsucating variety of problem-solving approaches in your career.
The first, best tool that we can bring to bear on the problems before us is
language</strong>, and thankfully we have a variety from which to choose.</p>
Languages lend themselves to solving a subset of the problems you’re likely to
encounter. When selecting a language to fill a particular niche in your
toolset, there are a number of considerations: performance, popularity,
ubiquity, familiarity, capability, amongst others. Languages will overlap
niches: C, C++, Java, Go, and Rust are good choices for problems where
performance is a primary concern. Python, JavaScript, and Ruby are excellent
multitools where performance isn’t a problem. All are popular languages with
active ecosystems. There is no universal correct answer, even amongst languages
with overlapping capabilities, because every language has different values that
suit different people.</p>

As a programmer, I think it behooves us to think about not just the values
that we hold, but the values of the people that use our software hold and, as
a programmer, you should use tools that align with your values.</p>
</blockquote>
– “How Rust Views Tradeoffs”</a>, Steve Klabnik, @steveklabnik</a></p>
I haven’t talked about Bash yet.</p>

So, what does Bash value? One might be tempted to say “ubiquity”. Bash is
available nearly everywhere these days; but ubiquity is a quality that is
conferred from outside the language. Preserving ubiquity can</em> be a value –
see JavaScript and backwards compatibility, or C and the PDP-11 – but it’s
difficult to positively attain</em> ubiquity.</p>
Instead, Bash values a very</em> particular fitness for purpose: the management
and coordination of programs. This includes spawning processes, connecting
processes via pipelines, and parallelizing processes. The purpose of Bash is
running other programs</em>. Consider the following:</p>
tail</span> -q /var/log/nginx/access.log</span> |</span> awk '{print $7}'</span> |</span> sort</span> |</span> uniq</span> -c</span> |</span> sort</span> -rn</span></span></code></pre>
“Follow the ngnix access log</a>; print the 7th word (the path); sort those
paths, count the number of unique instances, and then sort the results by
occurrence count.”</p>
const</span> { spawn }</span> =</span> require</span>(</span>'child_process'</span>)</span></span>
</span>
const</span> tail</span> =</span> spawn</span>(</span>'tail'</span>, [</span>'-q'</span>,</span> '/var/log/nginx/access.log'</span>])</span></span>
const</span> awk</span> =</span> spawn</span>(</span>'awk'</span>, [</span>'{print $7}'</span>], {stdio</span>:</span> [tail</span>.</span>stdout,</span> 'inherit'</span>,</span> 'inherit'</span>]})</span></span>
const</span> sort</span> =</span> spawn</span>(</span>'sort'</span>, [], {stdio</span>:</span> [awk</span>.</span>stdout,</span> 'inherit'</span>,</span> 'inherit'</span>])</span></span>
// ...</span></span></code></pre>
This is a contrived example, but there’s a lot more going on</em> in the Node
example than there is in the Bash one. The Python and Ruby versions would be
similar. The core data primitive of Bash is a running process</em>. It has
distinct syntax for dealing with the input and output of those processes, for
capturing that output as a variable, and dealing with process exit conditions.
The boilerplate you’d type in another programming language is the default</em> in
Bash.</p>
It values running programs over assigning variables, over branching, over
everything. If you don’t believe me, consider that the common mode for if</code>
statements in Bash is if [ <test> ]; then <stmt>; fi</code>. Now go run less /bin/[</code>. That’s right: [</code> is a program</strong> being run by the if</code> builtin.</p>
Bash’s choices don’t make a lot of sense coming from other languages: 0</code> is
“true” or “success”, strings inside variables often need to be re-quoted on
use, the loop and conditional logic is off-the-wall; but it makes sense in
context</em> of its values (and, well, history.)</p>

It’s true that a lot of what I’ve said about Bash is equally true of other
shell scripting languages: Fish, Zsh, Powershell, and Nushell all value similar
things. It’s worth learning these languages! However, if you’re dealing with
Linux servers on a daily basis, it’s definitely worth learning Bash. Even if
you’re not, it’s worth learning just to understand what these other shell
scripting languages are reacting to. Bash’s for</code> loops might be weird, but if
you can familiarize yourself with them, you can use that knowledge everywhere
from your personal laptop to a server running on AWS and back down to a shell
running on your local router.</p>
You (can) use Bash every time you open a terminal session. This knowledge
compounds over time: you don’t have to learn the entire language to be more
efficient. You can layer in new concepts every few weeks and before you know
it, it’ll be muscle memory. Any time you’re presented with the need to write a
one-off program to accomplish a small task, take the opporunity to puzzle out
how to write it as a shell script. The difficulty of learning Bash is less in
learning the language itself, and more in identifying what programs you should
call to accomplish your goal. This is like any other language: finding the
right API and the right way to call it forms the bulk of day-to-day programming
effort. So, give it a whirl! Remember that there are alternatives</a> to tricky
APIs! If you do, you’ll have a valuable tool that fills a niche most other
popular programming languages struggle to address.</p>


context
Unknown — Sat, 27 Mar 2021 00:00:00 +0000
I’ve heard it said that coding, as a skill, rewards those who have a high
tolerance for remembering lots of trivia.</p>
Trivia, here, means “knowledge divorced from context”. That is, the sort of
thing you’d hear in a bar trivia game, such as “what year was Waterloo?” or
“what are the names of the active volcanoes around Portland?” The observation
is funny in a useful way: it subverts the image of programmers as magicians by
reframing it as a sort of stubborn tolerance for an unfortunate situation.
However, the important thing isn’t really the volume</em> of trivia one can
memorize, but the desire to continually contextualize</em> trivia both for oneself
and for others.</p>
Contextualizing knowledge is tricky. We don’t help each other do it as much as
we should.</p>
A professional StarCraft player, Sean Plott said the following</a>:</p>

I think one of the biggest things that irks me is when someone who’s trying
to teach me conflates the learning process with knowledge. Which is, “Sean,
in this situation, you need to build X</em> because that counters their build.”</p>
That is someone sharing a tidbit of knowledge, not considering the sequence of
steps of learning that are necessary to even use that knowledge</em>, or what that
knowledge is sitting on top of.“</p>
</blockquote>
Much of the craft of software engineering (AKA programming alongside other
programmers</a>) is about communicating knowledge in
context</em> so that others may build a mental model of the program being developed.
The best programmers I’ve worked with recognize that programming is theory
building</strong></a>, and so give great weight to leaving artifacts behind
that communicate meaningful context for their decisions. The aforelinked
author, Naur, makes a useful recommendation about what to communicate:</p>

That which helps the next programmer build an adequate theory of the
program</em>.</p>
</blockquote>
Even so, it’s no small feat to identify all of the necessary context that
future readers might require, let alone to marshal it into natural language in
an easily understood fashion. As a profession, we’re terrifically bad at
sharing context. I care a lot about sharing adequate context! I know I’m
still</em> bad at it, though. What can we do?</p>

Identify your audience. Are there gaps in your skills? A delta in your
experience levels? What might they not know that you are taking for granted? Be
gracious: you might misestimate them: apologize up front for retreading ground.
This is incredibly important when talking to folks early in their careers.</p>
A brief aside: back in 2007, my first paying programming gig was maintaining a
college radio station website & DJ song log by myself. I was self-taught: a
feral 1</a></sup> programmer. I knew how to program</em>, sure, but I didn’t know about
things you’d absolutely take for granted, like version control. As part of a
archival project, another programmer</a> came in to help digitize the station’s
vast CD and record collection 2</a></sup>. They had many suggestions that were, in
retrospect, incredibly insightful. At the time, though, I didn’t have the
necessary context to make that knowledge actionable. I pushed back on a lot of
it because I simply couldn’t understand it. I was frustrated</em> and in turn I
was frustrating</em>3</a></sup>.</p>
I met up with that programmer a few years later, after I had some professional
programming experience under my belt. To my great surprise, so much of what
they said made sense to me</em>. They had been clear this entire time: it wasn’t
them! It was me!</em> Look out for these sorts of gaps. It’s not always so
pronounced: it might be that you’re talking to someone whose expertise is
simply in a different area.</p>

To some degree part of the problem particular to programming is that the solo
act involves considering many different courses of action and weighing their
outcomes.</p>
That is, you spend time thinking about the problem before typing your
solution. One of the developmental stages commonly seen in programmers is to go
from working the solution out in the editor to thinking the problem through
before committing to typing it out. The more you practice simulating the
effects of edits ahead of making them, the more automatic it becomes! The more
automatic it becomes, the more efficient you become as a programmer. It’s a
virtuous cycle right up until another programmer joins you on the program.</p>
In my experience, pair programming helps a ton here. Pairing forces both
programmers to verbalize the unconscious act of simulation [^4]. This can feel
like a terrible slog at first, but it’s useful on a number of levels:</p>

By talking through one’s thought process and deliberating on approaches in
tandem, the mental model is shared, reducing the lottery factor</a> of the
program.</li>
It’s a great way to share cognitive strategies & problem solving approaches
which are otherwise hard-won.</li>
Verbalizing the process requires surfacing the unconscious process to conscious
examination. Flaws in thinking can be identified and fixed.</li>
It builds rapport, which is key to maintaining a healthy social structure
around the program. Since the program will inevitably reflect</a> the structure
that built it this is is essential work.</li>
</ul>
Just remember: you and your pair now share context that the rest of the
organization may not have. Luckily, you’ve verbalized that context! It’s still
incumbent on you to record it for others. Even if you don’t combat this by
pairing, it’s helpful to consider that the act of conveying context is in
tension with the act of coding.</p>

There are no silver bullets here: communication is difficult. Practice
contextualizing what you’ve learned. Teach others. Find communication you
admire and emulate it. But most of all, don’t give up: continually seeking to
improve your communication skills is an excellent way to distinguish yourself
as a talented engineer. Giving up early – or worse, imagining that you have no
room to improve – is a surefire way to frustrate your peers.</p>
^1</sup>A phrase I borrow from ceejbot</a> who in turn received it from goofbone</a>.</p>
</div>
^2</sup>Using python scripts to automate an 80-spindle CD player & associate the resulting FLAC and mp3 files with metadata from musicbrainz</a>. Still a very cool project.</p>
</div>
^{3</sup>
Probably.</p>
</div>}


Deploying at Eaze
Unknown — Sat, 19 Dec 2020 00:00:00 +0000
Deploying at Eaze</h1>
Late January 2020 was a bummer</a>. In the wake of some pretty</a> dire cuts</a>,
ceejbot</a> and I found ourselves the sole custodians of Eaze’s
infrastructure. I remember looking at the scope
of problem ahead of us and asking myself “what am I giving up doing now</a>
because I am taking on this work?”</p>
Our first task: deploy a newly written identity verification service
for trust & safety. One small step.</p>
Flash forward to the end of 2020. We built a new infrastructure team. Deploys
went from low-confidence, hourlong ordeals involving careful coordination of
shared resources to blink-and-you’ll-miss-it</em>, safe operations against
on-demand</em> development clusters. As a result of our work, our cloud provider
bill dropped by 66%. The work to get the system to move again was deliberate
and intense, but like the compounded effects of an ion engine’s paper-weight
pressure against a starship, we’ve achieved some incredible velocity over time.</p>
Oh, and we put Rust in production!</em></p>
So, what did we do? What did we learn?</p>

starting at zero</h2>
For a long time, bright lines of responsibility existed between teams, ones
that could not be crossed: a backend engineer could not expect much access to
the AWS console, say. Likewise, it was frowned upon for frontend engineers to
cross into the backend. Predictably</a>, this resulted in a system that
mistrusted itself. The code developed a Velcro-like consistency at the seams
where teams met, born out of defensiveness and mutual mistrust. These codebases
represented silos of knowledge. When you can’t cross team boundaries, you can’t
socialize knowledge about your corner of the system! Over time, maintainers of
the system went on to new opportunities, unintentionally leaving huge knowledge
gaps in their wake.</p>
The technical debt was holding us back and the company realized this. Thus,
over the fall and summer of 2019 I was tasked with working across teams to
build a next-generation infrastructure for Eaze. However, the company had
not yet come to grips with the organizational debt it accrued, so this
project was doomed: because of the mistrust between teams, it was built in
a corner without interacting with the other systems. This insulated the
decision-making of the next generation project at the expense of making
sure those decisions were ultimately noncommittal; they could not affect the
working system or the existing team structure maintaining it.</p>
The next generation project was to be built using Kubernetes and Golang, the
former as a means to placate management and the latter as a means of resolving
a long-standing conflict between C# and Node developers in the company. Within
a week of landing at the company in the summer of 2019 I bounced off of Go.
This is not to say it is a bad language, but it and I do not share the same
values, and I was hesitant to hand it to the team.</p>
In particular, I was spoiled by my experience with the Rust compiler; I was
disappointed to find that Go’s compiler was not nearly as helpful or respectful
of my time. Thus began my campaign to get Rust into production at Eaze. I
nerdsniped</a> ceej into writing small Rust CLI tools as glue for the nextgen
project with me. She had prior experience with Rust’s 2015 edition; ergonomic
improvements in the 2018 edition made the sale easy. Ultimately, Rust was the
only lasting contribution of this failed nextgen project.</p>
The important point here is that we were working to tame a feral</strong> system, one
whose maintainers had long since left, and about which documentation was sparse
or non-existent. There was a small mountain of existing infrastructure managed
by a variety of homegrown tools, all of the knowledge of which had just been
lost from the company. On hand, we had pieces of a next-gen,
Docker-and-Kubernetes approach, written to avoid ruffling feathers. We
looked at what we had and said: “no.”</p>
shoe-string service mesh</h2>
To be frank, Kubernetes and Docker were overkill for our problems and team
scale.</p>
We needed something fast, correct, and ready yesterday. ceej and I had
previous, shared context for such a system: we wanted something similar to the
deploy system at NPM. This was ceej’s second time implementing such a system,
as she was the primary author of NPM’s deploy system; I previously put
together something similar for my personal infrastructure having found great joy
using that system at NPM.</p>
Our goals and values were:</p>

Developers should feel safe to deploy at any time of day, any day of the
week, limited only by their sense of the impact of their changes & their
commitment to work/life balance.</li>
We should start with a simple working system for a single service, then
add features as needed to support subsequent services.</li>
What we build should be able to be operated by any engineer: we should
not hide important infrastructure tooling from them. (We want people to walk
away with experience with Terraform, not “eaze’s
bin/apply-changes-somehow.sh script.”)</li>
When in doubt about a decision, consider that we want our work to generalize
to multiple clusters.</li>
We intend to use this new infrastructure to tame our existing feral services,
to make them malleable so we can take them back under control & begin making
necessary changes to our system.</li>
</ol>
The gist of where we landed is: services run directly on EC2 instances.
Services run 4 processes, load-balanced by a single colocated reverse proxy.
When a deploy starts, we take each service process out of rotation, update the
code & configuration, roll the process, wait for success from a ping endpoint,
then add it back into rotation. Rinse and repeat until all service
processes are back up. Report progress into a special slack channel.</p>
These EC2 instances are managed by an autoscaling group, and (at least initially)
each service had an application load balancer proxying to the nginx on each box.
(We later moved to single ALBs with multiple listener rules, thanks to maciej</a>.)</p>
Configuration and deploy event information comes from a watchable key-value
store. Service configuration combines keys of the form
<servicename>/ENV_VAR_NAME</code> with a global/ENV_VAR_NAME</code> namespace to produce
a single environment file suitable for feeding to systemd. Since the key-value
store is watchable, there’s a special deploys/<servicename></code> key containing a
JSON blob with the S3 URL of a tarball containing the service executable, &
other deploy metadata. Whenever this changes, a deploy is triggered.
Configuration changes, on the other hand, aren’t watched & can be triggered
out-of-band.</p>
This service deploy key is managed by a lambda listening for S3 bucket event
changes. Whenever an object lands in the bucket, the S3 metadata from that
object is turned into JSON along with the object’s location in the bucket. That
JSON is written to the appropriate consul deploy key.</p>
The S3 object is provided by a GitHub action, which builds & zips up the
service on a push to a deploy/$cluster</code> branch and drops the result into S3.
(We eventually wrote a GitHub workflow to synchronize service GitHub deploy
workflows, but that’s a story for another time.)</p>
We tracked Honeycomb trace IDs in our build metadata, so we could show the
fruits of our effort in concrete manner.</p>

  
</a>
More important than the particulars of what</em> we built are how</em> we built it
and what we learned along the way.</p>
thesis, antithesis, and synthesis</h2>
With the exception of a few months in 2018 and 2019, Ceej and I have worked
together for the last five years. We have a lot of shared context and values –
whether that’s as a result of working together so long or why we’ve worked well
together, it’s unclear; probably a little of column A and a little of column B.</p>
We approached rewriting Eaze’s infrastructure stack with a lot of
opinions about how</em> we should work together. Most usefully: having gleefully
worked each other into burnout in the past, we understood which behavioral patterns
were most likely to burn us out.</p>
As a result, the rule was that we only worked on the infrastructure together in
pairing sessions, and never let our enthusiasm for the work spill out into
evenings or weekends. Any time there was some bit of code we felt particularly
passionate about writing, we’d confer and make sure we both got a chance to
work on it during work hours. In order to get some quiet programming time
apart, we would take the time to define some self-contained tasks: writing a
CLI tool with well-known inputs and outputs to be plugged in to a bash script
written separately, for example.</p>
We pair programmed on all of this work over Zoom. The day to day work took on a
morning/afternoon cadence – a few hours before lunch, followed by a longer
pairing session in the afternoon. At the end of the work day, one or both of us
would summarize the work in a public Slack channel, both to communicate
progress out as well as to help us get started the next day. We both monitored
each other’s stress levels and would regularly make sure to trade off driving
duties at least every other day. We’d switch off more often for particularly
tricky tasks. (In mid-spring I took to working outside to get some sunshine.
Little things helped a lot with my morale, since this was around the outset
of the pandemic)</p>
More important than the act of driving or navigating was the decision-making
cadence. When we needed to make a tactical decision with strategic
ramifications, we’d compare the outcomes against our values going in. Often
multiple solutions would present themselves. We’d then converse about the
solutions, often adopting a thesis/antithesis pattern: “If we do X, Y gets
easier”, “ah, but if Z is important, we should reconsider because X makes it
more difficult.” If we got too far into a rabbit hole, we’d back out and
re-examine the larger problem.</p>
We continue these practices even today!</p>
This might not seem especially relevant in a technical context, but in my
opinion, this is the meat and potatoes of programming in a team context. There
always must be a balance of interests, a drive to share knowledge, and respect
for boundaries. Successful engineering has lot more to do with working well in
a team than it does with being technically “correct” individually.</p>
the most valuable single line fix</h2>
We visited each service in our system working in this cadence. By our third or
fourth node service, the pattern for service resources was clear, and we
started to modularize our Terraform code. We had a fairly workable pattern
established: packer</a>, powered by ansible</a>, to provide base Amazon Machine
Images</a> for use in autoscaling groups provisioned by Terraform</a>. Terraform
additionally handled the brackish residue</a> of per-service instance state via
cloudinit</a> templates – configuration service location, service name, etc.</p>
The system was coming together. We had one last hurdle: Eaze’s largest
codebase, a monolithic C# application written against Microsoft IIS on a fairly
ancient version of .NET. All, of course, running on Windows. This still ran on
a dangerous, error-prone deploy system written on top of Chef and AWS
CodeDeploy.</p>
There were the expected, mundane difficulties: the friction of having to switch
operating systems, learn PowerShell, & figure out the difference between
chocolatey and nuget. The sting of finding out that only a particular
combination of python version, ansible version, and ansible plugin version
worked to connect ansible runs. Trying in vain to remember that the directory
separator is \\</code>. But we prevailed, and – we thought – we had a workable
solution we could deploy darkly, taking no production traffic.</p>
We were wrong.</p>
When we spun up our newly configured windows boxes in production, the
performance of the live application absolutely tanked. We quickly
decommissioned our boxes and went to investigate.</p>
It turned out that we had a frankly ridiculous number of Redis connections.
Looking through the infrastructure repositories for context around this we
found that – shortly after April 20th, 2017, an infra engineer scaled the
application to 36 processes per box in reaction to unexpected failures.
Similarly, the redis box had been scaled up massively. While our load was not
insubstantial, ceej and I had some context here too – we had both seen much
smaller Redis instances meet the needs of much larger traffic loads at NPM.
This smelled like a misconfiguration issue.</p>
We looked over the application setup and eventually saw the culprit: an
inversion-of-control container was responsible for registering a Redis
provider. Whenever this Redis provider was instantiated, it would attempt to
create a pool of 20 active connections to Redis. However, the provider was
configured to be instantiated on every request to any endpoint</strong>. Our
production windows box starved the live application of connections via the
healthcheck endpoint.</p>
After consulting with our comrades, we changed the configuration and
redeployed. Success!</p>
We were immediately able to downscale the managed redis instance we were
using. Having cleared that hurdle, we were able to proceed with replacing
the old deploy system in its entirety, including the Windows boxes.</p>
It’s important to call out that this is, again, the bread and butter of
successful software engineering in legacy systems: it is anthropological study
of the situation that caused the system to arise, informed by second-hand
artifacts from the people involved in creating it. Lines of code are not a
valuable metric here; the value was in understanding the circumstances in which
the mistake was made, and determining that it was safe to fix the mistake. In
this case, the root problem was that there was such a divide between the prior
infrastructure team and backend teams – neither could find the problem by
themselves.</p>
growing the team and moving on to new problems</h2>
Throughout this process, we had also been pairing with Joe</a> and Chris</a>. They
had expressed interest in getting into infrastructure early in the year, though
they had historically only worked on Eaze’s website prior to that. They’re both
deep experts in that, but once trusted to work on the infrastructure they did
some amazing work in further reducing our AWS spend, moving services to the new
deploy flow, and grappling with our CDN configuration.</p>
This informed the strategy for growing functioning teams: start by pairing on a
hard problem. Gradually, as the system begins to take working form, fold other
engineers in. The goal is to end up with agreement on values & team direction.
(This cuts both ways: not only are you onboarding the new teammate to your
values, but you’ve also got to be willing to take on some of their values!)
Gall’s law</a> applies to human systems.</p>
Ceej and I stopped focusing on infrastructure around mid-to-late May. As our
final task, we added affordances for provisioning and decommissioning clusters.
We onboarded a familiar former coworker – Maciej</a> – before moving on to our
next task.</p>
There are still features I want to see in our infrastructure stack (automatic
nightly recreation of clusters!), but since Joe, Chris, and Maciej have
taken over the work, it’s only gotten better in their hands. The deploy system
has been a huge success: it started the long thaw of the technical debt
freezing our system in place. Others</a> have done incredible work accelerating
that thaw.</p>
2020 has been a tough year, but this little corner of my life was hugely
fulfilling. I’m proud of the team we built and the work we did.</p>


What happened to July?
Unknown — Fri, 07 Aug 2020 00:00:00 +0000
July is over. We’re already seven days into August. Time sure does fly when
you’re quarantining, I guess. So what happened?</p>
Well, between these two points, the feds were in Portland, OR., where I live.
Work on the Rust rewrite of our delivery system began in earnest. I started
helping with Orogene, another JavaScript package manager. I finished Zelda
(again.) I picked up Ghost of Tsushima, which is beautiful but empty by
comparison. Truly this generation of game consoles has been the “open world
with a bow-wielding, horse-riding protagonist” era. A friend’s brother died.
I… barbequed a lot more, because it’s been too hot to cook indoors. Another
person I know (distantly) died. Krysten and I painted a TV console (our first
time painting furniture!) We also finally broke into Overcooked. It’s been
a strange couple of weeks; time feels especially soupy these days.</p>
I spent a lot of the last month struggling with the dissonance of things going
well at home vs. the wanton brutalizing of Portland protestors. Struggling to
maintain my focus and not let others set my pace for me. Struggling to balance
all the things I want to do in my free time with my need to rest, to do
nothing</em>.</p>
For a few weeks I tried working on my WASM parser on my off hours from work;
then tried (& failed) to resist the urge to contribute to Orogene. I noticed
my mood at work dip more and more: the longer I moonlit on these side-projects,
the more irritable & less effective I was during the day.</p>
So, I took the last week “off.” All I’ve done in the evening is cook & play
video games. I’ve been getting to bed at a reasonable time, & reading comics
before that. Work this week has been fun, mostly, as a result.</p>
The phrase – which sounds better in my head – that comes up when I’m trying
(and failing) to resist picking up side projects is “a torque on one’s soul.”
I’ll experience this as some new disquiet that’s compelling me to move forward,
towards a goal I didn’t have before. This is occasionally useful: I’ve
learned a lot by following this urge. The problem is that I have trouble
ignoring</em> it.</p>


PLAN update
Unknown — Wed, 15 Jul 2020 00:00:00 +0000
Vacation is over! I’ve been back for about three days now.</p>
I ended up writing a WASM text format</a> (or “WAT”) parser in Rust over
vacation, in between watching movies projected on the side of my garage,
replaying Zelda: BOTW, and barbequeing</a>. My intention was to build a tiny
language runtime targeting WASM, as frequent readers will recall</a>. I didn’t
quite get the parser done in that time, but I got pretty close: I finished up
fixing the last known bugs tonight.</p>
Writing a parser against a language spec is a nice, meditative experience. It
involves repetitive work interleaved with research, leaving your mind free to
wander inbetween reading Backus-Naur Form notation</a>. You get the discipline of
balancing letting your focus drift to the implications of your present technical
decisions in the near future while reining it in to take the next smallest
step in the right direction. Maybe I let my mind wander too far while writing
it – after all, I didn’t get done until today, ha.</p>
All in all, it’s really not unlike the experience of drawing: a nice mix of
flow state, low-commitment strategic planning, and constant tactical feedback.
(I’m starting to feel the itch to illustrate again, to be honest.)</p>
I’m working in a private repo right now; once I clean up the code a bit I’ll
open source it on GitHub. I think the next steps are:</p>

Move the parser combinators and their tests into separate files.</li>
Pull the AST node types up into a separate package workspace so they can be
used without having to pull in the parser. (Useful if you want to compile to
WASM but parse a different language.)</li>
Add a canonicalization method to the nodes, to handle the “abbreviations”
present in WAT. (A sidenote: WAT had way</em> more abbreviations than I expected.)</li>
Emit WASM from canonicalized nodes. This should be suitable for running in
Node’s WASI</a> module.</li>
Pull in rusty_v8</code></a> and uvwasi</code></a> to make a standalone runtime.</li>
</ul>
That said.</p>
I’m starting to get a bit of RSI in my right ring finger knuckle, so it’s
likely that I’m going to have to take some time to let that heal. Karma for
foolishly claiming that not mousing helps prevent RSI</a>, I suppose.</p>


Career Progression
Unknown — Sat, 04 Jul 2020 00:00:00 +0000
Career Progression</h1>
A few days ago, I was promoted to Principal Engineer at $dayjob</a>.</p>
While I feel grateful for the recognition, I have to admit I’m a bit of a
skeptic when it comes to career ladders and titles. People aren’t comparable,
nor are their talents or knowledge. Attempts to reduce a group of
humans to points on a number line erase a lot of individual strengths.
On the other hand, titles are a way for the group to officially
recognize the personal practice of improving at the craft of
engineering. So, I’ve been stewing on this for a couple days now: what
does this promotion mean?</p>
Speaking for the tech industry, titles are a measure for your experience,
efficacy, and ability to deliver value to your organization. They
represent how the organization at large views your talents and often
how they value your contributions. In theory, they reflect your day to
day responsibilities.</p>
However.</p>
The measure, as a proxy for the job, is doubly flawed. For one, it doesn’t
describe the job accurately across organizations</a>; further, two people with
the same title within the same organization may execute the title in wildly
different fashions. For two, we measure individuals, but the our ability to
contribute on an individual basis depends greatly upon the team around us, and
that team’s perception of us.</p>
On the other hand, the measure corrects for insidious biases: the title can
lift some of the burden of constantly having to prove your worth, especially if
you’re not a member of a historically privileged group. It can also correct for
compensation bias: two staff engineers should be making the same amount, even
if one of them is a white guy and the other isn’t. (Compensation can get
perilously out of whack without banding tied to titles.) The bias is still
there, it’s been compartmentalized into convincing the org that you meet a
rubric, but you can at least bring receipts to that conversation.</p>
So, I’m conflicted.</p>
I am an ardent believer in self-improvement – that the craft of engineering
can be practiced like art in any other medium. I also believe that you will,
over the course of your career, encounter engineers who have practiced far
more (or less) than you, possess raw talent for thinking (analytically or
otherwise), whose communication skills and techniques you will envy and
seek to emulate for years afterward. You’ll work with people all across the
spectrum of possibility. Trying to reduce them to a number line erases so
much of what they have to offer, encourages zero-sum mentality, and stands
in the way of your own self-improvement. But that’s the realm of the ideal:
only a privileged few are afforded the ability to treat programming as an
art that can be practiced in isolation from society.</p>
On the other hand.</p>
I’ve been at the pointy end of a shift in organization culture. Two years ago
at a previous job, my manager told me point-blank that I had been doing the
Principal Engineer job for some time per the career ladder rubric, but because
of organizational changes there was a new VP who could not be convinced that I
should be promoted.  Ironically, before the change in management, I was one of
the parties responsible for designing the rubric</a> for that career ladder! That
same VP had pulled me aside earlier to explain that, because I had worked so
closely with the prior, unceremoniously booted, management, it “wasn’t clear”
what I had contributed over the past four years. I stuck around for months from
that point, trying to do everything I could to change their mind.</p>
I still feel hurt by this, obviously. The insecurity I had internalized from
this experience factored into my decision to turn down a Principal Engineer
position during my subsequent job hunt. I’m here now though, so it’s all for
the best, I suppose. The lesson, however, is clear: a career ladder is a social
construct that is only as real as the people in power at your organization make
it.</p>
Titles hurt in other ways: I’ve seen a brilliant, absolutely stellar programmer
get turned down for a position at a company because their ability to impress
their interviewer with a whiteboard and dry-erase marker wasn’t sufficient to
justify the title they were interviewing for. Again, there’s a lesson here: a
focus on measuring humans against a number line tends to turn technical
interviews into a hazing ritual. The point of the ritual hazing is to buy the
necessary cachet to justify a title with the existing engineering staff. This
is stupid and cruel, as you know.</p>
So much of what exhausts me in tech is tied up in the constant drive to
establish a pecking order, and that might be what I’m really reacting against.
Career ladders are a pragmatic concession to this: if we’re going to keep
trying to rank engineers, we may as well do so with an established rubric. At
least then we can try to confront our biases. It’s worth considering the title
a floor when considering a person’s talents: they may be wildly more talented
than their title. There is no iron-clad rule that says an associate engineer
cannot teach a staff engineer something valuable.</p>
I suppose this is all to say: I’m grateful for the recognition. I take it with
a grain of salt, because titles and career ladders are a mushy, social
construct that changes as people enter and leave the organization; I won’t let
it go to my head because there’s obviously so much more I can learn from my
colleagues, no matter where they are on this ladder. My intention as a
principal engineer is to highlight the achievements of my coworkers, work to
make sure they’re empowered, trusted, and happy, and to become better engineers
by working together.</p>
I’ll write more about what’s worked for me in a subsequent post.</p>


PLAN update
Unknown — Sat, 27 Jun 2020 00:00:00 +0000
I’m coming up on a chunk of vacation, and I’d like to work on a
technical project during that time. I have a couple of plates already
spinning; Boltzmann documentation, for example. The itch(es) I’d like
to scratch are:</p>

Parsing in Rust, possibly using pest</code></a> or nom</code></a>.</li>
Learning more about WASM and WASI, possibly using them as a language compile
target.</li>
Revisiting control flow graph generation of JS in Rust, using ratel</a>,
RESSA</a>, or esprit</a>.</li>
Doing something graphical: some sort of map generation</a> would be neat.
There’s quite a learning curve for me here; I’d probably be picking up
Amethyst</a>, Rendy</a>, or wgpu</code></a>.</li>
</ul>
Each of these has a research component: what is the state of these ecosystems?
How do the various pieces fit together? What does it look like to work with these
in a borrow-checked language like Rust?</p>
So, breaking down the projects a bit:</p>
Write a tiny language</h2>
Admittedly, there’s not a huge technical need for the sort of langauge I’d
write, nor or any sort of gap in existing tooling I’m attempting to address!
I’m interested in “walking the path”: what does it look like to write a language
today? What role might WASM play in early prototyping? What gaps exist?</p>
And indeed, writing a scripting language is on my personal programming bucket
list. I’ve written tons of parsers in my day (and a single control-flow-graph
generator leveraging partial evaluation), but I haven’t had the chance yet
to write an executable language.</p>
I plan to start small: iterate on a LISP-like language at first. Targeting
WAT</a> (“Web Assembly Text format”) and emitting WASM may be a good start. That
would let me target Node’s WASI API</a> (“Web Assembly System Interface”</a>) to
get started, or perhaps wasmtime</a>. Eventually this could lead to writing a
self-hosted runtime using rusty_v8</code></a> and uvwasi</code></a>. (That would be a fun
learning experience, since it’d take me through using bindgen</code></a>.)</p>

Control flow graph generation</h2>
Six years ago (!!) I experimented</a> with building a control flow graph</a> generator for
JavaScript. My initial goals were to:</p>

Build tooling to search a corpus of JavaScript for library use, in support
of estimating the potential fallout of making changes to Node.JS builtin
APIs.</li>
Visualize code diffs from a novel angle: instead of looking at the textual
change, two graphs would be compared and the differences would be displayed
visually. (It would be easy to catch that new code introduces an exception edge
when looking at two directed graphs, less so in textual form.)</li>
</ul>
I had some success; over the course of the year I spent at Wal-Mart Labs I
wrote ESControl, ESToc, and ESSim: a control flow graph generator, dependency
tracker, and simulator. However, looking back, I wish I had brought a bit more
rigor to my work: early on in the project it struck me that due to the nature
of JavaScript, very nearly ever node on the control flow graph would have to
draw an exception edge. That many edges with a common destination would’ve
completely obscured the rest of the graph when displayed visually, and yet
the exception edges represented valuable control flow information.</p>
I cut that Gordian knot by backing into partial evaluation: as I walked the
graph I would keep a stack state machine that would allow me to make assertions
about certain exception edges as they appeared: as an example, if I had drawn
an exception edge for x is undefined</code> earlier in the graph, I could omit the
edge on the next (straightline) lookup of x</code>:</p>
'use strict'</span></span>
x</span> +=</span> 1</span> // draw an exception edge: what if "x" is undefined?</span></span>
x</span> +=</span> 2</span> // if we reached this point we definitely know that "x" is defined!</span></span></code></pre>
Approaching the problem this was expedient, but tipped the scale on other
load-bearing decisions: because it had the stack machine information available
to it, ESControl aggressively inlined functions. Eventually it reached a point
that for some small programs, I could generate ESControl IR and run the program
step-by-step with the same output.</p>
I’m interested in revisiting this in Rust for a couple of reasons:</p>

I never got to the point of visualizing code diffs as control flow graph
changes, and I’m still intrigued by that</li>
The memory usage characteristics of the JavaScript ESControl were not ideal;
Rust would force more rigor in memory use.</li>
I’m interesting in taking door number two, so to speak: separating the control
flow graph generation from a simplification step that would run the partial evaluation
necessary to erase edges from the graph.</li>
</ul>
This would require familiarizing myself with the state of JS parsing in Rust. I
would not be mad about that.</p>

Map Generation</h2>
This one is a bit out there. At my $dayjob</a> we’ve been tasked with rethinking
how delivery fulfillment works; I’ve become enthusiastic about the idea of
modeling this as actor-owned task queues. Put another way, the data the system
operates on would work much like how real time strategy games model commands
issued to units. From there we’d layer on systems responsible for generating
and assigning efficient task queues. (This might not seem like much of a brain-wave,
but to be honest the way the system works right now was clouding my framing
of the problem.)</p>
Of course, thinking about real-time strategy games got me thinking about game
programming again; at the same time, I’ve been missing some of my more visual</a>
experiments. It’d be fun to build a terrain or city generator, then model unit
task queues on top of that! Kat’s recent work</a> in this space is inspiring, also.
(The presentation</a> Kat linked to earwormed me.)</p>

Currently reading / open in tabs:</h1>
Yoshua Wuyts’ writeup on compilers</a></strong>: Yosh is super thoughtful and thorough; I’m
excited to read more of his writing. (Compliments aside, you can imagine how a
writeup on compilers might dovetail with my current interests!)</p>
“Why we love Rust”</a></strong>: a nice grab-bag of tips & tricks learned while delivering
an impressive low-latency video & audio streaming solution. I’m going to try integrating
some of the tools they suggest, like cargo-flamegraph</code>.</p>
Essentials of Metaheuristics</a></strong>: linked by a co-worker, Eli</a>. As you can
imagine, optimizing delivery involves a lot of</a> (NP-hard) problems</a>, for
which heuristic approaches are a good fit. My experience with this is a bit
distant, so I am refreshing my memory by re-reading the simulated annealing</a>
article on wikipedia. (This also dovetails with my desire to build a CFG diffing
tool, which abuts the NP-hard-in-the-general-case longest common subsequence</a> problem,
for which a dynamic programming</a> approach can solve specific cases in polynomial
time. This is all to say: it’s up my alley, and look, there’s a Rust library for it</a>!)
I’m also taking a peek at Optaplanner</a> on his recommendation.</p>
Grain Language</a></strong>: a new language! Linked by Blaine</a>. I like Blaine’s
taste; if he’s enthusiastic about a language it’s probably worth checking out.</p>
StaffEng.com</a></strong>: A great overview of what work in a staff engineering
position looks like. I feel some relief at the seeing the enumeration of the
types of roles staff engineers play at companies: I am somewhat allergic to
working at large companies with lots of process, so I’m not a great fit for
teams that use staff engineers to enforce process: whether that’s serving as
architecture review boards or tech leads responsible for assigning work.
However, I have it on other folks’ authority that I’m a good teacher, and I
know how to tactically deploy my technical skill in service of my larger
strategies. I’m in my feelings about this one still – that last year at NPM
really did a number on me, not least of all because the goalposts of what
staff engineering meant changed all at once. It’s probably worth a more
complete blog post in the future.</p>

Other thoughts which may yet become blog posts:</h1>
I’ve been ruminating a bit on my career progression lately. One of the
aspects of staff engineering I’m interested in is how we process negativity, or
grumpiness. It seems to me that as a profession we tend to reward negativity,
so I (& others) displayed more of it when mid-career. I’ve walked a lot of
that back over the last four years, but false positivity isn’t tenable in the
long run, either. I’ve been stewing on Steve Klabnik’s keen observation</a>:</p>

As a programmer, I think it behooves us to think about not just the values
that we hold, but the values of the people that use our software hold and, as
a programmer, you should use tools that align with your values. I really like
programming languages and learning new ones, but there are some that I have
seen where I’m like, “You know what? This language is not for me, so I’m just
not going to use it.” I’m not going to denigrate any languages by naming
them, but it’s true that I would be unhappy if I had to program in some
languages and that’s because they value different things than I value and
that’s totally chill.</p>
</blockquote>
Ceej relayed the following from her husband, David:</p>

Anger is a useful signal: it means your values are being violated in some way!</p>
</blockquote>
Speaking of grumpiness because of violated values, I’d like to see a positive
workflow for ES Modules framed, if possible. Right now the story is all veggies
and no dessert.</p>
Finally on this note: I’ve been thinking about React SSR quite a bit lately.
The thought bouncing around in my head is: React’s origin as a tool for managing
a live DOM makes it ill-suited for use as a server-side template language, which
is how most React SSR solutions position it. It’s another thing I find myself
grumping about, and it might be worth digging into in the future.</p>


Boltzmann & Nostalgia
Unknown — Thu, 18 Jun 2020 00:00:00 +0000
Right now I’m writing templating middleware for Boltzmann (a JS framework.)
Three years ago around this time, I was writing something similar for Spife
(another JS framework) while waiting in the Oakland airport for my flight back
to Portland. I was entirely drained from another NPM all-hands experience, and
I kept staring out the windows at planes landing and taxiing to gates. It was
in the evening; everything outside was tinted lavender with orange highlights.
I was excited to get back home, but so tired; sad to be leaving, hoping I had
done good by my coworkers while I was out.</p>
It’s last year, around this time. I’m writing the first version of Boltzmann
for Entropic while I’m in San Francisco for my two week (!!) onboarding at
[$dayjob]. At this point I’ve been camping out at the dark, slightly-dingy
hotel bar for about a week’s worth of evenings, idly poking at my laptop. By
the time I’m off work, I don’t really have the motivation to make it much
farther than “the hotel lobby.”</p>
The bartender is pouring excellent Manhattans, which is why I keep coming back,
to be honest. By the end of next week, we’ll be on good enough terms that
he’ll pour me the occasional free shot of whiskey. At this point in my
life, I am shellshocked by how quickly and completely my NPM has changed
character, and my ego is deeply bruised by the preceding months. It’ll be
almost half a year before I start to feel better. I keep typing. We’re
presenting Entropic in 3 weeks. I miss Krysten and my cats.</p>
It’s two years ago – 2018. I’m in Seattle to give a talk about Spife at a
local user conference in January or February. Our only engineering manager at
NPM quit. For a few months I’ve played the part of project manager, engineering
manager, and tech lead for the rewrite of the website. I am incredibly wound up
and anxious. I worry a lot about letting my coworkers down. Sometimes this
anxiety gets in the way of seeing how they’re rising to the occasion; how
they’re supporting me. I’m on a managers call; I’m gutted by how badly I’ve let
down one of my reports. This has been the winter of spam: attackers flooded
pretty much public content-hosting site with SEO spam, and NPM was not spared.
We didn’t have the tools or processes to deal with it, so we’re fighting a
rear-guard action on nights and weekends.</p>
A package that wasn’t spam got deleted accidentally the weekend before, which
caused a fire drill. Someone reporting to me had warned repeatedly – weeks
ahead of this – that this was coming. Fuck. The manager’s meeting wrapped up.
I walked to the venue and talked about Spife, and about how it’s building your
own framework to address your problems is, in fact, good. Six months later
someone else at the company came to power & Spife fell out of fashion. I was
sidelined. After my talk, I saw a show.</p>
It’s today. I’m writing boltzmann template middleware. We’re talking about
rewriting the website at [$dayjob]. I’m fretting about taking my car for some
maintenance tomorrow. Will people be wearing masks? Should I plan on staying there
while they work on the car, or should I rent a car and get back home? It’ll
be the longest I’ve been out of the house in public since quarantine started.</p>
I traveled a lot more than I thought I would over the last five years. The
moments that stand out the most to me are the quiet ones, near the end of the
trip, waiting at the airport to go back home. Feeling melancholic about
leaving, but exhausted and happy to head home. I don’t know if I miss traveling
for work, but I’m definitely nostalgic for the feeling of coming home.</p>

It’s ten years ago. I’m at Johnny’s Tavern</a> in North Lawrence, Kansas. I’ve
been working nights and weekends on a callback-based implementation of
Django’s</a> template language, to integrate into a JS framework called Wilson.
I’ve just finished writing the template loading integration. My friends –
largely more seasoned Python programmers – nod along; they seem enthused,
supportive, and gently skeptical. After the evening winds down and we
say our goodbyes, I walk the few blocks home down Massachusetts street;
the sky is slate blue and peach, the future is bright and full of
possibility. There might be a future for me in this Node thing.</p>


Watching: The Forgotten Art of Structured Programming
Unknown — Tue, 16 Jun 2020 00:00:00 +0000

    </iframe>
</div>
This talk is a gold mine:</p>

Impress your friends by casually using “afferent” and “efferent”
in conversation</li>
Don’t miss “Reach for the banana, get the gorilla”, which might
be my new favorite turn of phrase. (Immediately applicable, too:
at $dayjob</a> today, we’re porting a Wordpress server to the new
Terraform-based infrastructure setup. We only use Wordpress for
its WYSIWYG editor!)</li>
Discover the one weird reason bash uses fi</code> to close if</code> statements.</li>
</ul>
Some links I drew from the talk, which I’ll be reading (or re-reading) in
the coming week:</p>

Structured Programming (Dahl, Dijkstra, Hoare)</a></li>
The Paradigms of Programming (Floyd)</a></li>
Go To Statement Considered Harmful (Dijkstra)</a> - not new to me, but always
interesting to re-read. Niklaus Wirth is responsible for the click-bait-y
title, which I did not know until this talk.</li>
(In talking to my co-worker, ceej</a>, she mentioned a great pun from Niklaus
Wirth – “Europeans call me by name - ‘veert’ – Americans call me by value –
‘worth’”)</li>
Testing is the Rigor of Software Engineering (Ford)</a></li>
Unit Testing Culture (Bland)</a></li>
Software Requirements & Specifications (Jackson)</a> - linking to a quotes page,
but I’m interested in reading the full book.</li>
</ul>

My day job involves paying down a small mountain of tech debt: the previous
infrastructure team was laid off in January shortly before COVID was a going
concern. (Ha, at the time I was worried work</em> would stress me out!)</p>
So, since early February I’ve been pair programming with ceej on a daily
basis. She’s observed, accurately, that left unchecked we will gleefully burn
each other out by working too much – we both find the work fun, and ratchet each
other up into working more.</p>
Looking back at the work we’ve gotten done in the last 4 months, I’m both
impressed and a little scared. Time started turning into soup before COVID, if
I’m being honest. We started by ditching our previous attempt at a new deploy
system which was stymied by, err, organizational problems.</p>
We wrote a deployment system inspired by npm’s deploy system – a git flow that
acts on pushes to deploy/<cluster name></code>. Along the way I’ve had to learn a
lot</em> about ASP.NET and Windows server, Terraform, and GitHub actions. We’ve
been writing a lot of Bash, which is handily one of my favorite languages, and
a little bit of Rust for tooling. I’m happy with the result! Deploys used to
take thirty minutes, or up to an hour or more. Now they take minutes. Our
infrastructure costs a third of what it used to.</p>
Really, our success here is down to breaking down the wall between
infrastructure and developers and adopting a (now-classic) devops approach.
This let us identify application-change level wins by observing the behavior of
the system in the real world.</p>
To illustrate: A particularly pernicious one-line bug in a C# dependency
injector, coupled with a lack of communication between developers and
infrastructure, resulted in our system maintaining 40,000 redis connections at
a time, forcing us onto the most expensive elasticache instance. The wall that
existed between devs and infrastructure kept either side from identifying the
problem: we were creating a new Redis connection pool on every request to any
endpoint. The problem? The Redis pool manager was attached to the application
as a “scoped lifestyle” dependency – that is, instantiated at request time –
and it was indirectly depended upon by every endpoint. Changing the lifestyle
of the object fixed the bug and allowed us to drop to a reasonably sized
Elasticache instance. Sorry AWS.</p>
It’s probably the most value I’ve ever introduced at a company with a one-line
change. So that’s been good for my ego.</p>
This has been both the easy part and the hard part, though – hard, in that it
took major organizational changes to get here; easy, in that “running server
processes on an EC2 instance” is a fairly well-understood problem, even on
Windows. Now, we’re looking at rewriting a half-decade old (or older?) C#
monolith. We’re nearly out of the woods with the infrastructure problem –
we’ve dutifully turned our technical debt into documentation debt, as we
rewrite the system to match the needs of our current engineering team.</p>
I can’t shake the feeling that we’ve only just started hiking up the mountain,
though. This application is layered, but each layer is attached to the next
like velcro. Methods hand database IDs to each other in lieu of passing objects;
methods may expect to be invoked by multiple callers for different purposes at
different layers. I found an entire class dedicated to making integers positive.
A critical delivery modality is, essentially, type-punned on top of a completely
different delivery modality. (“What if you built a pizza delivery company by
repurposing fleet planning software built for ice cream delivery trucks?”)</p>
Everyone I know needs a vacation, but no one can leave their house.</p>


a statement of intent
Unknown — Mon, 15 Jun 2020 00:00:00 +0000
I’m building a new blog. It’s been 3 months since the start of quarantine.
Neighbors are in the streets protesting police brutality against Black people.
And I’m building a blog.</p>
(I would be remiss not to say this: Black lives matter</strong>. Defund the
police. Dismantle white supremacy.)</p>
Why am I doing this? Why now?</p>
Partly to self soothe, partly because I want to build a habit I can be
proud of a few years down the line. I want to practice the crafts of
journaling, technical writing, digital scrapbooking, and self-editing.
To some degree this is an exercise in nostalgia: when I started out as a
programmer, I really loved reading the PLAN files</a> of developers of
game devs like John Carmack</a> and the Dynamix team</a>. I miss the
internet of that era: carving out a corner of the web and, yanno, just
talking about your interests and being a person.</p>
Social media feels a bit like being I’m being rented a birdhouse for a bit of
my soul (HT TMBG</a>, sic). I don’t really own my twitter account, my Facebook
account, or my instagram account. I pay for it with my attention. I pay for it
by contorting myself to fit the platform. I feel constrained by the medium.</p>
I want to be careful</u></strong> about that point, because it treads dangerously
close to the gross, entitled white guy mode. I mean “constrained” in the sense
that I unconsciously consider the audience when I post on social media. It is a
performative feedback loop and the act of participating changes me. The tiny
endorphin hit of posting something popular on Twitter and seeing notifications
roll in shapes what I’m likely to post in the future. I become lopsided. I have
been lopsided.</p>
The other problem with this rented birdhouse is that by using it, I take up
space that others might use more effectively. I’m thinking of the #blackout
Tuesday</a> Instagram tag in particular. In the middle of the protests, in an
attempt to show solidarity with protesters, folks started posting black
rectangles. Contrary to their intent, this had the effect of pushing useful
information about the protests out of Instagram feeds.</p>
Posting about tech on Twitter right now feels like taking the bullhorn out of
the hands of people who need it to yell something inane about my hobbies. I
don’t want to do that, but I do want to write about my hobbies (Especially
right now as a desperately-needed form of self-soothing.) The internet
doesn’t have to be a zero-sum game this way, but that’s how social media works.</p>
So, I’m going to write about my hobbies here. In the interest of
becoming a less lopsided person, I’m going to write about all</strong> of my
hobbies here, so tech stuff will be mixed in with art stuff, and that’ll
get mixed in with links to songs I’m listening to or videos I’m watching
or articles I’m reading. If it goes well it’ll make someone happy, even
if it’s just me. Watch this space.</p>

How does it work?</h2>
Okay, you asked for it – let’s navel-gaze.</p>
This blog is HTML and assets served by Cloudflare from a public AWS S3
bucket. GitHub actions are responsible for downloading a zip file full
of MarkDown from Dropbox using their API, then rendering them with
Zola</a>.</p>
Elsewhere, I’ve registered a Dropbox app and provided a webhook that
calls an AWS Lambda function via their API Gateway system. The Lambda
dispatches GitHub repository dispatch</a> events whenever a file in my
Dropbox changes. I have a locally symlinked ~/blog</code> directory, in which
I write files using NeoVim.</p>
To the extent possible, all of the infrastructure involved is managed
through Terraform</a> via my personal infrastructure</a> project.</p>
Notably, my use of these services is not an endorsement. In particular,
GitHub is problematic for its support of ICE; AWS for its treatment of
workers. Tech is complicit in white supremacy. I use these services
because they are where my expertise lay and I want to start this
practice today. I will work to move off of problematic platforms where
I am able to, and will agitate for them to do better where I am not.
(It’s not a lot, admittedly, but.)</p>


x-post: adding promise support to Node.js
Unknown — Sat, 13 Feb 2016 00:00:00 +0000
This was originally posted on my Medium account</a>; I’ve reposted it here
for posterity.</em></p>
Adding Promise Support to Node.js Core</h1>
Two Sundays ago, I opened a pull request</a> to the nodejs/node</code>
repository</a> introducing support for Promises. There’s been a lot of
discussion since that point, with over 400 comments in the main thread alone.
Several discussions have split into other issues. This post will summarize the
state of the discussion as it stands.</p>
Why add Promises?</h2>
Given that many discussions around promises turn adversarial, and that
solutions currently exist in the ecosystem, why include them in core?</p>
While it’s true that the topic can be contentious, I believe that this is
largely due to the framing of the conversation. It is often set up as a
zero-sum game</a>: for promise users to make gains, callback users have to lose
ground, or vice versa. However, I believe that we can frame this in a more
productive fashion. If we accept that both callbacks and</em> promises are both
valid ways to build asynchronous programs</strong>, the discussion becomes about better
supporting a set of users, instead of about invalidating current practices.
Callbacks are, and will continue to be, a sturdy, reliable pattern on
top of which to build your application — as will promises. This discussion
is about making life easier for a set of our users without incurring a
cost for another set of users. It is not</strong> about which pattern is best.</p>
It’s natural to ask, “Why put them in Core, if they’re already useful as-is?”</p>
The current situation front-loads a lot of decisions onto authors of
promise-based packages. That is, if I and another package author wish to
interoperate using promises, that agreement alone is insufficient for us to
proceed.</p>
In order to interoperate, we have to agree on the shim, implementation, and
specific version of promises in order to allow promises to safely be used
between packages. Otherwise, a tricky (and manual) process of wrapping and
unwrapping promises at the edges of the package APIs is necessary. For most
users, both of these approaches are not ideal — they just want to use native
promises, not specify all of the details. Users that wish to use different
implementations still can, but the requirement of marshalling and unmarshalling
at package boundaries become “opt-in” for the majority of users.</p>
Compounding this situation, async/await</code> syntax uses promises as the
underlying mechanism for representing asynchrony. Users that wish to use this
feature, at present, will have to pick a shim, implementation, and version in
order to use this syntax with Node APIs. This introduces friction for newcomers
to Node, especially those coming from the frontend where their preferred
patterns “just work”.</p>
Exposing a Promise API aligns our interests with the interests of our
underlying VM. Promises are already part of the Node platform, by way of V8 and
the JavaScript specification. The language has woven them into multiple
syntaxes now — from the module specification to async/await</code>. Some number of
our users will expect Node to provide a JavaScript runtime comparable to
browsers environments. Node has a history of trying to align these platforms —
from small things, like making the timer API look-and-feel like the
setTimeout</code> offered by the browser, to bigger things, like providing users our
own TypedArrays in-tree before our vendored V8 offered them.</p>
Finally, this helps heal a rift in the community. By including a core promise
API alongside the callback API, at a platform level we reinforce the notion
that both patterns are valid. It does not matter to Node which pattern suits
you best. This makes participating in the Node ecosystem and</em> in Node core
more welcoming for promise users.</p>
The Approach</h2>
The callback API will not change to suit the promise API. It will remain the
same. The current approach installs a promisified version of each method
alongside the original method. We have the option</em> of exposing core’s
promisification method via util</code>, but currently it is internal</strong> to core.</p>
In presenting the promise API as a regular transformation on top of the
callback layer, we ensure that the maintanence cost of the promise layer is a
small constant added to the maintanence cost of the callback layer. We also
ensure that callback users are not paying a performance tax to support promise
users.</p>
This approach doesn’t preclude the later inclusion of a submodule or ES2015
module-based approach to surfacing a promise API, however, that is out of scope
for this PR. In that vein, returning a promise from callback APIs when no
callback is passed is not workable because some callback APIs already have
significant return values separate from their callback value. Additionally,
a few crypto APIs are synchronous when called without a callback.</p>
The API is experimental and if approved will land behind a flag,
--enable-promises</code>, which will be set to false</code> by default. Work may continue
on the API once it has landed. It would be at least</em> one major version until
the flag is lifted, and if the API proves untenable to support it may be
removed before that point. At least</em> two major versions later, it would become
officially supported, meaning that it will not enter LTS support until another
year after that, as I understand it. The timeline on this feature is long</em> and
there are go/no-go checks before it becomes supported.</p>
The earliest a flagged version could land is the week of the 22nd, February
2016. If landed, the first go/no-go check for unflagging would happen in
October 2016.</p>
const</span> fs</span> =</span> require</span>(</span>'fs'</span>)</span></span>
</span>
fs</span>.</span>readFilePromise</span>(</span>'some/file'</span>)</span>.</span>then</span>(data</span> =></span> {</span></span>
  console</span>.</span>log</span>(data)</span></span>
})</span></span>
</span>
// shortcut for top-level promise API</span></span>
const</span> {readFile}</span> =</span> require</span>(</span>'fs'</span>)</span>.</span>promised</span></span>
</span>
readFile</span>(</span>'some/file'</span>)</span>.</span>then</span>(data</span> =></span> {</span></span>
  console</span>.</span>log</span>(data)</span></span>
})</span></span></code></pre>
Only “single occurrence” async APIs provide promise variants at the moment.
In practice, this means that the following core modules will include promise
variants:</p>

cluster</code></li>
child_process</code></li>
crypto</code></li>
dgram</code></li>
dns</code></li>
fs</code></li>
zlib</code></li>
</ul>
Some methods of net.Socket</code>, readline</code>, and repl</code> also provide promise
variants.</p>
The Node Core Technical Committee is planning on meeting to determine how to
move forward on this PR sometime during the week of the 22nd.</p>
Technical Issues & Proposed Solutions</h2>
AsyncWrap, Domains and Microtask Queue</h3>
Native promises currently interact incorrectly with Domains and AsyncWrap. This
is largely due to a lack of visibility into, and control over, V8’s “microtask
queue.” The microtask queue is a language-mandated queue of functions to be run
after the exhaustion of a JavaScript stack. In the case of promises, settling a
promise with pending handlers OR the addition of handlers to a settled promise
is mediated through the microtask queue. AsyncWrap currently relies on Node’s
ability to exert full control over the top-level entry points from event loop
to JavaScript in order to provide hooks for users to track new sources of
asynchrony. Domains also lean on this ability. The microtask queue is
opaque</strong> to Node — Node cannot see when</em> functions are added to the queue,
or specifically when</em> those functions are executed inside the queue. Given
this, it’s impossible for Node to fire the appropriate AsyncWrap hooks, or to
associate the appropriate domain with a promise handler.</p>
I am exploring making V8’s Microtask Queue pluggable, in the style of the
ArrayBuffer::Allocator</code> API. This should allow us to add the necessary hooks
for AsyncWrap</code> and domains, while affording us the opportunity to consolidate
the nextTick</code> and microtask queues into a single queue.</p>
Promises Unsuitable for Post Mortem Debugging Tools</h3>
The @nodejs/post-mortem</strong> working group has expressed concerns about Promise
use interfering with post-mortem debugging tooling. Specifically, the reliance
on using throw</code> as a mechanism for error propagation means that in many cases,
processes that crash on unhandled rejection will have already unwound the
stack. Unwinding the stack is highly undesirable for post-mortem purposes, both
because the stack contains stack frame parameter information (which Error</code>
objects lack), as well as because distancing the exceptional state from the
serialization of program state means that valuable heap information may have
changed before the program was serialized.</p>
Two mitigations are currently being investigated:</p>

Skip intermediate non-user-installed handlers on promise rejection. This
allows the rejection to propagated to the unhandledRejection</code> handler
without unwinding the stack. Change the default unhandledRejection</code>
behavior so that in the absence of a user-installed
process.on('unhandledRejection')</code> listener the process crashes, and investigate
making synchronous rejections signal unhandledRejection</code> immediately, versus
the current “on next tick” behavior. These steps would combine to make
unhandled rejections work “as expected” with current post-mortem tooling,
and align unhandled rejection behavior with uncaught exception behavior.
This also makes explicit an unwritten rule already in effect for package
authors, which is “avoid leaning on unhandled rejection behavior,” since
authors cannot currently rely on consumers not to crash the program when
unhandled rejections emerge. See this comment</a> for more.</li>
Explore adding a V8 Context</code>-associated hook for VM-originated programmer
errors, like ReferenceError</code>, TypeError</code>, etc. This would allow post-mortem
users to short-circuit promise (& try/catch) machinery and abort on programmer
error.</li>
</ol>
I believe that we should have a clear answer for the Post Mortem WG’s concerns
before unflagging the feature, but not necessarily before landing the PR.</p>
Rejecting Operational Errors Hides Programmer Errors</h3>
Some users wish to limit the use of try/catch</code> to truly exceptional,
unexpected situations. Node currently caters to this need by only throwing
exceptions from asynchronous methods if the provided arguments are invalid for
the API in question. These users would prefer to continue to reserve exceptions
for unexpected behavior as they start using async/await</code>. Most existing shims,
and indeed the original PR, treat operational errors — EEXIST</code>, ENOENT</code>,
EMFILE</code> — as rejections, which map to thrown exceptions in async/await</code>
programs. As an example:</p>
async</span> function</span> reader</span> (filename) {</span></span>
  try</span> {</span></span>
    const</span> data</span> =</span> await</span> fs</span>.</span>readFilePromise</span>(filename)</span></span>
  }</span> catch</span> (err) {</span></span>
    // could be a programmer error ("filename was undefined") or</span></span>
    // it could be an operational error ("filename does not exist").</span></span>
  }</span></span>
}</span></span></code></pre>
For many users the above example represents acceptable use and is analogous to
the fs.readFileSync</code> API. Others wish to keep programmer errors separated from
operation errors.</p>
There are three potential mitigations:</p>

Via @littledan</strong></a>: When implementing the Promise API, use destructuring to
return [err, value]</code>.</li>
Via @zkat</strong></a>: Allow a recovery</code> object to be passed to Promise-wrapped
functions in order to specify behavior in case of operational errors.</li>
If implemented, the first mitigation approach from the post-mortem section
would separate programmer errors from operationals error by crashing the
programs lacking a process.on('unhandledRejection')</code> handler on synchronous
rejection.</li>
</ol>
Examples follow:</p>
// Mitigation 1 with async/await</span></span>
// - err may be a programmer error _or_ a operational error</span></span>
const</span> [err, value]</span> =</span> await</span> fs</span>.</span>readFilePromise</span>(path)</span></span>
</span>
// Mitigation 1 with raw promises</span></span>
fs</span>.</span>readFilePromise</span>(path)</span>.</span>then</span>(([err, value])</span> =></span> {</span></span>
})</span></span>
</span>
// Mitigation 2 with async/await</span></span>
// - throws exception on bad "path" param</span></span>
const</span> value</span> =</span> await</span> fs</span>.</span>readFilePromise</span>(path, {</span></span>
  ENOENT</span> () {</span></span>
    return</span> null</span></span>
  }</span></span>
})</span></span>
</span>
// Mitigation 2 with raw promises</span></span>
fs</span>.</span>readFilePromise</span>(path, {</span></span>
  ENOENT</span> () {</span></span>
    return</span> null</span></span>
  }</span></span>
})</span>.</span>then</span>(value</span> =></span> {</span></span>
</span>
})</span></span>
</span>
// Mitigation 3</span></span>
// - throws operational errors (ENOENT)</span></span>
// - when passed a bad path, crashes the program (regardless</span></span>
//   of try/catch)</span></span>
const</span> data</span> =</span> await</span> fs</span>.</span>readFilePromise</span>(path)</span></span></code></pre>Where to Participate</h2>
The main PR</a> is the best place to participate. I’ve added a FAQ at the top
with links to specific responses to make navigating the thread a bit easier. If
you don’t have the time to read the entire thread, please continue share your
concerns and I will attempt to address and direct them as they come in. If you’re
nervous about wading into the thread, please feel free to contact me directly
at christopher.s.dickinson at gmail dot com, or via Twitter @isntitvacant</strong>.</p>
Thanks for your time!</p>
Chris Dickinson

you have to know what to wish for

testing, one two

"The Economics of Programming Languages - Evan Czaplicki"

Understanding Wasm, Part 3: You Are Here

Understanding Wasm</h1>

Processes and Virtual Machines</h2> Processes are a universal abstraction in modern operating systems. They comprise three capabilities:</p>

WASI</h2> (Finally.)</em></p>

A Component Model</h2> Through the Wasm Component Model</strong>, WASI preview 2 fundamentally rethinks the process model.</p>

Epilogue</h2> So, whew. That’s a lot.</p>

Bibliography and Timeline</h2> So many PDFs this time around!</p>

Understanding Wasm, Part 3: A System Interface

Understanding WASM</h1>

Processes</h2> Processes are a near-universal concept in modern operating systems. They comprise three capabilities:</p>

Bibliography and Timeline</h2> So many PDFs this time around!</p>

Understanding Wasm, Part 2: Whence Wasm

Understanding Wasm</h1>

GUARDA COME DONDOLO / Edouardo Vianello

understanding wasm pt 2: whence wasm

understanding wasm</h1>

Anxiety Groove | Louie Zong

Plain Text Journaling · peppe.rs

Understanding Wasm, Part 1: Virtualization

Understanding Wasm</h1>

A Mirror for Rust: Compile-Time Reflection Report - Shepherd's Oasis

In defense of blub studies | benkuhn.net

6.6

Dead 80's | Mrs. Magician

Don't Try | Radioactivity | Dirtnap Records

JAPANTHER - FIRST OF ALL | JAPANTHER | Seayou Records

Never Again | Thin Lips

Nostalgia of Island | Tatsuro Yamashita

Simulation Swarm | Big Thief

Something About Lemons | Chumped

x-post: LARPing a Gossip Protocol about Reality

"The UNIX Programming Environment", reviewed

(2019) Keynote: The Path to Components - Luke Wagner, Distinguished Engineer, Fastly - YouTube

IPv6 Internet Is Broken

Refactoring Is Not Just Clickbait - Kevlin Henney - NDC Oslo 2022

Running LXD Containers in Macpine

So you're Staff+ ... now what?

The Continuity of Splines - YouTube

The Rust Programming Language for Game Tooling

toast - crates.io: Rust Package Registry

Watchexec

Color Studies

Color Studies</h1> TKTK</p> After the last post, I want to back away from tech for a bit, so I’m writing about color palettes today.</p>

F-Zero: Big Blue</h2> </div> </div> </div> </div> </div></div> The nineties aesthetic loves a beach. We hit a bunch of loud, hyper-saturated colors here: deep blues, palm tree greens, and neon oranges. The rambunctious colors are grounded by a staid, almost wine-like, magenta.</p>

Portland Springtime Cherry Blossoms Part Two</h2> </div> </div> </div> </div> </div></div> Found via this Reddit post</a>.</p> Deep dream rainbow</h2> </div> </div> </div> </div> </div> </div> </div> </div></div> Found via this Tweet</a>.</p>

Deep dream rainbow</h2> </div> </div> </div> </div> </div> </div> </div> </div></div> Found via this Tweet</a>.</p>

The Canvas Cares Not

A Front-Row Seat at the “Best Art” Awards</h2> First, art. Why do I say software engineering is art? Art is communication: between people, between the artist’s past self and present self, between the author and the audience. Music is art, prose is art, cooking is art, architecture is art.</p>

Finding One’s Voice</h2> First, release the idea that it is possible to be right about software engineering.</strong></p> Second, I am going to quote John Gardner’s “The Art of Fiction”, which is as applicable to fabulists as it is to software engineers:</p>

SSH to Cloud Managed Servers

Using ProxyCommand to SSH to Cloud Managed Servers</h1>

My Note-Taking Setup

the case for bash

The Case For Bash</h1> esac</code></small></p> Friends, coders, I come not to praise bash, but to contextualize it.</p>

context

Deploying at Eaze

What happened to July?

PLAN update

Career Progression

PLAN update

Boltzmann & Nostalgia

Watching: The Forgotten Art of Structured Programming

a statement of intent

How does it work?</h2> Okay, you asked for it – let’s navel-gaze.</p>

x-post: adding promise support to Node.js

Color Studies</h1>
TKTK</p>
After the last post, I want to back away from tech for a bit, so I’m writing about color palettes today.</p>
I used Colormind</a> to derive palettes from images that resonated with me.</p>

Portland Springtime Cherry Blossoms Part Two</h2>
</div>
</div>
</div>
</div>
</div></div>
Found via this Reddit post</a>.</p>

Deep dream rainbow</h2>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>
Found via this Tweet</a>.</p>

Deep dream rainbow</h2>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>
Found via this Tweet</a>.</p>
