Do you know what a Monad is1? People have struggled to explain what they are, which is not particularly surprising since it is a very abstract concept. There’s even a timeline recording the history of (in)famous monad tutorials. Many try to explain monads through analogy, which really doesn’t work. The tutorials got so bad someone wrote an article complaining about them and unintentionally created a meme that monads are like burritos.

Here’s what a monad is, as far as a programmer (not a mathematician) is concerned, in some Rust-like pseudo code:

trait Monad<M, A> {
  fn new(v: A) -> M<A>;
  fn and_then<B>(m: M<A>, f: fn(A) -> M<B>) -> M<B>;
}

That’s it, that’s all it is. The names of the two functions vary, “new” is sometimes called “pure” or “return”2, while “and_then” is often called “bind” or “flat_map”. This is a higher-kinded type, meaning that it is generic over a type constructor (the M above), something most languages can’t natively express3.

The important part is that “and_then” function, the other function just means there’s a way to construct the type. The “and_then” function lets you sequence callbacks that stay “trapped within the monad”, so to speak. What’s that useful for you ask? Well it appears in many different contexts:

• Rust’s Option<T> is a Monad: here’s and_then. Lets you sequence callbacks that work with values if there are any and abort early if one is missing.

• C#’s Task<T> is a Monad: and_then is ContinueWith. Lets you sequence asynchronous callbacks. Promises in JavaScript are the mainstream example.

• Java’s Stream<T> is a Monad: and_then is flatMap. Lets you sequence callbacks that produce streams of values into producing one big stream.

If you have never used these functions/methods, don’t worry too much about it, they aren’t that useful most of the time, but now you know the pattern.

What the “Monad trait/typeclass/protocol/interface” makes possible is to write code that is generic over all of these different types.

For most programmers the fact that these types share a common structure is utterly useless, I honestly struggle to think of any practical use case. BUT, for a language developer, it enables some neat tricks.

Haskell has something called do notation that works with any monad. It turns imperative-looking code into calls to “and_then” (which is called >>= in haskell):

• If you use it with the Maybe/Either monads (Option/Result in Rust), you get behavior similar to checked exceptions or the ? operator, but not hardcoded into the language.

• If you use it on MonadYield from the yield package, you get generators that aren’t hardcoded into the language.

• If you use it on MonadAsync from the async package, you get async/await that isn’t hardcoded into the language.

But also logic programming, constraint programming, probabilistic programming, etc. Here’s an example of async in Haskell (from an async package test case):

async_poll = do
  a <- async (threadDelay 1000000)
  r <- poll a
  when (isJust r) (assertFailure "")
  r <- poll a   -- poll twice, just to check we don't deadlock
  when (isJust r) (assertFailure "")

The async above is not a keyword, it is not built into the language. This is the async monad plus the do notation syntax sugar. It gets expanded to something like:

async_poll =
  -- (\x -> foo) is how you make anonymous functions in haskell
  async (threadDelay 1000000) >>= (\a ->
    poll a >>= (\r ->
      when (isJust r) (assertFailure "") >> (
        poll a >>= (\r ->
          when (isJust r) (assertFailure "")))))

The >>= (bind/and_then) operators are sequencing the callbacks. The >> operator is just a wrapper that doesn’t pass along any value. This is similar to how async/await in JavaScript is essentially just syntax sugar for callback hell, but here the syntax sugar works for any type that defines the >>= operator.

F# has a similar feature in its Computation Expressions. It can’t represent the Monad concept as a type, so it just directly checks for a method called Bind (our friend and_then). As with Haskell’s do notation, this gets you checked exception-like early returns, async, generators, etc. with one language feature.

Isn’t this neat? A two method interface plus some rather trivial syntax sugar lets you implement all these other fancy language features as regular library code. Monads + “do notation” creates a sort of “uber language feature” that lets you implement other language features.

By implementing just one language feature, you get all those other fancy language features “for free”, even ones you haven’t thought of yet.

Sadly, however, you don’t get their combinations for free.

Monads don’t compose very well by default. If you want to mix async with exceptions, you either need to implement the combined Monad manually, or you need to use something called Monad Transformers. The simplicity is not so simple anymore4.

Thankfully, there is another language feature that shares this magical ability to model nearly any crazy language feature you can think of, is even easier to understand, and composes extremely well: Algebraic Effects.

Algebraic Effects

An algebraic effect is a conceptual framework in programming that separates the declaration of a side effect (what the code wants to do) from its execution (how it actually happens). A function raises (for example) a yield effect, and then an effect handler installed higher up the callstack handles the request.

What’s an effect and what is an effect handler, you ask? Unlike Monads where analogies quickly break down, the easiest way to describe effects is as a generalization of exceptions:

• exception → effect

• exception handler → effect handler

The main difference being that effects are resumable, potentially multiple times. Exceptions can be modeled as effects that just never resume.

Two caveats about this conceptual generalization:

• Resumable exceptions are a pretty awful idea, don’t do that.

• Performance would suck if implemented as exceptions are usually implemented.

Alongside the runtime aspect, there is also a type system aspect to effects. They’re a bit like fixed checked exceptions in that regard, you’ll see what I mean below.

The closest thing to a mainstream language that supports effects is OCaml, though it only implements them as a runtime feature. Haskell also supports them as a library by implementing them using monads. Remember what I said about monads letting you implement nearly any crazy language feature you can think of? Well, that happens to include algebraic effects too somehow.

Research languages like Koka, Eff and Effekt are where most of the action is happening. We’ll use Effekt here since it can run in the browser and has a rather familiar looking syntax, if you want to try it out.

First, the exception effect which will look very familiar:

effect throw(msg: String): Nothing

// We use double in the example because int division already does this
def div(n: Double, m: Double): Double / { throw } = {
  if (m == 0.0) {
    do throw("Division by zero")
  } else {
    n / m
  }
}

This gets you something like Swift’s approach to error handling, which requires adding a throws annotation to any function that can throw an exception, except here the throw annotation is an effect annotation like any other.

The effects a function performs are recorded in a set after the return type. Effekt can infer the set of effects, so you don’t have to explicitly write it out, but it’s always a good idea to do so. Effects are triggered with the “do” keyword, and indirectly by calling a function that performs effects.

Handling an effect can be done as follows:

def zdiv(a: Double, b: Double): Double / {} = {
  try {
    div(a, b)
  } with throw { msg =>
    0.0
  }
}

Notice how the effect is discharged from the effect set after the return type. So far, this is little more than improved checked exceptions. Let us do generators next:

effect yield[A](x: A): Unit

def fib(limit: Int): Unit / { yield[Int], throw } = {
  if (limit < 0) {
    do throw("expected positive limit")
  }
  var last = 0
  var current = 1
  while (limit == 0 || current < limit) {
    do yield(current)
    val next = last + current
    last = current
    current = next
  }
}

Here’s a generator of infinite Fibonacci numbers with an optional limit. I have an article explaining how to do this with coroutines in C if you prefer, but here we’re using effects. It also throws an “exception” if the given limit is a negative number.

Let’s create a silly function that uses the yield effect: it keeps printing the sums of the Fibonacci numbers until they exceed the given limit.

def sumFibs(limit: Int): Int / {} = {
  var sum = 0
  try {
    fib(limit)
  } with yield { i =>
    sum = sum + i
    println(sum)
    resume(())
  } with throw { msg =>
    println(msg)
    // no resume
  }
}

The language has no support for yield built-in nor “exceptions” in the traditional sense, we did all of this as a library. If we wanted we could also allow yield to return a value, so the handler can feed values into the function creating bi-directional communication.

I only code in C sir, why do I care?

Have I got a treat for you, Algebraic Effects in C: libhandler.

There’s also a newer library from the same authors that builds on top of multi-prompt delimited control (another one of these uber features): libmpeff.

They’re not the nicest thing in the world to use but it’s amazing how some assembly tricks are enough to make this work in a language very much not designed for it.

Ok, neat, how do I implement these?

If you want to handle it at the level of the compiler, rather than delegating the runtime to libhandler/libmpeff, the most efficient way is with evidence passing.

It basically involves rewriting functions into a monadic form and passing in the handlers as a hidden argument. A bit hard to explain (even I struggle to understand exactly how it works) but we can implement a limited form of algebraic effects (one-shot resume, resume is a keyword not a function) without too much pain, by translating functions that do effects to coroutines.

First, we define effects and their signatures. We’ll use a tagged union of effects for simplicity (great for the “internal to the compiler” case), but if we wanted it extensible it’s pretty easy to do as well, just more code.

typedef enum EffectKind {
  EFF_NONE,
  EFF_THROW,
  EFF_YIELD_INT,
} EffectKind;

typedef struct Effect {
  EffectKind kind;
  union {
    struct { const char* msg; } throw;
    struct { int v; } yield_int;
  };
} Effect;

#define DONE (Effect){EFF_NONE}
#define do_throw(msg) (Effect){EFF_THROW, .throw = {msg}}
#define do_yield_int(v) (Effect){EFF_YIELD_INT, .yield_int = {v}}

Next we turn fib into a coroutine. Every time an effect is triggered it “pauses” and returns a request for the effect to be performed (or the NONE effect when finished). We’ll use GCC’s first class label extension for clarity and to make the code shorter, but it is not necessary (see my article on generators in C to see what to do instead).

typedef struct FibFrame {
  // "program counter", tracks where we are in the coroutine
  void* pc_;
  // args
  int limit;
  // locals
  int last;
  int current;
  int next;
} FibFrame;

Effect fib(FibFrame *f) {
  if(f->pc_) goto *f->pc_;
  if (f->limit < 0) {
    f->pc_ = &&after_throw_1;
    return do_throw("expected positive limit");
  after_throw_1:
  }
  f->last = 0;
  f->current = 1;
  while(f->limit == 0 || f->current < f->limit) {
    f->pc_ = &&after_yield_1;
    return do_yield_int(f->current);
  after_yield_1:
    f->next = f->last + f->current;
    f->last = f->current;
    f->current = f->next;
  }
  // reset the function
  f->pc_ = NULL;
  return DONE;
}

Finally, let us convert sumFib. It triggers no effects of its own and handles all effects of the functions it calls, so it does not need to be a coroutine, but for consistency we will also turn it into one.

typedef struct SumFibsFrame {
  void* pc_;
  int limit;
  int sum;
  FibFrame fib_frame_1;
  // final return value
  int return_;
} SumFibsFrame;

Effect sum_fibs(SumFibsFrame* f) {
  if (f->pc_) goto *f->pc_;
  f->sum = 0;
  // try block 1
  {
    Effect effect = {EFF_NONE};
    f->fib_frame_1 = (FibFrame){.limit = f->limit};
  resume_fib_1:
    effect = fib(&f->fib_frame_1);
    if (effect.kind != EFF_NONE) goto handlers_1;
    goto try_end_1;
  handlers_1:
    switch(effect.kind) {
    case EFF_YIELD_INT:
      f->sum += effect.yield_int.v;
      printf("%d\n", f->sum);
      goto resume_fib_1;
    case EFF_THROW:
      printf("%s\n", effect.throw.msg);
      break;
    }
  try_end_1:
  }
  f->return_ = f->sum;
  f->pc_ = NULL;
  return DONE;
}

Phew… even for this simplified version the code gets quite gnarly, but this is meant as something a compiler generates. If multi-shot continuations are allowed, the code becomes a lot more complicated as it requires managing heap allocated copies of the frames AND of the handlers because when the function is resumed and it triggers an effect it needs to be handled by the try block that originally captured the resumption5.

Out of scope for an already very large blog article.

Absolute Power Corrupts Absolutely

These two features, Monads + “do notation” and Algebraic Effects, are a little too powerful. The main problem with putting this sort of power in the hands of library developers rather than the language developer is that it can cause a large splintering of the ecosystem: multiple ways to do exceptions, multiple ways to do async, multiple ways to do generators, multiple ways to do implicit contexts, etc.

You don’t really want competing implementations of these features, you want those things standardized. This is also a problem in languages like Lisp where everyone makes their own little domain-specific languages using macros.

So I’m not sure exposing this as a user-facing language feature is a particularly good idea. What I would do is implement it in the compiler, and then implement user facing features (exceptions, generators, async, etc.) in terms of it. That way their interactions are well define and the implementation far simpler.

I would also expose the feature publicly but only if an “-experimental” flag is passed to the compiler. That allows the community to experiment with interesting uses of effects (like implementing a Prolog-like backtracking engine within the language) before they are standardized, but making it clear it’s not meant for production code.

Conclusion

Two fancy language features demystified in one blog post, aren’t I ambitious?

I really think these features are too powerful, having them widely used and abused will lead to a messy ecosystem. Most code should be made up of simple function calls.

But if you’re making a new language and you want to take it as far as it will go, here are two new tools for your tool-belt.

That question lead to me asking this followup question:

If you know any formal language theory, you probably know the following:

LL grammars cannot have rules containing left recursion.

Why? Because LL grammars are defined as “context-free grammars that can be parsed by an LL parser”, with an LL parser defined as a parser that “parses the input from Left to right, performing Leftmost derivation of the sentence”1, and a parser that works like that cannot handle left recursion2.

Different issues occur depending on how the parser is implemented. Consider the following grammar:

expr : expr op term
     | term;

op : "+" | "-"

A recursive descent parser (which fits the definition above) will enter an infinite loop:

def expression():
  expression()
  op()
  term()

A table-driven parser will detect a conflict because it cannot decide how to make progress, usually with a cryptic message if it hasn’t been explicitly designed to detect and point out the left recursion.

So far, the sentence seems to be true. But it’s also not very useful, because in practice it is only really true for indirect left recursion.

Eliminating Left Recursion

It is well known that simple cases of left recursion can be eliminated pretty easily. Some tweaks to the grammar mostly avoid the problem. The grammar above (presented again for convenience):

expr : expr op term
     | term;

Can be rewritten into the following:

expr : term expr_tail;
expr_tail: op term expr_tail | ε;

Or, if your LL parser generator allows it, to the following:

expr : term (op term)* | term;

If you studied formal language theory in university your professor probably taught you this trick. If you read the documentation of an LL parser generator like ANTLR33, this is how it suggests to handle expressions.

This results in a weakly equivalent grammar, meaning it recognizes the same language (and generates the same set of strings), but with a different parse tree. The parse tree tilts to the right rather than to the left.

Given an input like 1 - 2 - 3, the original grammar parses it as (1 - 2) - 3, whereas the transformed grammar parses it as 1 - (2 - 3).

So why does this matter? You probably didn’t even think of it, but I snuck in a little gotcha above:

“Or if your LL parser generator allows it”. That little * operator there, is that LL? What about the group ()? Is that LL?

Reductions

If you’re implementing a table-driven LL parser, the EBNF grammar with those operators needs to be simplified (reduced) to a standard BNF form. That means the grammar that actually ends up being recognized is not the same grammar you fed the parser generator. It was rewritten4.

Now, my question to you is, do you consider parser generators that support such operators (like ANTLR3, JavaCC, Yapps, etc.) to be LL parser generators? Most people seem to think so, and they refer to them as such. ANTLR3 is LL(*), others LL(k) or LL(1), but LL regardless.

They may not do the reduction to standard BNF internally because they’re not table-driven, instead directly generating recursive descent code, but does that change anything? Recursive descent code can handle full PEG. The part that makes it LL and not PEG is the lack of backtracking right? The important part is not that it is parsed by a table-driven solution, but that it could be.

ANTLR3, being LL(*), should actually be considered to be in the domain of TDPL (because it has syntactic and semantic predicates), making it closer to a restricted form of PEG, not your typical LL parser. But everyone is ok with calling it an LL(*) parser generator.

Which gets me to ANTLR4. ANTLR4 is an Adaptive LL(*) parser generator (whatever that means), and it supports direct left recursion. Does that disqualify it from being an LL parser? Maybe ALL(*) is a special case and doesn’t really count.

But I vibe coded an LL(1) parser generator in Python that handles direct left recursion just fine:

expr : a=expr "+" b=term { a + b } 
     | a=expr "-" b=term { a - b }
     | t=term { t } ;
term : a=term "*" b=factor { a * b } 
     | a=term "/" b=factor { a / b } 
     | f=factor { f } ;
factor : "(" e=expr ")" { e }
       | "-" f=factor { -f }
       | token=NUMBER { float(token.text) }
       | token=IDENT { ctx[token.text] } ;

This grammar is accepted and evaluates to the result you’d expect for a given expression with the correct precedence.

It can work in a table-driven manner, and the left-recursion rewrite sits alongside the same code that rewrites groups () and repetitions *.

It can also directly generate recursive descent code, in which case it translates the direct left-recursion into a fold, similar to a Pratt parser. Or it can translate the rewritten BNF grammar into a recursive descent parser too; it still works, albeit less efficiently. The rewrite is completely trivial, I showed you the trick previously, but here is the internal BNF, generated automatically:

expr : t=term _v=$expr_star[t]  { _v } ;
term : f=factor _v=$term_star[f]  { _v } ;
factor : "(" e=expr ")"  { e }
       | "-" f=factor  { -f }
       | token=NUMBER  { float(token.text) }
       | token=IDENT  { ctx[token.text] } ;
$expr_step[a] : "+" b=term  { a + b }
              | "-" b=term  { a - b } ;
$expr_star[a] : _i=$expr_step[a] _v=$expr_star[_i]  { _v }
              | { a } ;
$term_step[a] : "*" b=factor  { a * b }
              | "/" b=factor  { a / b } ;
$term_star[a] : _i=$term_step[a] _v=$term_star[_i]  { _v }
              | { a } ;

Where do you draw the line? Some PEG parsers, which also cannot handle left-recursion, also handle direct left-recursion, like OhmJS.

If the definition of what makes an LL(k) parser is “parses the input from Left to right, performing Leftmost derivation of the sentence, with k tokens of lookahead”, then my little parser generator above fits the bill.

Reductions are the Norm

In other computer science areas of study, we’re not usually too worried about the exact input that goes into something, but what can be reduced to that input.

A classic example is the Boolean Satisfiability Problem (SAT), which asks whether there exists an interpretation that satisfies a given Boolean formula. In other words, it asks whether the formula's variables can be consistently replaced by the values TRUE or FALSE to make the formula evaluate to TRUE.

SAT is the prototypical NP-complete problem. If you want to know if a problem is in NP, one way to figure that out is by coming up with a reduction to SAT.

What makes SAT useful are precisely these reductions. Not just theoretically, but in practice too. Checking if a propositional logic formula is satisfiable isn’t particularly interesting, but finding a solution to Sudoku is, and Sudoku can be reduced into SAT.

Most SAT solvers don’t even accept arbitrary boolean formulas, they require them to be rewritten into Conjunctive Normal Form. You can even reduce the problem to 3-SAT, which only allows a maximum of 3 variables per disjunction. Still just as expressive.

It’s that expressive power that matters. The fact that you can solve Sudoku, not that the solver is actually working with conjunctions of boolean variables internally. Not the input format, but what can reasonably (i.e., in polynomial time) be reduced to it.

Theory and practice are at odds here as well. NP problems have exponential complexity, O(2ⁿ), but SAT solvers can tackle absolutely massive industrial problems with millions of variables and constraints. The algorithms they use can exploit hidden structures in the problem and converge to a solution incredibly fast.

The CPU in your computer probably has a SAT solver to thank for its existence, either for how its internals are laid out, or for its correctness (or both!). This wouldn’t be possible if SAT solving was always O(2ⁿ). O(2ⁿ) is the worst case scenario, most problems can be solved much more efficiently. The worst case is important to know, but what can actually be done in practice is perhaps even more so.

Back to LL

So given the above, I hope I’ve shown you why the sentence:

LL grammars cannot have rules containing left recursion.

Should instead be:

LL grammars can only have rules containing direct left recursion5.

Because an LL grammar is defined by what an LL parser can recognize, and an LL parser can easily deal with direct left recursion. And direct left recursion happens to include the most relevant category of left recursion: mathematical expressions.

If you search around the web or ask your favorite LLM (including Google’s if you need a free one) why one should favor LR over LL, they’ll tell you something like:

Support for Left Recursion: LL parsers will fall into infinite loops if the grammar contains left-recursive rules (e.g., A → A α). LR parsers natively handle left recursion, making it easier to define structures like mathematical expressions.

This statement is mostly false. This is not a reason to use LR over LL. Definitely not for the specific rule example Gemini used (which is directly left recursive). Even the “infinite loop” part of the answer is wrong, you’ll get a conflict unless you’re translating to a recursive descent parser by hand.

This type of answer is what you get when you only focus on worst-case theoretical limitations rather than what can be done in practice.

But the thing that makes the sentence not entirely false is one word: natively. It is true most LL parsers do not natively handle any left recursion. They could handle some, but they don’t. This is an engineering problem, not a theoretical one.

Conclusion

When a silly discord question leads a person to vibe coding an LL parser generator. Programmers sure are a weird bunch, or at least I am.

I’ve made my point as to what I think the answer to the original question is and why: parsers that can handle direct left recursion can 100% still be considered LL, because what matters is how they work, and preprocessing is normal anyway.

But I’m curious to know what you think.

That was the main reason for the drought, but another reason is a project I’ve been working on: a UI library called SUIT (Stylish User Interface Toolkit).

Think Electron, but in Reverse

Every project needs a raison d'etre, a reason why it exists. Sometimes that’s just to scratch an itch or learn something new, other times it’s an attempt at targeting an unmet market need. SUIT is the latter, I’m trying to target a similar market to Electron, but in reverse1:

There are already various libraries that are used to develop native apps2 that can also target the web (Qt, Avalonia, etc.), but that’s not really what I mean. Those libraries produce native apps and those native apps can then run on the web, but the resulting “web app” is not really a “web-native” app, so to speak.

Just as Electron apps can feel somewhat out of place, Qt and Avalonia apps feel extremely out of place on the web, far worse than Electron apps do on the desktop. Qt and Avalonia custom render everything, so you don’t get the usual HTML and CSS behaviors one would normally expect from a web app. They’re also very large, and that size adds to the size of the exported web app. Small size still matters on the web, and neither Qt nor Avalonia are very good in this regard.

SUIT does not work like Qt or Avalonia, it is more like libUI or wxWidgets in that it wraps the native UI toolkits provided by the operating systems, plus HTML + CSS on the web. This introduces some very harsh design constraints, but so far nothing insurmountable.

What does SUIT do differently from libUI and wxWidgets then? It has to do with the “S” in the name (which stands for “Stylish”); SUIT neither uses nor mimics the native platform’s appearance. The core tenet of SUIT is:

Why not use the native look? Read on.

The Native UI Toolkit Disaster

There are no two ways about it, the native UI toolkits mostly suck. I’ve covered the absolute disaster that is Windows previously, so I won’t go into it in detail again, but the TL;DR is that Windows has an absolute graveyard of half-finished UI toolkits, and you simply cannot trust any Microsoft provided API not called Direct3D3. I’ll explain what I do regarding Windows later.

Apple-land fares better, but the cracks are starting to show. AppKit is very good, but even AppKit has quite a few limitations compared to some of the things you can do on the web. As an example, even something as trivial as a different radius per corner is something CoreAnimation does not support (you can mask some to 0, but that’s it). Same with a different border color per side. You can always do it yourself with multiple vector shape layers or by custom drawing with CoreGraphics, but that’s a lot of extra work.

Now there’s SwiftUI which Apple refuses to decide if it’s just a React-ish wrapper around AppKit, or 100% “the future” and AppKit is just an implementation detail for the time being. We might be looking at a Microsoft-like situation here eventually.

On Linux there are Qt and GTK, and both have issues.

Qt is quite good, but it’s been a bit of a mess since the introduction of QtQuick. Why they thought forcing people into QML instead of it being just a nice wrapper on top of a C++ API I have no idea, not to mention the amount of churn QtQuick has gone through. QtWidgets are still there and work very well but they’re clearly an afterthought these days.

GTK is actively developer hostile. They have backtracked on what must have been the stupidest thing I’ve ever read from the developer of a “platform API” (i.e., actively breaking backwards compatibility every 2 years, on purpose), but the fact that they wrote it at all is horrifying. They’ve also removed lots of widgets that could have been reimplemented as wrappers around their new APIs, no thought given to developers who just wanted an easier upgrade path from GTK 3 to 4.

Qt also breaks backwards compatibility between major versions, but it goes out of its way to make the transition as painless as possible.

The Native UI Style Disaster

On top of all of this, you also have the native style churn. The shift from Windows 7 to 8 was brutal, there was absolutely no way a “native app” could have adapted. The WPF ribbon control kept the very skeumorphic gradient for a long time, which looked horrifically out of place on Windows 8 (not to mention it broke the window chrome). Windows 8 was also hideous, but that’s just my subjective opinion.

Apple has now introduced liquid glass which is 100% the ugliest desktop UI style ever created, and this one is my objective opinion4. Here too adapting a native app that followed older styles would be a nightmare, even if it used native widgets almost exclusively (you have to adjust how your app is laid out to make the glass effect actually matter, or you end up like Apple’s own half-baked apps that look downright schizophrenic).

And there’s absolutely no way to design a cross-platform app that would “fit in” everywhere. Not only do style guidelines keep changing within the same operating system family across versions, they’re also different across operating systems. The only way to get something truly native looking is to maintain an entirely separate UI for each platform and to redo each of those UIs whenever an OS provider decides to overhaul the style to distract from the falling quality of their software.

And for what? Aero and Aqua were beautiful and hard to replicate, it would have cost you a pretty penny to hire a design team that could pull off that sort of UI, so using the OS-provided widgets bought you a lot. But Windows 8 Metro? Single color rectangles? Trivial. Liquid Glass? Hard to replicate but it’s hideous, why would you want to make your app hideous? If the OS provider design teams suck this hard, you might as well just do your own thing. And software companies have.

The SUIT Proposition

SUIT rejects this whole mess, and uses the system-provided toolkits only where they provide actual value, their default looks be damned. Your app, your branding.

Nearly every cross platform app these days has given up on “native look and feel”. Even when the app isn’t just a web app bundled with Chromium, the app follows the company’s own style guidelines and branding, rather than the operating system provider’s guidelines (with good reason, cuz they suck these days!). If you use Photoshop on Windows or on macOS, it looks the same; if you use Blender on Windows or macOS, it looks the same.

SUIT is being developed with this reality in mind. It mostly gives up on native look, while trying to retain some of the feel. It achieves this by using only a limited set of native widgets (e.g., text fields) while removing their default styling entirely. Appearance is then almost completely under the control of the developer. It’s not pixel perfect across platforms, but it doesn’t need to be.

Because it uses the native frameworks, you can always host a native widget alongside the custom ones if you so wish. Some widget like scrollbars are particularly relevant because there are system settings that affect their behavior, which is why SUIT supports native scrollbars by default even though that goes against the “custom appearance” goal (you can use custom scrollbars as well if you prefer).

Even some Electron apps put in the effort to support native context menus, because that little extra bit of platform-native feel makes a big difference.

How SUIT works

SUIT is a C API that communicates with backends written in various languages, like Objective-C on macOS or Javascript on the web.

Backends are responsible for implementing certain primitive operations (like file dialogs and vector drawing) and providing certain primitive widgets (like text fields and scrolling viewports). Backends must also translate SUIT stylesheets into native styling or drawing commands.

More complex widgets like tab bars, tree views and split panels are implemented by SUIT itself, on top of the primitive widgets. The design is very much “composition over inheritance”, where widget constructors are just plain functions that wire up lower level widgets into higher level ones.

Memory management is mostly automatic despite SUIT being a C-library, because deleting a widget also deletes all of its children, and widgets are tracked via IDs rather than pointers, so a widget-related use-after-free always causes a crash rather than memory corruption.

SUIT’s styling system is heavily inspired by CSS, albeit simpler, as styles only affect the widgets they are applied to, there is no “cascading”. The reason it is CSS inspired is not out of preference, but rather to allow a direct translation to CSS on the web. That’s what makes the web backend work so well.

Here’s a little taste of the current API, defining the calculator display style for the Windows 7 lookalike calculator shown previously:

static const SuitStyleProperty display_style[] = {
    suit_height({.px = 49}),
    suit_height_policy(SUIT_SIZE_POLICY_FIXED),
    suit_padding_left(8),
    suit_padding_right(7),
    suit_h_align(SUIT_ALIGN_END),
    suit_v_align(SUIT_ALIGN_CENTER),
    suit_overflow_x(SUIT_OVERFLOW_HIDDEN),
    suit_font_family("Helvetica"),
    suit_font_size(24),
    suit_text_color(WIN7_TEXT),
    suit_background_image_gradient(display_light_gradient),
    suit_border_width(1),
    suit_border_color(WIN7_DISPLAY_BORDER),
    suit_border_radius({.px = 2}),
};

The API is nowhere near finished and has some very rough areas still, but as I keep working on it the eventual API shape starts to emerge.

The Nightmare and the Clankers

When I first started this project, I was quite positive on it. And in many ways, I’m still quite positive on it. The foundational idea does work. SUIT produces highly customizable UIs that nonetheless retain a nice native feel. This is already true today. Typing in a SUIT text field on macOS feels the same as typing in any other macOS text field, all the functionality like autocorrect and such is there. Scrolling a SUIT viewport on macOS feels just like scrolling any other viewport in macOS, including the little bump animation when you scroll against the boundary. It feels native because it is native.

Same goes for the two Linux backends (though they are unfinished). Windows… well, we’ll take about Windows later.

But sadly I severely underestimated the sheer scope of the project. A UI library on the level of something like Qt needs a lot… a LOT of features. Layouting, styling, input and event handling, keyboard navigation, animation, accessibility, all the different widgets, etc. Now multiply all of that to 5 different backends. It’s not insurmountable, but it is massive.

I also made the mistake of trying to have a “neutral” backend, meaning one that does all the rendering itself instead of relying on the native widgets. I should have used Skia and Harfbuzz like everyone else, but instead I decided to just target the platform provided vector rendering and text APIs. I wanted to avoid non-OS-provided dependencies to keep SUIT as easy to set up as possible and to keep the resulting executables as small as possible.

Big mistake. Huge. I got it working in the end, but it was a nightmare, totally not worth it. The main culprit? Rich Text.

You don’t know hell until you’re trying to get text alignment and selection to work correctly across runs of bidi text on multiple different text layouting and rendering APIs (DirectWrite, TextKit, Pango).

So I turned to clankers5 for help, Claude and GPT to be specific. A faustian bargain.

LLMs are amazing, and yet they suck

Spoiler alert: they eventually pulled it off, but it involved constant, unending regressions (despite plenty of test cases, it’s a hard problem and mostly visual, which is hard to test).

People sometimes describe the experience of working with LLMs as being like working with a junior dev, but it’s not like that at all.

Juniors start out not really knowing what they’re doing, but slowly develop into a proper developer. You, as a senior, maintain a mental model of the junior’s capabilities, allowing you to predict the outcome of their work and thereby delegate tasks to them effectively. You know what to expect and can plan accordingly. Very quickly you begin to trust them, as least for the level of tasks you know they can currently tackle. And as they improve, the more you can feel confident in delegating.

LLMs are nothing like that. LLMs are like working with a consulting company that sends a different person to work on the project every day, sometimes every few hours. Each individual person is actually quite skilled, much more than a junior and often better than many seniors… but they just arrived at your project. They have no idea what it is about, so they first have to get up to speed. They have no attachment to it, no understanding of the company culture that led to the project being structured the way it is. They’ll half-heartedly follow the general guidelines, but everyone knows those are always incomplete.

And because they’re a “different person” each time you give a task to the LLM, you cannot develop a mental model of them, of what they’ll do. The only thing you can trust is sending them a very detailed spec with automated tests. The quality of the code you get in the end? Who knows. Did they touch stuff they shouldn’t? Maybe, you gotta check. Pretty much the way it is with a consulting company made up of humans today, only 10x worse because instead of having the same person for a few months, you have them for less than a day.

But with my life being as exhausting as it was these past few months, I just delegated everything to the LLMs after the whole rich text adventure started. The more they wrecked the codebase the more I delegated to them because I couldn’t muster the willpower to clean it up. It’s amazing what they ended up accomplishing, but that lack of consistency in their output is killer. You have to not care about the final result to really take advantage of LLMs at the moment, but I’m unable to do so.

What’s Next

I’m not going to stop using LLMs for this project, but I will start being a lot more intentional about their use. There’s no way I’m releasing a (partially) vibe-coded monstrosity into the world. I will use the LLMs to do experiments and to take care of annoying boilerplate, but I will take full ownership of the code, meaning the most I’ll allow an LLM to produce at a time will be a few hundred lines of code I can immediately review and fix.

My goal is to have the first public version of SUIT out by the end of the year. It’ll be open source, likely under the MIT license.

Side-note: The Windows Problem

Windows is a bit of a special case, as building things out of the classic Win32 common controls doesn’t really work very well, because they don’t compose. They also do not support any sort of alpha transparency. Maybe there’s a good way to make it work but I haven’t figured it out yet. Instead the current idea is to custom render with Direct2D, DirectWrite, and DirectComposition, and use an API called ITextServices to handle all the text editing. Context menus can be native Win32 since they are not part of the main window. Only scrolling viewports will unfortunately not be very native feeling, but not much I can do about that.

The alternative is to use the latest Microsoft recommended API, WinUI3, included as part of the WindowsAppSDK. Sadly, WinUI3 has the problem that it is not actually distributed with the OS, not in the same way Direct3D is. Rather, the user must install the appropriate version of the runtime.

There are three different ways to do this. The recommended way is to distribute the application as a .msix installer, which can then pull the correct 100+ MB version of the runtime from the Microsoft Store if it is not already installed. Think Flatpak runtimes on Linux for something somewhat equivalent. I might support this as an alternative Windows backend in the future if WinUI3 proves itself, but remember what I said about a graveyard of half-finished UI frameworks? Yeah…

The other two ways are:

1. Distribute the 100+ MB runtime installer alongside your application’s installer. Fine if your application is huge, not fine if your application is less than half a megabyte like the SUIT notepad demo. You have to install the correct minor version of the runtime, it is not backwards compatible.

2. Distribute just the necessary DLLs with the application. Sadly we’re talking more than a dozen DLLs here, totaling at a minimum around 30 MB in size.

Considering that SUIT can do what WinUI3 does in tiny fraction of the size, you really have to wonder what in the world WinUI3 is doing under the hood.

Conclusion

So that’s what I’ve been up to, not dead (yet), and I’ll hopefully get back to writing articles more regularly soon. Thank you all for not unsubscribing meanwhile, I really appreciate the support.

Small administrative note: I’d like to apologize for the long drought between posts. I’ve been working on an article related to UI, and in order to accurately discuss some topics I’ve spent quite a bit of time working on a prototype. The issue is that it appears I’ve bitten off more than I can chew, given my very limited free time. Hopefully I’ll be able to get that article out eventually, but it is not coming out any time soon, so I decided to make a smaller post in-between.

In this post I want to discuss some ideas I’ve had regarding software design. I’ll put a disclaimer right away that I have not put these ideas into practice in any serious capacity. These ideas are, however, based on other existing ideas that have been very successful in practice, but not necessarily described in these terms or placed in the same “conceptual bucket”, despite their commonalities.

The main idea I want to discuss is what I call Transaction-Oriented Programming (because I am horrible at naming things). Right away what probably comes to your mind are databases, and databases can be seen as an instance of the idea, but what I’m thinking of as a “transaction” is broader.

The idea of Transaction-Oriented Programming is to figure out “user-relevant units of work” (more on what those are below), and to implement those units as transactions in your application, following the usual ACID properties1.

My hypothesis is that structuring software this way leads to a pit of success. Figuring out these units plus the constraint of making them transactions provides clear and actionable guidelines regarding software architecture, error handling, etc., that I believe ultimately lead to better software.

User-Relevant Units of Work

What constitutes a User-Relevant Unit of Work is somewhat fuzzy (or at least fuzzy in my head), and I don’t have a clear definition for it, so instead I’ll try explain it with examples.2

What is a User-Relevant Unit of Work for the cd (change directory) command in the terminal? Well, it’s the whole command. There’s no subset of work the cd command does that is user-relevant. You either changed directories, or you didn’t3.

I believe this is why using abort() as an error handling solution on many batch applications doesn’t feel particularly wrong, not in the way it would for a more interactive application. There’s no point in propagating errors around nor retrying the operation nor anything of the sort if the only valid response is to terminate the application outright. Consider the following:

• If the error was reported by throwing an exception, then the exception would be caught, its message printed, and the application would exit with an error code.

• If an error code was propagated through a chain of function calls, then main would check the error code, map it to an error message, print it, and exit the application with an error code (Result<T,E> and friends are the same).

• If the error was placed in some internal error log, and then the rest of the program operated on a bunch of “null-objects” (as in Ryan Fleury’s error handling approach), then eventually main will check the error log, print out the message, and exit the application with an error code.

No matter which error handling strategy you use in this case, you arrive at the same destination, so the correct error handling solution is whichever is the simplest/most convenient/most efficient: in this case, calling abort.

In this example, the entire command forms a trivial transaction. It either does the task completely, or not at all, and since it does not do multiple tasks in parallel it is trivially isolated (durability is not relevant).

For other commands it’s less clear. What is a User-Relevant Unit of Work for the cp (copy) command? It’s actually not the whole command anymore, necessarily, because of directories. If you copy a directory, then each file being copied can be considered a user-relevant unit of work most of the time.

If an error occurs (e.g., one file could not be copied), then this should usually not lead to the complete command failing, it should just log the error and continue on with the larger task at hand. In this case, aborting the program outright on the first file that failed to copy would be a pretty bad error handling solution.

But what about partially copying a file? Partially copying an individual file is very rarely wanted (since it would end up corrupted), so copying part of a file is not normally a user-relevant unit of work. The copy of each file is a unit, and each unit is a transaction: either the file is copied completely, or not at all.

But writing part of a file can be a user-relevant unit of work when downloading a file via BitTorrent for example, so what constitutes a user-relevant unit of work varies depending on the context, but in all cases error handling happens around the unit, and each unit is handled as transaction.

Guidelines for Error Handling

Whichever error handling mechanism (exceptions, error codes, etc.) you use, they will all ultimately have to accomplish the same thing: abort the current transaction and inform the user. If the user is presented with an option when this happens, that’s a new transaction, the other one already failed.

So it doesn’t really matter which error handling mechanism you use, so much as how you use it. For some problems error codes will require you to propagate errors unnecessarily4, leading to needlessly noisy code, while for other problems aborting the program outright is clearly not a valid option. All mechanisms have to do the same job in the end, so pick whichever works best for the task.

You might think that exceptions are the perfect solution here, since you can put a “try-catch” block around the block of code that implements the complete “user-relevant unit of work”, but you’d be wrong. What makes you think there’s such an obvious block of code? The software might be implemented using a task queue and a thread pool, with multiple tasks running per “user-relevant unit of work”, which greatly complicates things.

You need to keep in mind that you are running a transaction, and you need a plan on how you’re going to ensure it is ACID.

Guidelines for Software Architecture

In the example above we already saw how these “units” provide some guidance on how to structure a software application in terms of error handling: When an error occurs, you need to abort the current transaction.

But that begs the question: how exactly does one abort a transaction correctly? As a reminder, these are the four ACID properties you need to ensure:

• Atomicity: How do you ensure the transaction either happens in its entirety, or fails in its entirety?

• Consistency: How do you ensure that there are no “leftovers” polluting the app state and its environment when the transaction is aborted?

• Isolation: How do you ensure that tasks can execute concurrently without interfering with each other?

• Durability: How do you make the results of the transaction persistent? This may or may not be relevant for the application, but is worth considering.

If you think about it, you need to ensure most of these properties for software to be reliable, specially consistency! If an error occurs and your software is now in a weird in-between state, that corruption will eventually lead to broken behavior and the user needing to restart the application.

So how do you ensure these properties? Well, there are many ways, and different software architectures can make it either very easy, or very hard.

Distributed software is obviously a nightmare in regard to the above, since you need a lot of additional coordination to ensure every process stays on the same page. But distributed software often simply stores the serious “app state” in a database anyway, and the database has those properties, so it works out in the end.

Object-oriented software (as in, software architected around communicating objects with isolated state), is just a local and synchronous form of distributed software. The same problem is there, but now there’s no database to fallback to, so it is a disaster in this regard. It’s well known in OOP circles that maintaining object invariants is extremely important (the private keyword is there for a reason), but a web of consistent objects is not necessarily consistent as a whole!

Smalltalk works around this problem by being really good at Durability. If you break the Smalltalk image, you can just revert to an older working version. If not for that, you’d be screwed, since you can very easily change the state of a large web of objects in a way that is essentially impossible to recover from.

Software with a “Functional Core, Imperative Shell” fares much better. If you think about it, the “functional core” is trivially a transaction! It only does compute, producing a command or list of commands that are then handled by the “shell”. If there’s an error, then there cannot have been any pollution of application state since it was never touched. While the shell itself might have issues performing the commands, the “surface area” where problems can occur has been greatly reduced.

Some very prominent software developers vouch for this software architecture, and this paradigm I’m proposing gives some answers as to why it works well.

The fact that the core is functional actually matters very little. It might have benefits for testability and such, but the fact that the app state remains consistent is the important part. Similar to how functions with local mutation are not really all that different from 100% pure functions, what matters is the referential transparency.

The same outcome could be achieved with a “double buffered” application state, where the code imperatively writes to the new state and it is only switched with the old state if no error occurs. Or by being able to “undo” the work you did up to the point the error occurred. They all implement the same thing: a transaction.

Immediate mode UIs, Reactive UIs, etc., are also all ways of reducing the chances of the application entering an inconsistent state. The more derived state, the less the app state can go haywire. But having the state be derived is just a mechanism. If you can ensure that every transaction always sets both the app and UI states, or neither if it fails, you end up in the same place! Derived UI state is just the easiest way to achieve the intended outcome.

Transaction-Oriented Programming

Paradigms aren’t nearly as important as the specific properties they embody that are beneficial in practice, so I don’t think a whole new paradigm matters5.

If you are doing “Functional Core, Imperative Shell”, you’re already reaping most of the benefits an imaginary “Transaction-Oriented Programming” paradigm would give you. Same is true if you’re using a database, since that’s already an implementation of the paradigm by nature.

But as a fun exercise, what would Transaction-Oriented Programming look like? And how would you implement a generic and composable “transaction” concept that programs can be built around?

I’m going to assume “just implement the relational model” isn’t a valid answer, that’s another more specific paradigm. Plus you can implement it in a way where the database state is consistent but the application overall isn’t (i.e., due to side-effects).

A Python Prototype

There are two ways we can implement transactions: either the transaction collects all the work to be done, and then commits it at the end if everything succeeded, or the transaction does the work right away, but then undoes it if there is an error. Depending on the task one or the other is better. We’ll implement the undo version but you could also have the two kinds co-existing:

class Transaction(ABC):
  def __init__(self):
    self._error: Exception | None = None
  
  def run(self):
    self._execute()
    if self._error:
      raise self._error

  def _execute(self) -> bool:
    try:
      self._task()
      return true
    except Exception as e:
      self._error = e
      self._undo()
      return false

  def _abort(self):
    if self._error is None:
      self.undo()
      self._error = AbortedTransactionError()

  @abstractmethod
  def _task(self):
    pass

  @abstractmethod
  def _undo(self):
    pass

For simplicity, we’ll assume transactions cannot be canceled midway, which would require using generators or async.

The only public method of a Transaction is run(), which executes the transaction to completion, possibly throwing an exception.

The run() method is, however, implemented using a separate _execute() method, which traps any exception that occurs. This split is what will allow us to compose transactions into larger ones that can be handled atomically.

Each transaction must implement two methods: _task(), which does the actual work, and _undo(), which reverses the work that was done up to that point.

The _abort() method is used by composite tasks to revert sub tasks that already finished successfully within the larger transaction.

We can make our first primitive transaction now:

class CopyFile(Transaction):
  def __init__(self, source: str, dest: str):
     super().__init__()
     self._source = source
     self._dest = dest

  def _task(self):
    shutil.copyfile(self._source, self._dest)

  def _undo(self):
    if self._source != self._dest:
      try:
        os.remove(self._dest)
      except:
        pass

t = CopyFile("hello.txt", "world.txt")
t.run()

Not particularly interesting, but this is our building block. Now let us create a composite transaction that executes its sub transactions sequentially, aborting if one of them fails:

class Sequence(Transaction):
  def __init__(self, children: *Transaction):
    super().__init__()
    self._children = children
    self._failed: int = -1

  def _task(self):
    for i, t in enumerate(self._children):
      if not t._execute():
        self._failed = i
        raise t._error
        
  def _undo(self):
    for i in range(0, self._failed + 1):
      self._children[i]._undo()

f1 = CopyFile("a.txt", "b.txt")
f2 = CopyFile("b.txt", "c.txt")
s = Sequence(f1, f2)
s.run()

We can also make a task that always succeeds, even if there is an error:

class Optional(Transaction):
  def __init__(self, sub: Transaction):
    super().__init__()
    self._sub = sub

  def _task(self):
    # ignore the error
    if not self._sub._execute():
      self._sub._undo()
  
  def _undo(self):
    if self._sub._error is None:
      self._sub._undo()

We could then use these building blocks to implement more complex transactions, for example, a transaction that recursively copies files from a directory to another.

Transaction-Oriented Language

The above would let you create and manipulate transactions in an existing programming language, but what would a transaction-oriented language look like? I’m honestly not sure, but maybe something like the following:

transaction copy_folder(source, dest):
  mkdir(dest)
  for p in dir(source):
    if is_directory(p):
      copy_folder(join(source, p), join(dest, p))
    else:
      copy_file(join(source, p), join(dest, p))  

transaction copy_file(source, dest):
  f1 = open(source, "r")
  f2 = open(dest, "w")
  write(f2, read(f1))

Each operation is a made up of smaller transactions. If any fails, undo is called accordingly, a bit like exceptions + RAII. The main difference is that operations that succeeded are also undone! Either the entire transactions happens, or nothing does.

Primitive transactions would define different sub-operations:

foreign transaction mkdir(directory):
  on execute:
    native_mkdir(directory)
  on abort:  
    native_rmdir(directory)

You could also support a different variant of transaction that only commits its work when everything succeeds, might be important for printing which cannot be undone:

foreign transaction print(text):
  on execute:
     pass
  on commit:
    native_print(text)

In this case, every committed operation would be placed in a FIFO queue to be executed when the parent transaction terminates. Two operators could be added to support “partial success”: the optional operator (?) and the commit operator (!).

# even if this transaction fails the parent can continue
my_transaction()?
# do not undo this transaction even if the parent fails
my_transaction()!

Compiling transactions like this into efficient code is somewhat tricky since being able to call transactions in a loop requires keeping track of which particular transactions in the loop succeeded or not, such that they can be undone.

That will usually mean heap allocation for an undo stack (a bit like how defer works in the Go language) and/or a commit queue, but one could use a temp Arena just for that purpose, which would be reasonably efficient.

Conclusion

Transactions are good. Ted Codd was a genius and we’ve all suffered massively for not listening to him and building software using the relational model, instead chasing silly ideas like object-oriented programming and garbage like SQL6.

Reliable software implements transactions, whether the developers realize it or not. Software architectures can be evaluated in terms of how easy or how hard they make it to implement transactions. Correctness of error handling can be rephrased in terms of implementing transactions correctly.

We might not need an actual “Transaction-Oriented Programming” paradigm, just as most of the benefits of functional programming can be achieved in a procedural language with local mutation and an avoidance of side-effects, but thinking in the “native language” of a paradigm provides clarity.

Later I played around with FrontPage and Dreamweaver, which enabled me to design websites visually. Very similar in spirit to Delphi, only targeting HTML instead. Events could be mapped to Javascript functions, which had to be written as monospaced ASCII text.

Next came Game Maker, the pre-Studio version developed by Mark Overmars. It went even further on the visual aspects. It included an image editor for sprites and backgrounds, a tilemap editor, and a visual editor for “objects”, which included a drag-and-drop interface for programming the object’s behavior1. Actual visual coding! But the only way to have any sort of abstraction was to create a “Script” and write monospaced ASCII text (which sadly didn’t map 1-to-1 to the drag-and-drop interface).

From what I can tell, over the years we’ve only moved backwards towards monospaced ASCII text. Even things that should be visually edited like GUIs are now mostly written as monospaced text, maybe with “live preview” to help out. Computers have made absolutely incomprehensible leaps in performance and capability since the 1940s, but we still program them like we did in the 1980s! Even back in the day all that visual editors were really doing was write monospaced ASCII text under the covers.

The standard interface to most programming language compilers is an emulator of a device from the late 1800s. The most popular editor these days, Visual Studio Code, is downright alergic to toolbars. We use tools whose only job is to move ASCII characters around to avoid bike-shedding about where to place them on a grid, even though the specific positioning of a curly bracket matters very little as long as indentation is done properly. Our IDEs have to repeatedly parse monospaced ASCII text to develop an index with all the types and functions in the project, something the compiler also has to do all over again when invoked. Its silly.

How did we end up here?

Asking a Robot to Bang Rocks

As a little side-note, I find it absolutely hilarious that the current “state-of-the-art” in software development (according to some people at least) is to chat with the closest thing to artificial intelligence we’ve built (so far), and to ask it to spit out monospace ASCII text and run commands in a teletype emulator. Regardless of how effective they are at the task, I find the whole setup hilarious, like a caveman asking a robot to bang rocks in order to start a fire. LLMs are only going to exacerbate this local optimum we’re currently stuck in.

The Advantages of ASCII Text

In order to understand why Visual Programming doesn’t seem to catch on outside of some very specific niches, we have to first understand why ASCII text continues (and likely will continue) to be the dominant programmer-machine interface.

1. The platform APIs are provided (ultimately) as ASCII text. If I want to program an application targeting any of the current mainstream platforms, an API described in a language whose interface is ASCII text is going to be involved at some point. When the limitations of the visual interface are hit, and the programmer has to “drop down” to ASCII text, it’s almost always very painful.

2. The languages that we use do not lend themselves well to visual programming (see: modern Game Maker’s awful node-based UI). Working with structured programming in the form of a flow-chart or node-graph results in a massive loss of information density and development speed.

3. Directly manipulating the AST of a programming language, as in Jetbrains MPS, is worse in nearly every way to working with ASCII text. All the disadvantages with almost no advantages. The loose nature of ASCII text is a feature, not a bug. The ability to copy-paste and tweak some characters or “find-and-replace” a bunch of times to refactor some piece of code (with “broken” intermediate steps), arrives at the destination much faster than having to follow a limited set of always valid but otherwise rigid transformations.

4. We have lots of tools that work with ASCII text, most available for free or very cheaply. It is a lot easier to develop tools that work with ASCII text than fancy visual editors.

5. Most Visual Programming tools that do exist are designed as low-barrier-to-entry solutions for non-programmers, rather than tools meant to increase the productivity of expert programmers. The so called “no-code” tools targeted at large businesses are all vendor-lock-in scams (IMO).

It also doesn’t help that programmers have been burned very badly with the one major industry attempt at professional “visual programming” (or architecting rather): UML. UML class diagrams map very poorly to actual code, something everyone that had to deal with UML learnt very quickly and very painfully. Harel Statecharts (also part of UML) are actually pretty great, but they weren’t the focus in development circles2.

If programming is to move away from ASCII text, then Visual Programmings needs to shift focus from being low-barrier-to-entry (even if that’s a nice property to have), to being a productivity boost for experts. It must surpass, in terms of development speed, all the shortcuts ASCII text allows, which these days includes having a robot do some of the work for you.

Where Visual Programming Succeeded

While most mainstream software development activities have completely abandoned visual programming, including visually editing of parts of the program (like the UI) or even having a freaking build button in the IDE, that’s not true of every domain.

Tools like Construct 3 and Game Maker are pretty popular despite their UX issues3. Godot and Unity don’t have official visual programming toolkits, but they are otherwise highly visual. Unreal Engine has Blueprints which is a low-barrier-to-entry for non-programmers (e.g., level designers) solution. RPG Maker, as long as you’re making a bog-standard turn-based 2D JRPG, requires no “programming” in the usual sense. I also have a soft spot for the extremely powerful Warcraft 3 level editor4.

So at least for game development some degree of Visual Programming is still around. But it is very high level and with very poor abstraction capabilities. It’s not really a serious visual alternative to programming in monospaced ASCII text.

Non-software engineering fares better. Tools like Simulink and Amesim are almost entirely geared towards visual modeling. Simulink has MATLAB under the hood but Amesim doesn’t have any actual textual language as a base. The Modelica language has a textual representation, but the tools all provide graphical modeling.

These tools, unlike the game development ones, have proper visual-abstraction capabilities, and can be extremely expensive (some 10s of thousands of dollars).

Why have these tools succeeded where others have failed? I believe the main reason is what I like to call immediacy. It’s how close the level you’re working at is to the level you’re thinking in, how quickly you can get feedback, and how easily you can reflect it back onto the code.

This is related to, but not equivalent to, low-barrier-to-entry. Software like Simulink and Amesim is not low-barrier-to-entry, these tools need extensive training to use effectively, and their core audience is made up of experienced professionals.

But they have very high immediacy. I can drag and drop a variable from a port of a block in an Amesim model and instantly get a plot showing how its value evolved over time in a simulation. There is an absolutely monstrous library of ready to use components (6500+), at different levels of abstraction, with familiar looking icons, that can be used to build more complex systems. I don’t mean packages that can be installed, they’re all part of the “standard library”. It also includes many domain-specific UIs for specific problem domains with custom visualizations.

Some customers report an 80% reduction in development time. That’s easily worth the 10s of thousands of dollars a single seat license costs. Is there any visual programming platform for software development that can make a similar claim? Yes, RPG Maker, if you’re making a turn-based 2D JRPG and don’t mind its defaults.

Not Drawing, Seeing!

For programming to escape its monospaced ASCII shackles, focus must be placed on what actually helps, and it’s not drawing lines and rectangles. We’ll talk about node-graphs in a bit because they enable certain things, but rectangles and polylines are not what changes the game.

The really helpful part of Visual Programming is not the “drawing”, it’s the “seeing”5. Even in the monospaced ASCII world we make use of visualization: Indentation and syntax highlighting are visual aids to help our brains map ASCII text to a mental AST more quickly. Debuggers let us see how variables change over time as we step through the code. In VSCode I can hover over, e.g., a Rust identifier and get useful information like its documentation, type, size, etc. or jump to its declaration and implementations.

These are all helpful forms of visualization, but also very limited. We can (and should) take this further, much further.

Visualized Programming

One of my favorite things awhile back was the Elm Reactor. It was a time traveling debugger that took advantage of Elm’s event driven architecture and functional purity to allow “rolling back time” (with a literal slider) on a running application, unlike post-mortem time traveling debuggers like rr. After rolling back time you could continue to interact with the application and “change the future”. You could even change the code to some extent and keep going! It was really incredible to me.

Creating something like the Elm Reactor is only possible if the architecture of the application allows for it. Elm imposed an architecture, and that made the Reactor work for every application developed in Elm (until they broke it but that’s another story). General purpose time traveling debuggers cannot compete with something like Elm Reactor in performance or UX, because they have to support all sorts of code.

Architecture and semantics have many second order effects, and that is also where “nodes” come in. But representing a function’s body as a flowchart is almost completely useless6. Most of the time a function’s control flow is pretty obvious, and your brain can already “see” everything it needs just fine. A drawing is only useful if it helps you see something that you otherwise couldn’t, or at least not as quickly.

At the level of an individual function, ASCII text is a very good interface. Smalltalk understands this for example. You’re editing a live program when you program in Smalltalk: there are no source files, there is no ASCII text. You directly manipulate classes and methods that exist in memory as bytecode, but they’re presented to you, and edited, as monospaced ASCII text. At that level text works just fine.

In Enso it’s possible to visualize the data flowing through individual nodes as it goes through them and changes over time. It has a textual language that is isomorphic (except for node positioning) to the visual programming side. You can edit the code either visually or as text, it is the same language. Text and graphics are not at odds.

Game Maker could have worked like that too, drag-and-drop and text being two ways to edit the same language, specially in the older list-oriented design. But in Game Maker’s case it would mostly be helpful to newbies. The drag-and-drop UI provides no advantages to an expert. Enso does, because of the visualization. This is what I want you to keep in mind: seeing, not drawing.

The Power of Nodes

Where nodes provide value is at a higher level. In order to provide value they must let you “see” something you otherwise would struggle to know, specifically the application’s inner workings. I don’t mean visualizing a class or package diagram, that’s not really helpful unless you’re completely unfamiliar with the codebase.

I mean the drawings must reflect the actual logic of the application, they must help you see what it does. For example, a Harel Statechart (a hierarchical form of State Machine) lets you make application states and their transitions explicit, and to see the whole thing in a single screenful. You can then select a particular state and “dig down” to see the internal states it can be in, and so on and so forth.

Harel Statecharts also allow for concurrent state machines, which can send events to each other, so you can program complex concurrent applications with them. If you’ve already been converted to the Church of Coroutines, it’s that on steroids.

But I’m not suggesting we should all be programming in hierarchical state machines. There will be many applications where that is not a very natural way of working. The point is thinking in terms of “seeing”. What helps programmers “see” what they need to see? The UX to edit the application’s logic should reflect this, such that mapping the results of what was “seen” back to the code is as easy as possible.

In a game engine, that could be a live editor experience, where you can pause the game, click on an enemy character, and then tweak its AI script. Immediacy! The script could be a state machine, a flow chart, an event list, or plain ASCII text. The choice is nearly irrelevant in the face of the live editor experience.

Thinking in those terms, how to maximize immediacy, is what matters. How quickly can the programmer find out what they need to find out? How quickly can they map their findings back to the code? Optimizing for that is what Visual Programming should be about.

Even if a programmer avoided one or more of those three paradigms like the plague, they’re likely at least aware of them and what they’re about. Or they’re applying one of the paradigms while denying that they’re doing so, like Haskell programmers using the IO or State Monads (procedural programming), or C programmers writing structs of function pointers (object-oriented programming), or Java programmers using streams (functional programming).

The same is sadly not true of logic programming1. While some programmers are aware of its existence, and might have experienced a little bit of it in university, it’s not even close to the popularity of the other paradigms. I’d go so far as to say that the majority of programmers have no idea what it’s about, and that’s a shame because logic programming is really good at tackling certain kinds of problems.

OOP and FP are easy to explain in terms of procedural programming concepts, and it’s also pretty easy to explain how to implement them. That’s not really the case for logic programming, but when has that ever stopped me?

What better way to learn something than to implement it?

Why Logic Programming?

If you’ve ever lost your marbles trying to model complex relationships between various concepts as objects with bi-directional pointers to each other and derived properties that need to be cached and all that jazz, then that’s a great example of a problem where you should have used logic programming instead (looking at you OMG and your fancy SysML v2 standard2).

For a rather big list of examples of what you can do with logic programming, check out this blog post from Phillip Zucker.

In logic programming we don’t program with functions as we do in the other paradigms (procedures and methods are also functions). Functions have a set of inputs and a set of outputs, and mutable inputs can be viewed as just another kind of output.

Rather, in logic programming we program with relations. They’re also called predicates in logic programming, but they’re the same thing really (much like procedures and methods are kinds of functions).

The difference between a relation and a function is that a relation doesn’t have a clear distinction between what is an input and what is an output.

I’ll use what is probably the most well known logic programming language, Prolog, to illustrate. Let’s start with a simple example:

male(dicky).
male(randy).
male(mike).
male(don).
male(elmer).

female(anne).
female(rosie).
female(esther).
female(mildred).
female(blair).

In the example above, male and female are what are known as predicates in Prolog. Predicates are defined in terms of clauses, and a clause can be either a rule or a fact. A fact is just a rule that is always true. Every “statement” in the example above is a fact.

By writing male(randy) on its own as above, we’re saying that “it is a fact that randy is a male”. This being an is-a relationship is just our own interpretation of what male(_) means. All Prolog cares about is that there’s a fact for predicate male that states male(randy). A predicate/relation means whatever we want it to mean.

The various names of people above are atoms in Prolog. They’re basically interned strings (same as symbols in Lisp, Ruby, Julia, Smalltalk, etc.), but atoms can also be integers and even complex structures. Prolog is dynamically typed so we don’t need to declare any types, but that’s not the case for logic programming in general3.

All we’ve done so far is state that a bunch of people are either male or female. Not very interesting, but we’ll make use of this information soon. Let’s move on to a more interesting set of facts:

parent(don, randy).
parent(don, mike).
parent(don, anne).

parent(rosie, randy).
parent(rosie, mike).
parent(rosie, anne).

parent(elmer, don).
parent(mildred, don).

parent(esther, rosie).
parent(esther, dicky).

In the example above we’ve added a new predicate (parent) and some associated facts, this time relating two people. When we write parent(don, randy), we’re saying that Don is one of Randy’s parents. Relations don’t really have an ordering, this is just how we’ve chosen to interpret each of the two arguments.

The Power of Rules

So far, nothing special, but now we can write our first rule:

father(X, Y) :- male(X), parent(X, Y).

The rule above says that for any given X and Y (uppercase letters are variables), if X is a male, and X is a parent of Y, then X is the father of Y4. The weird :- symbol is a reverse implication, while the comma means “and”. In Prolog “or” would be a semicolon, or just another rule for the same predicate. You can read the example as:

father(X, Y) if male(X) and parent(X, Y)

Anyway, we can now start doing some queries (?- is the query operator)5, for example:

?- father(X, randy).

This will return X=don. If instead we query:

?- father(don, X).

We get X=randy, X=mike, and X=anne.

As I mentioned previously, rules don’t have any clear distinction between inputs and outputs. What constitutes an output depends on the query. I can write the query:

?- father(X, Y).

And get every possible pair of father and child in the Prolog interpreter’s “database of facts” (so to speak). The word database is quite relevant actually, since as the name relation implies, logic programming is closer to relational programming (e.g., SQL) than it is to the other paradigms.

But logic programming can be a lot more powerful (and succinct) than crappy SQL. For example, we can keep adding more interesting rules:

son(X, Y) :- male(X), parent(Y, X).
daughter(X, Y) :- female(X), parent(Y, X).

sister(X, Y) :- daughter(X, P), parent(P, Y), X \= Y.
brother(X, Y) :- son(X, P), parent(P, Y), X \= Y.

aunt(X, Y) :- sister(X, P), parent(P, Y).
uncle(X, Y) :- brother(X, P), parent(P, Y).

These should hopefully be pretty obvious. Notice how we can introduce new variables in the body of the rules, like P above. Prolog uses unification to link variables together, the same process used by type inference.

The real power comes from recursion:

ancestor(X,Y) :- parent(X,Y).
ancestor(X,Y) :- parent(X,Z), ancestor(Z,Y).

The above is similar to how you do pattern matching in Haskell, in case you’re familiar with that. We can also, for example, add some special cases inspired by the creation myth of Abrahamic religions:

ancestor(adam, X) :- male(X); female(X).
ancestor(eve, X) :- male(X); female(X).

If we declare adam as male, then we end up with adam as being an ancestor of adam, which is pretty funny (but also pretty easy to fix).

Neat, how do I implement it?

Well, I could go over how to implement a Prolog interpreter, but honestly I don’t think you should. Prolog is honestly kind of jank, and I’m not talking about good vintage jank like C.

The main issue is that Prolog is not truly declarative; how you write the rules has a major impact on how the interpreter behaves. You might end up with repeated answers for a query or it might enter an infinite loop. Prolog also allows IO, so the order in which things get executed is critical. It’s Turing-complete, for better or worse.

If you really want to implement Prolog specifically, you have to follow its execution semantics, which are based on SLD resolution using depth-first search and backtracking. Prolog programs depend on the specific order in which rules get executed, so you have to perform the search the same way as all the other Prolog implementations do. Check out Warren’s Abstract Machine on how to do it efficiently (also a good source for implementing unification).

But we don’t need Turing-complete Prolog. We already have whatever Turing-complete, multi-paradigm programming language we’re using on a daily basis, we just need to power it up with logic programming.

Most people would do this by implementing miniKanren, another logic programming language that is specifically designed to be embedded into a host language, originally Scheme. A later developed simplified core, microKanren, is so small it can be implemented in 39 lines of Scheme code without any macros.

But I am not a big fan of the miniKanren family. Its design is very functional, which has its advantages, but to me the “database” aspect is important. Unlike Prolog, where facts can be added and removed while the program is running, in miniKanren you set up the universe of facts you care about each time you make a query.

This results in a very clean implementation6, but it leaves a lot of performance on the table. To me, maintaining a stateful database of facts is a core part of the job. That’s what we’re really doing: implementing a fancy database. You can of course bolt on a stateful fact database to a miniKanren implementation, but it’s not how miniKanren is meant to work.

Instead, we’ll turn our attention to Datalog, a subset of prolog that is not Turing-complete. You can’t use Datalog to develop complete applications, but it sure is great at modeling relationships. In fact, I wish it would replace SQL as the language of choice for databases7. SQL isn’t even a good relational language, and logic programming is just on another level entirely8.

Since Datalog is a subset of Prolog, we could just implement the same algorithm we’d use for a Prolog interpreter and it would work, but the lack of Turing-completeness gives us a lot of room for maneuver.

For one, techniques from databases are applicable: B-trees, query optimization, index selection, etc. Datalog is also very amenable to partial evaluation, as employed by the highly optimized Soufflé dialect.

Here we’ll keep it simple and implement what I think is the most basic algorithm for interpreting Datalog: Naïve Evaluation. It’s a simple bottom-up fixpoint algorithm that repeatedly applies each rule until no new facts can be derived. We’ll also abuse operator overloading to make the code look sort of like Datalog.

Modeling Datalog

We’ll use Python to keep things as simple as possible. The first thing we need is some way to represent variables, values and usages of a predicate (that we’ll call atoms):

class Variable:
    def __init__(self, name: str):
        self.name = name
    
    def __repr__(self) -> str:
        return self.name

Value = bool | int | float | str
Term = Variable | Value

class Atom:
    def __init__(self, predicate: str, terms: Tuple[Term, ...]) -> None:
        self.predicate = predicate
        self.terms = terms

This Atom class is just used to model a predicate usage like father(X, bill) and such, either with a variable or a value in each argument. Not sure what else to call it. Note that this is different from the meaning of an atom in Prolog.

Also notice that we cannot pass predicates as arguments to other predicates. This is allowed in Prolog but not in Datalog. Datalog has some restrictions to ensure it always terminates (they can be loosened, but we’ll stick to the basics):

• negation is not allowed.

• complex terms as arguments of predicates, e.g., p(f(x), y), are not allowed.

• every variable that appears in the head of a clause must also appear in an atom within the body of the clause.

For values we only allow a few builtin Python types to keep things simple. With variables, values and atoms in place, we move on to predicates:

Fact = Tuple[Value, ...]

class Rule:
    def __init__(self, head: Atom, body: Tuple[Atom, ...]):
        assert len(body) >= 1
        self.head = head
        self.body = body

class Predicate:
    def __init__(self, name: str, arity: int):
        self.name = name
        self.arity = arity
        self.facts: Set[Fact] = set()
        self.rules: List[Rule] = []

We split clauses directly into separate facts and rules. A fact is just a row of values belonging to a predicate. A rule has a head (left side of the :- operator), which is just a single atom, and a body with one or more atoms (right side of the :- operator). Arity is the expected number of arguments.

To manage our “database” we’ll create a class simply called Datalog:

class Datalog:
    def __init__(self) -> None:
        self.variables: Dict[str, Variable] = {}
        self.predicates: Dict[str, Predicate] = {}

    def variable(self, name: str) -> Variable:
        assert name not in self.variables
        v = Variable(name)
        self.variables[name] = v
        return v

    def predicate(self, name: str, arity: int) -> Predicate:
        assert name not in self.predicates
        c = Predicate(name, arity)
        self.predicates[name] = c
        return c

We also store the variables here to make the API a bit nicer and to allow us to safely use reference equality later on for better performance. We could do the same with string values (turning them into symbols) but I leave that as an exercise for the reader.

Next, we need a way to create atoms and to add facts and rules to a predicate. We’ll add a bit of nasty operator overloading to the Predicate class to achieve this:

def __getitem__(self, terms: Term | Tuple[Term, ...]) -> Atom:
    # make sure we always work with a tuple
    terms = terms if isinstance(terms, tuple) else (terms,)
    if len(terms) != self.arity:
        raise ValueError()
    return Atom(self.name, terms)

def __setitem__(self, 
    terms: Term | Tuple[Term, ...], 
    rhs: Atom | Tuple[Atom, ...]) -> None:
    # make sure we always work with a tuple
    terms = terms if isinstance(terms, tuple) else (terms,)
    # if the rhs is the empty tuple, we're adding a fact
    if rhs == ():
        # NOTE: facts cannot contain variables, add a check!
        self.facts.add(cast(Tuple[Value, ...], terms))
    elif isinstance(rhs, tuple):
        self.rules.append(Rule(Atom(self.name, terms), rhs))
    else:
        self.rules.append(Rule(Atom(self.name, terms), (rhs,)))

What we’ve done above allows us to write the following:

dl = Datalog()

parent = dl.predicate('parent', 2)
ancestor = dl.predicate('ancestor', 2)

X, Y, Z = dl.variable('X'), dl.variable('Y'), dl.variable('Z')

parent['alice', 'bob'] = ()
parent['bob', 'carol'] = ()
ancestor[X, Y] = parent[X, Y]
ancestor[X, Y] = parent[X, Z], ancestor[Z, Y]

We have square brackets instead of round, = instead of :-, and need to write = () to declare a fact, but it sure looks a lot like Datalog. Even the = () is actually correct because pred(foo, bar). in Prolog and Datalog is just syntax sugar for:

pred(foo, bar) :- .

So all we’re missing is that little bit of syntax sugar.

Interpreting Datalog

We can model Datalog programs now, but we cannot perform any queries nor infer any new information. First thing we’ll need is an extra datastructure:

Substitution = dict[Variable, Value]

Pretty simple, just a mapping of variables to values. We’ll make heavy use of this datastructure soon, but first, the most important method — infer:

def infer(self) -> None:
    while True:
        newly_added_facts: List[Tuple[Predicate, Fact]] = []
        for predicate in self.clauses.values():
            for rule in predicate.rules:
                for sub in self.evaluate(rule.body):
                    fact = tuple(
                        sub[t] if isinstance(t, Variable) 
                        else t for t in rule.head.terms)
                    if fact not in predicate.facts:
                        newly_added_facts.append((predicate, fact))
        if not newly_added_facts:
            break
        for p, f in newly_added_facts:
            p.facts.add(f)

Method infer is the core of our Datalog engine. It implements the fixpoint algorithm that expands rules into new facts. You should call this method each time you manually add a new set of facts and/or rules.

Every loop iteration, for each rule, we call another method called evaluate, which lazily outputs all possible substitutions given the body of the rule. Each substitution is then used to create a new fact by replacing every variable with the associated value in the head of the rule (keeping any values already in the head).

If that fact was not already in the list of known facts, then we’ve derived a new fact. Once we’ve gone over every rule, if we found any new facts, we update our fact database and iterate again. If no new facts were derived, we’re done.

Method evaluate is just a wrapper around another method called search:

def evaluate(self, atoms: Sequence[Atom]) -> Iterable[Substitution]:
    return self._search(0, atoms, {})

def search(self, i: int, atoms: Sequence[Atom], 
            sub: Substitution) -> Iterable[Substitution]:
    if i == len(atoms):
        yield sub
        return
    atom = atoms[i]
    for fact in self.clauses[atom.predicate].facts:
        new_sub = sub.copy()
        if unify(atom, fact, new_sub):
           yield from self._search(i + 1, atoms, new_sub)

Method search implements a lazy depth-first search. Its goal is to successfully unify every atom in the body of a rule. To do this, it picks the atom at index ‘i’, and for every fact associated with the corresponding predicate, it tries to unify the atom with the fact. If unification succeeds (meaning we’ve obtained a substitution for every variable in the atom), we recursively call search again, only this time starting at the next index and enforcing the current substitution.

As an example of how it works, consider this rule:

ancestor[X, Y] = parent[X, Z], ancestor[Z, Y]

The first atom is parent[X, Z]. These are the known facts of parent:

parent['bob', 'carol']
parent['alice', 'bob']

We start with the first fact. We unify X=’bob’ and Z=’carol’. We then call search recursively, moving to the next atom, ancestor[Z, Y], with that substitution in hand.

From the other rule of ancestor:

ancestor[X, Y] = parent[X, Y]

We probably already derived that every parent is an ancestor, so the current facts of ancestor are also:

ancestor['bob', 'carol']
ancestor['alice', 'bob']

We try to unify with the first fact and fail, because the substitution enforces Z=’carol’ which cannot unify with ‘bob’. The second fact also fails to unify, so this is a dead end.

That branch of the recursion dies and we’re back trying to unify the first atom. Now we unify with the second fact, obtaining the substitution X=’alice’ and Z=’bob’. We recurse again with that substitution.

We try to unify ancestor[Z, Y] with its first fact, and we succeed, because we can set Z=’bob’ and Y=’carol’. We managed to do a complete substitution, so we yield it.

The function unify, used within search, is pretty simple:

def unify(atom: Atom, fact: Fact, substitution: Substitution) -> bool:
    for t, v in zip(atom.terms, fact):
        if isinstance(t, Variable):
            if t in substitution and substitution[t] != v:
                return False
            substitution[t] = v
        elif t != v:
            return False
    return True

It takes in an atom, a fact, and a substitution, and pairs up each term in the atom with the corresponding value in the fact at the same position. If the term is a variable, we do the following:

• If the variable is in the substitution, then we check if the value associated with it and the value in the fact match. If they don’t, unification fails.

• If the variable isn’t in the substitution, then we update the substitution by mapping the variable to the value in the fact.

If instead of a variable we see a value, then we just compare that value with the value in the fact directly.

To finish up our little interpreter, we only need one more method, and it’s a trivial one:

def query(self, *atoms: Atom) -> Iterable[Substitution]:
    return self.evaluate(atoms)

Yup, query is just evaluate, but variadic to make the API a bit nicer. With that, we can finally write our tiny Datalog program:

dl = Datalog()

parent = dl.predicate('parent', 2)
ancestor = dl.predicate('ancestor', 2)

X, Y, Z = dl.variable('X'), dl.variable('Y'), dl.variable('Z')

parent['alice', 'bob'] = ()
parent['bob', 'carol'] = ()
ancestor[X, Y] = parent[X, Y]
ancestor[X, Y] = parent[X, Z], ancestor[Z, Y]

dl.infer()

for result in dl.query(ancestor[X, 'carol']):
    print(result)

Which outputs (in any order):

{X: 'alice'}
{X: 'bob'}

Conclusion

Well that ended up a lot longer than I expected, but the actual implementation is pretty small all things considered. It’s not efficient, but some small optimizations go a long way, for example switching to semi-naïve evaluation.

The main difference is that instead of always going over every fact in each iteration of the loop, we only apply each rule to facts we derived in the previous iteration.

Another optimization is dynamically sorting the atoms in the body of a rule based on the number of values each atom contains and the number of associated facts, to try and cause search branches to fail as soon as possible.

We could also add support for arithmetic and composite atoms (like lists), which introduce some challenges if we wish to stay “Turing-incomplete”.

Either way, you now have a new tool in your arsenal. No more horrible object graphs desperately trying and failing to model relations, you can now simply use the best paradigm for the job.

Update: This article was discussed on hackernews.

• Tracing Garbage Collection (GC for short)

• Reference Counting (RC for short)

• Linear/Affine Types (RAII + std::unique_ptr is similar)

• “Borrow Checking” (i.e., Static Lifetime Analysis)1

Each solution has different tradeoffs: GC requires a hefty and complex runtime to be both global and efficient; RC can’t handle cycles and has some performance overhead to track the reference counts; Linear/Affine Types/RAII impose a tree-like structure to memory; Borrow Checking is insufficient by itself and imposes quite a few constraints that can be hard to understand (so called “fighting the borrow checker”)

But there is actually one more approach2 that could be used: Pointer Tagging. Often discussed in the context of hardware solutions like CHERI, pointer tagging can also be implemented in software with a reasonable performance cost.

The basic idea is to add some extra tag bits to a pointer such that we can determine if the memory address it points to is still valid. If the tag bits in the pointer and the tag bits of the allocation don’t match, then the data at that memory address has been freed and overwritten, and we have therefore caught a Use-After-Free.

But how do we produce and manage these tag bits? Read on.

Generational Handles

The idea of pointer tagging is similar to a common approach to track slot reuse in object pools: generational handles.

When you add an object to a pool, you get back a handle: an index to the slot where that object is stored. You can use this handle to grab the corresponding object and work with it. But what happens if the object has been freed and a new object was added to the same slot in the pool? How do you know if the handle remains valid?

The simple solution is to track the “generation” of the object in the slot. Whenever an object in a slot is deleted, that slot’s generation counter is incremented, so the next object placed at that slot has a higher generation count.

Instead of returning just an index as a handle, the pool returns an index + generation pair. When accessing a slot using a handle, its generation value is compared to the generation value of the slot. If they don’t match, the handle is no longer valid.

If generation counts are allowed to wrap around, then “generation collisions” become possible, but such collisions are extremely unlikely given enough bits for the generation counter.

To give some perspective, if you do nothing but increment a 64 bit number every clock cycle on a 3 GHz CPU, it will take 97 years to overflow.

Generational References

Generational References, a term coined by the developer of the Vale programming language, extend the idea of Generational Handles to arbitrary pointers. They’re a form of pointer tagging inspired by generational handles, hence the name.

An implementation similar to the approach used for generational handles requires a custom memory allocator since we need to manage the generation counts separately from the allocations. But there’s another really clever solution that was implemented in Vale that avoids this issue, remaining compatible with malloc, and it is the one we’ll use: random tags.

tag_ptr<T>

We’ll implement Tagged Pointers / Generational References in C++ using a smart pointer class we’ll call tag_ptr. These smart pointers don’t actually do any memory management themselves, like std::unique_ptr or std::shared_ptr, they only provide memory safety.

The idea is as follows: A simple thread-local3 random number generator is used to produce a stream of 64 bit numbers to use as tags.

A function make_tagged<T> allocates space for a T plus an 8 byte header. A random number is generated and placed in the header as the tag. It then returns a tag_ptr<T> which stores a pair of a raw pointer to the allocated T and the tag.

When tag_ptr<T> is dereferenced it compares its tag with the tag in the allocation. One of 4 things can happen:

• The memory was freed and returned to the OS, causing a segmentation fault.

• The tags match and the pointer is valid.

• The tags don’t match, so we caught a Use-After-Free and abort the program.

• The tags match but this is actually a collision. For whatever reason the exact same value as the tag was written where the old header should be.

The last situation is extremely unlikely. There’s around a 1/(2^64) chance of collision which is an absurdly small number (around 0.000000000000000005%).

C++ Implementation

First, we need the random number generator. We’ll use xorshift* with 64 bits of state and 64 bits of output because it produces values that are well distributed, it is very efficient, and it never outputs 0.4

That last bit, never outputting 0, is typically a downside for a random number generator, but it is good for us, since 0 is by far the most common value for memory to have and we can clear a tag in an allocation header by simply setting it to 0.

Here is xorshift* (adapted from Wikipedia)5:

#include <cstdint>

using namespace std;

uint64_t generate_random_tag() {
    thread_local static uint64_t x = 1;
    x ^= x >> 12;
    x ^= x << 25;
    x ^= x >> 27;
    return x * 0x2545F4914F6CDD1DULL;
}

To tag data we’ll use the following struct:

template<class T>
struct tagged {
    uint64_t tag;
    T data;

    template<class... Args>
    tagged(Args&&... args)
    : tag(generate_random_tag())
    , data(std::forward<Args>(args)...) {}

    ~tagged() {
        tag = 0;
    }
};

This struct forwards arguments to the constructor of whatever type it is tagging and generates and stores a random tag. All the destructor does is zero out the tag.

This struct could be used directly (to enable tagging of stack allocated objects for example) but it’s mainly meant to be used by the following function:

template<class T, class... Args>
tag_ptr<T> make_tagged(Args&&... args) {
    tagged<T>* alloc = new tagged<T>(std::forward<Args>(args)...);
    return tag_ptr(*alloc);
}

This function is analogous to std::make_unique/std::make_shared, in that it allocates the data on the heap and returns it wrapped in a smart pointer. In this case a tag_ptr, our tagged pointer / generational reference implementation:

#include <cstdlib>

template<class T>
class tag_ptr {
  T* ptr;
  uint64_t tag;

  bool is_valid() {
    char* raw_data = reinterpret_cast<char*>(ptr) - sizeof(uint64_t);
    uint64_t source_tag = *reinterpret_cast<uint64_t*>(raw_data);
    return tag == source_tag
  }
public:
  tag_ptr() = default;
  explicit tag_ptr(tagged<T>& t)
  : ptr(&t.data), tag(t.tag) {}

  T& operator*() {
    if (!is_valid()) {
      abort();
    }
    return *ptr;
  }

  void destroy() {
    if (!ptr) {
      return;
    }
    if (!is_valid()) {
      abort();
    }
    char* raw_data = reinterpret_cast<char*>(ptr) - sizeof(uint64_t);
    tagged<T>* tagged_data = reinterpret_cast<tagged<T>*>(raw_data);
    delete tagged_data;
  }
};

Our tag_ptr class stores a raw pointer to the tagged data and a copy of the tag. The dereference operator compares the stored tag to (what should be) the tag of the allocation and it aborts the program if they don’t match.

Note: There is no destructor since the tag_ptr does not own memory! A destroy method must be explicitly called, which means memory leaks are possible.

A possible solution to avoid memory leaks is to allow tagged<T> to exist independently from tag_ptr<T>, and to have tagged<T> do the memory management using all the standard RAII stuff.

If you choose this route, get rid of the destroy method and the make_tagged functions, and add a convenience method to tagged<T> to produce tag_ptrs:

template<class T>
struct tagged {
    ...
    
    tag_ptr<T> get_ref() {
        return tag_ptr(*this);
    }
};

Tradeoffs of tag_ptr<T>

Compared to other smart pointers, tag_ptr<T> has advantages and disadvantages.

The main advantage (and it is a pretty major one), is that it is a “trivial type” to use C++ standard terminology. That means it can be bitwise copied freely, e.g., using memcpy. Its copy constructor has no custom logic.

The second advantage is that they have no problem dealing with arbitrary graphs, unlike std::shared_ptr (which imposes a DAG-like structure) or std::unique_ptr (which imposes a tree-like structure).

The major disadvantage is the space cost. Each copy of a tag_ptr is twice the size of a regular pointer. You’ll want to keep the amount of tag_ptrs to a minimum. The 8 byte header isn’t very substantial unless you’re heap allocating lots of small structs.

The second disadvantage is the performance cost. Allocations are slightly slower because of the random number generation, while dereferences require an extra branch to check for the tags. The branch should be predicted correctly every time so the cost is pretty minimal. You should also never pass tag_ptrs as function arguments unless the function might delete the data, you should take a const reference to the data directly instead (so the check happens only once for that call).

Something that is neither an advantage nor disadvantage is that tag_ptrs don’t manage memory. They’re more like a runtime equivalent of lifetime analysis. While this means you need to use another technique to manage memory, it also means they can be used with other techniques!

They work pretty well with memory arenas for example, the only constraint is that you can’t just reset the arena pointer, you must clear the memory with 0.

Weak Tagged Pointers

One thing that is a bit unfortunate about our tag_ptr implementation is that we can’t safely check if they’re valid before dereferencing them.

If the pointer is valid or the data has been overridden, it’ll work. But if the page was released back to the operating system at some point or its read permission removed, trying to compare the tags will cause a segmentation fault.

With the standard allocator there isn’t much we can do about this6, but if we’re in control of the allocator there’s a simple trick that can be done.

Instead of unmapping a page when it is no longer in use, you can tell the kernel you don’t need it anymore. On Linux you can use the following:

madvise(page_address, page_size, MADV_DONTNEED);

This releases the page and any attempt to read the page will result in a zero-fill-on-demand page. The address range of the page is still considered “in use”, not a big deal with a 64 bit address space but something to consider. The allocator should store the page in a freelist and reuse it at some point instead of just calling mmap.

Using MADV_FREE instead of MADV_DONTNEED means the page is only released when there is memory pressure.

The windows equivalent of MADV_FREE is MEM_RESET on VirtualAlloc. To replicate the behavior of MADV_DONTNEED requires decommiting and re-commiting the page.

With this in place, we can safely call the is_valid method on tag_ptr to check if the data it points to remains valid before dereferencing it.

Weak tagged pointers also allow us to throw an exception or return a default value instead of segfaulting when dereferencing, if that’s relevant.

Conclusion

Tagged Pointers are a pretty simple approach to achieve memory safety7 with a relatively minor performance impact. They have advantages and disadvantages compared to shared pointers and unique pointers, but their ability to deal with arbitrary graphs makes them highly complementary.

Using a custom memory allocator that marks pages as not needed instead of unmapping them allows tagged pointers to become weak tagged pointers, meaning you can check their validity before accessing them.

Weak tagged pointers are a much more efficient alternative to regular weak pointers (the sort often used in tandem with shared pointers).

The next time someone complaints about your use of a “memory unsafe” language, just ask them if they’re ok with a few % performance loss in exchange for safety. If calling .clone() in Rust or using Rc is considered acceptable, so should this.

I wish C had something like the following:

struct Vector<T> {
  T* data;
  size_t capacity;
  size_t length;
};

void vector_resize<T>(Vector<T>* v, size_t capacity) {
  v->data = realloc(v->data, capacity * sizeof(T));
  v->capacity = capacity;
}

void vector_append<T>(Vector<T>* v, T element) {
  if (v->length >= v->capacity) {
    vector_resize(v, v->capacity ? v->capacity * 2 : 8);
  }
  v->data[v->length] = element;
  v->length++;
}

void vector_clear<T>(Vector<T>* v) {
  free(v->data);
  v->data = NULL;
  v->capacity = 0;
  v->length = 0;
}

Vector<char*> v = {0};
vector_resize(&v, 10);
vector_append(&v, "abc");
vector_clean(&v); 

C++ can easily do the above with templates, and I personally quite enjoy programming in “C with methods, templates, namespaces and overloading”, avoiding the more complicated elements of C++ (like move semantics2) while still benefiting from it.

But sometimes you really want to program in pure C for whatever reason. There isn’t really a perfect way to do the above in C, but there are ways, some better than others.

I’m aware of the following four methods:

• Template Macros

• Template Headers

• Type Erasure with void*

• Inlining Macros

We’ll go over each of them and discuss their pros and cons.

BAD: Template Macros

C++ templates are an implementation of generic types based on monomorphization. What does that mean? It’s basically copy paste + find & replace.

When you write:

template<typename T> 
T id(T data) { return data; }
...
int i = id(10);
char* str = id("abc");

What the compiler spits out is something like:

int id_int(int data) { return data; }
char* id_char(char* data) { return data; }
...
int i = id_int(10);
char* str = id_char("abc");

Though the names of the functions aren’t nice and obvious like that, they’re actually unreadable mangled gibberish. The important point is that the templated code gets “copy-pasted” each time you call it with a different type, and everywhere T was used the concrete type is used instead.

We can implement a similar thing in C using macros for our Vector struct:

#define _VECTOR_TEMPLATE_INSTANCE(T) \
typedef struct { \
  T* data; \
  size_t capacity; \
  size_t length; \
} Vector_##T; \
\
void vector_resize_##T(Vector_##T* v, size_t capacity) { \
  v->data = realloc(v->data, capacity * sizeof(T)); \
  v->capacity = capacity; \
} \
\
void vector_append_##T(Vector_##T* v, T element) { \
  if (v->length >= v->capacity) { \
    vector_resize_##T(v, v->capacity ? v->capacity * 2 : 8); \
  } \
  v->data[v->length] = element; \
  v->length++; \
} \
\
void vector_clear_##T(Vector_##T* v) { \
  free(v->data); \
  v->data = NULL; \
  v->capacity = 0; \
  v->length = 0; \
}
#define VECTOR_TEMPLATE_INSTANCE(T) _VECTOR_TEMPLATE_INSTANCE(T)

The two levels of macros are necessary for Vector_##T to work. Without them it’ll just become Vector_T. The preprocessor is weird.

Anyway, we can then call this macro like so:

VECTOR_TEMPLATE_INSTANCE(int)

And all the code in the macro gets copy pasted with T replaced with int.

There are many issues with this particular solution. The first and most obvious is just how hideous it is. All those backslashes and pound signs… ugh. You also get no syntax highlighting or autocompletion and errors will only show up once you invoke the macro. But that’s not all! The following doesn’t work:

VECTOR_TEMPLATE_INSTANCE(char*)

Because the “name mangling” will be borked due to the *. The solution is to typedef pointer types or wrap them in a struct, or to add an extra “name” parameter to the macro like:

VECTOR_TEMPLATE_INSTANCE(char*, str)

But we also need to consider where to call this macro, because we need to be careful not to call it with the same type in multiple places, or we might end up with multiple definitions for the same function.

For an example of this approach in the wild, check out M*LIB. Personally I find this way of implementing generics in C to be all downsides.

GOOD: Template Header

The idea of the template header is similar to the template macro, but using a “parameterized” header file instead:

#ifndef T
#error you need to define T before including this header
#else

typedef struct {
  T* data;
  size_t capacity;
  size_t length;
} CAT(Vector,T);

void CAT(vector_resize,T)(CAT(Vector,T)* v, size_t capacity) {
  v->data = realloc(v->data, capacity * sizeof(T));
  v->capacity = capacity;
}

void CAT(vector_append,T)(CAT(Vector,T)* v, T element) {
  if (v->length >= v->capacity) {
    CAT(vector_resize,T)(v, v->capacity ? v->capacity * 2 : 8);
  }
  v->data[v->length] = element;
  v->length++;
}

void CAT(vector_clear,T)(CAT(Vector,T)* v) {
  free(v->data);
  v->data = NULL;
  v->capacity = 0;
  v->length = 0;
}

#undef T
#endif

We can then use this header file like so:

#define T int
#include "vector_template.h"

We’ve gotten rid of the backslashes, we get proper syntax highlighting, and by just adding a #define T int line temporarily we get type checking and autocompletion and all that good stuff.

The CAT3 macro is needed to implement the indirection for ##T we used in the macro version. There are ways to make it a little prettier (see below) but I wanted to show the basic solution first.

We still have the naming issue when using pointer types and the multiple definition issue, but thankfully they can be worked around without too much trouble by doing the following:

#ifndef T
#error you need to define T before including this header
#else

#ifndef SUFFIX
#define SUFFIX T
#endif

#define G(N) CAT(N, SUFFIX)

typedef struct {
  T* data;
  size_t capacity;
  size_t length;
} G(Vector);

void G(vector_resize)(G(Vector)* v, size_t capacity);
void G(vector_append)(G(Vector)* v, T element);
void G(vector_clear)(G(Vector)* v);

#ifdef VECTOR_IMPLEMENTATION

void G(vector_resize)(G(Vector)* v, size_t capacity) {
  v->data = realloc(v->data, capacity * sizeof(T));
  v->capacity = capacity;
}

void G(vector_append)(G(Vector)* v, T element) {
  if (v->length >= v->capacity) {
    G(vector_resize)(v, v->capacity ? v->capacity * 2 : 8);
  }
  v->data[v->length] = element;
  v->length++;
}

void G(vector_clear)(G(Vector)* v) {
  free(v->data);
  v->data = NULL;
  v->capacity = 0;
  v->length = 0;
}

#endif

#undef G
#undef SUFFIX
#undef T
#endif

We add an extra parameter called SUFFIX (which is mapped to T by default) to allow changing the suffix in the type and function names (could also have been PREFIX or NAME or whatever you want to call it). The CAT macro is wrapped in the G4 macro to make things nicer to look at and automatically apply the suffix. Lastly we employ the same approach used by header-only libraries to separate the function declarations from their implementation, solving the “careful where you use this” issue.

For an example of this approach in the wild, check out STC.

MEH: Type Erasure using void*

I mentioned that C++ generics use monomorphization, but that’s not the only way to implement generic types. Java, for example, uses type erasure.

Basically, when you write the following in Java:

class Vector<T> {
  public T[] data;
  public size_t capacity;
  public size_t length;
}

Vector<Integer> v = new Vector<Integer>();

What the compiler is actually doing is:

class Vector {
  public object[] data;
  public size_t capacity;
  public size_t length;
}

Vector v = new Vector();

The actual element type has disappeared. T and int have been erased. This approach to generics has pros and cons, but for our purposes the important part is that we can actually do something pretty similar in C without any macros:

typedef struct {
  void* data;
  size_t capacity;
  size_t length;
} Vector;

void vector_resize(Vector* v, size_t capacity, size_t t_size) {
  v->data = realloc(v->data, capacity * t_size);
  v->capacity = capacity;
}

void vector_append(Vector* v, void* element, size_t t_size) {
  if (v->length >= v->capacity) {
    vector_resize(v, v->capacity ? v->capacity * 2 : 8, t_size);
  }
  size_t offset = v->length * t_size;
  memcpy((char*)v->data + offset, element, t_size);
  v->length++;
}

void* vector_get(Vector* v, size_t index, size_t t_size) {
  size_t offset = index * t_size;
  return (char*)v->data + offset;
}

void vector_clear(Vector* v) {
  free(v->data);
  v->data = NULL;
  v->capacity = 0;
  v->length = 0;
}

We can use it as follows:

Vector v = {0};
vector_resize(&v, 10, sizeof(double));
double element = 4.0,
vector_push(&v, &element, sizeof(double));
vector_clean(&v);

Not a macro in sight, wonderful. But we had to pay a price. Every function now needs an extra parameter so we know the size of the data we are working with. We also can’t index the vector data directly, we have to treat it as an opaque blob of bytes to write into (using memcpy). We need vector_get for the same reason.

We also have to pass elements by pointer. That means we need the element to be an lvalue so we can take its address. That’s why 4.0 is stored in a variable first.

The biggest issue is the same one Java had before version 55, there’s no type safety:

Vector v = {0};
vector_push(&v, &some_int, sizeof(double));
vector_push(&v, &some_double, sizeof(char*));
vector_push(&v, &some_char_ptr, sizeof(int));

We can improve upon the sizeof issue by storing size information on the vector itself:

typedef struct {
  void* data;
  size_t capacity;
  size_t length;
  size_t t_size;
} Vector;

Vector vector_init(Vector* v, size_t element_size) {
  return (Vector){.t_size = element_size};
}

void vector_append(Vector* v, void* element) {
  if (v->length >= v->capacity) {
    vector_resize(v, v->capacity ? v->capacity * 2 : 8);
  }
  size_t offset = v->length * v->t_size;
  memcpy((char*)v->data + offset, element, v->t_size);
  v->length++;
}

void vector_resize(Vector* v, size_t capacity) { ... }
void* vector_get(Vector* v, size_t index) { ... }
void vector_clear(Vector* v) { ... }

Parameter t_size is a specific instance of a more general t_info6 structure, which stores all the information the container needs from the element type to function. Storing this information in the container, however, does not prevent us from appending a double to a vector of ints.

The main reason to use this approach is separate compilation. You can compile the code for a vector once and then link to it many time. Useful for dynamic libraries or FFI-related stuff, but it doesn’t provide much value when programming in C itself.

For an example of this approach in the wild, check out GLib.

GOOD: Inlining Macros

This one is essentially a “fixed” version of the template macro solution. The idea is to have individual macros for each operation, none of which declare anything:

#define Vector(T) struct { \
  T* data; \
  size_t length; \
  size_t capacity; \
}

#define vector_resize(v, capacity) { \
  v->data = realloc(v->data, capacity * sizeof(v->data[0])); \
  v->capacity = capacity; \
}

#define vector_append(v, element) { \
  if (v->length >= v->capacity) { \
    vector_resize(v, v->capacity ? v->capacity * 2 : 8); \
  } \
  v->data[v->length] = element; \
  v->length++; \
}

#define vector_clear(v) { \
  free(v->data); \
  v->data = NULL; \
  v->capacity = 0; \
  v->length = 0; \
}

We can use it as follows:

typedef Vector(char*) Vector_str;

Vector_str v = {0};
vector_resize(&v, 10);
vector_append(&v, "abc");
vector_clean(&v);

Compared to the template macro solution this has massive advantages. There’s no “name mangling” issue because the macros don’t declare anything. We don’t need to worry about where these macros are invoked for the same reason. Outside of function arguments we don’t even need the typedef.

As with the template macro approach, this approach also suffers from a lack of syntax highlighting and typechecking, plus those damn backslashes again. It also causes a bit of code bloat since the same code is getting inlined all over the place.

One way to minimize those issues is to shift some of the work to helper functions. In the our Vector example none of the macros is large enough or complicated enough to justify doing this, but for a HashMap you’ll definitely want to do it.

Regardless, since the we’re working with multiple smaller macros rather than one big unwieldy macro, it’s a lot more bearable.

For an example of this approach in the wild, check out CC (though CC goes even further with typeof tricks to not require the typedef). There’s also Sean Barrett’s stb_ds library which avoids the typedef by hiding the length and capacity behind the data pointer itself in memory.

I do not understand their hatred of the typedef, but both libraries are good.

Summary

I’ve gone over four approaches to implement Generic types in C using different techniques, each with pros and cons. Well, except the first one, template macros, where I can’t really find any pro, only cons.

The type erasure (void*) approach only makes sense if FFI or dynamic linking is involved, or you really must avoid code bloat at all costs.

The inlining macro approach is the most pleasant to use, but it is a bit annoying to develop since it requires programming inside macros (bleh). For more complex data structures than a dynamic array you’ll want to shift as much work to helper functions as possible, making it a bit of a hybrid with the void* approach.

The template header version is somewhat annoying to use but is in my opinion the most pleasant to actually program. It also works ok in an FFI or dynamic linking setting one level higher. For example, a graphics library might expose Vector_texture and Vector_vertex types and such.

So, to me, there are actually only two choices when all you care about is programming in C itself: template headers or inlining macros. The former is nicer to program while the latter is nicer to use. Try them out, see which one you prefer.

UPDATE: This article was discussed on hackernews.

#define _CAT(A, B) A##_##B
#define CAT(A, B) _CAT(A, B)

Disclaimer 1: Opinions expressed here are solely my own and do not express the views or opinions of my employer.

As part of my job I’ve been developing an implementation of the Systems Modeling Language 2.0 standard (typically referred to as SysMLv2). Think UML but for Systems Engineering (aerospace, automotive, etc.) and designed by the same people, OMG.

Content-wise I think it’s a pretty good standard. It has all the stuff I’d expect to find1 and while the way you specify some things is a little weird, it could be much worse. It’s certainly better than UML or its previous iteration (SysMLv1).

Specification-wise, however, it is abysmal. Implementing this thing is an exercise in frustration. There’s a whole separate spec (KerML) that has to be understood and implemented first even though maybe 3 people max actually care about the distinction. Every concept is specified as a UML class, some with 10+ levels of inheritance, including multiple-inheritance.

The associations between concepts are all bidirectional and are overridden multiple times throughout the inheritance chain, e.g., one class might have an association with a multiplicity of 0..*, unique, unordered; a subclass redefines it to have multiplicity 0..1; a subclass of that subclass redefines it again to be multiplicity 1..*, non-unique, ordered. Barbara Liskov and her substitution principle be damned.

But relationships in SysMLv2 are reified as their own objects, so what you technically have are bidirectional associations (in a UML sense) between SysMLv2 elements and SysMLv2 relationships (which are also SysMLv2 elements), and then between the relationships and other elements. You can’t just get the “value” of an AttributeUsage, there’s a thing in the KerML standard library that attributes must specialize which is how you’re supposed to access their values, and in order to actually find the data you have to dig through multiple levels of relationships to find a FeatureValue2 which is the relationship that has an association with the damn value.

It’s everything wrong with object oriented design dialed up to 11 until the knob breaks. I despise it. But this post isn’t actually about SysMLv2. The SysMLv2 specification is a manifestation of a much bigger issue: Object-Oriented Brain-Rot3.

I find that doing too much OOP leads to the degradation of mental faculties, intelligence, and common sense. Hence the term brain-rot.

Disclaimer 2: I have nothing against objects and interfaces, I think they’re nearly fundamental computational constructs. Even a closure can be seen as an object whose interface is a single method. This post is about a type of thinking.

Is a Square a Rectangle?

In geometry a square is a kind of rectangle. All squares are rectangles, but not all rectangles are squares. In OOP, things are a little trickier. Consider:

class Rectangle : Shape {   
  double width;   
  double height;    
  
  Rectangle(double width, double height) {     
    this.width = width;     this.height = height;   
  }    
  
  double getArea() {
    return width * height;   
  } 
}  

class Square : Rectangle {   
  Square(double length) {     
    this.width = length;     
    this.height = length;   
  }    
  
  double getArea() {
    return width * width;
  } 
}

The above code has a major issue:

Rectangle r = Square(10); 
r.height = 20; 
print(r.getArea()); // oops

Square breaks the Liskov Substitution Principle. I can do things with a Rectangle (like setting its height separately from its width) which I cannot do with a Square. The typical solutions are to make the objects immutable, or to make both inherit from Shape directly.

But why do Rectangle and Square need to be modeled as objects in the first place? What’s their behavior? What is the “Single Responsibility” of a Rectangle or a Square?

While the exercise is meant as a way to showcase potential issues arising from modeling real world taxonomies as object-oriented class hierarchies4, to me the very existence of the exercise is problematic, because it means people are trying to model things as class hierarchies that shouldn’t be class hierarchies in the first place.

What problem are you solving?

The issue starts with the very act of modeling Rectangles and Squares as a class hierarchy. Why are you doing this? What purpose do these classes serve? You’re building some sort of application, what is the purpose of Square in the application?

Consider as an example a video game. For whatever reason I may need to draw a square onscreen. Do I need an actual Square object that subclasses Shape with a draw method or so? No.

The square emerges from some game state that requires the appearance of a square onscreen. There’s no need to store the square itself at any point, it is derived from other data. A “square” can be something as simple as:

void draw_square(int x, int y, int length) {
  draw_rectangle(x, y, length, length); 
}

While a “rectangle” might be:

void draw_rectangle(int x, int y, double width, double height) {
  Point points[] = {
    {x, y}, {x+width, y},      
    {x+width, y+height}, {x, y+height}   
  };   
  draw_polygon(points); 
}

The screen gets redrawn every frame so keeping the data for the shapes around is completely unnecessary. Maybe you’re drawing a square because a unit has been selected in a strategy game and the square serves as the selection border. The square exists while the unit is selected, and does not exist otherwise.

But maybe you’re not actually working on a game. Perhaps you’re working on a vector drawing application where the user can add squares and rectangles and edit them. In this case you actually need to store squares and rectangles as actual entities.

But even then, do squares and rectangles need to be modeled as separate classes? A square is usually just a rectangle whose width and height are the same. In almost every vector drawing app you only have “rectangles”, even if you create a “square” you’re given the option of setting the width and height separately because that is what is most useful in a drawing application.

What about rounded rectangles? Is “RoundedRectangle” a subclass of Rectangle? Just have rounded corners be a property of rectangles. I have separate buttons to add rectangles and rounded rectangles in Pixelmator Pro but all they do is set a different default for the rounding of the corners, the entity produced is the same. There’s no need to model a separate concept. All a “rectangle” really is in such an app, ultimately, is a convenient user interface to manipulate some Bézier curves.

There’s no need to waste time worrying about how well whatever concepts fit into your class hierarchy. You don’t need the class hierarchy and you need the real-world relations between the concepts even less. Just write the code that solves the problem.

What if the relations are the point?

In SysMLv2 the relations between the various concepts are relevant. The taxonomy itself is important information. But imposing that structure onto a codebase is a very strange thing to do. Why does the code need to reflect the taxonomy? If you care about relations, just model them directly. Here’s how you model the relation between a square and a rectangle in Datalog or Answer set programming:

% every square is a rectangle with the same width and height 
rectangle(Id, L, L) :- square(Id, L).  

% every rectangle of equal width and height is a square 
square(Id, L) :- rectangle(Id, L, L).

With the above rules, I can add some facts to the database and have the set of squares and rectangles be kept consistent and up-to-date automatically:

square(a, 3).
square(b, 5).
rectangle(c, 1, 2).
rectangle(d, 4, 4).
rectangle(c, 8, 7).

If I query all the squares, then I get a, b and d. Even though I defined d as a rectangle, it is automatically defined as a square as well. Similarly if I query all the rectangles, I get all 5 entities. That’s how you model bidirectional relationships. Trivial.

By using the right tool for the job I made my life a lot easier. Had I suffered from OO brain-rot like OMG seems to, I would have made a class hierarchy. But then I could make a rectangle with equal width and height that for whatever reason is not a square.

It is entirely the wrong tool for the job. You’ve applied object-oriented design so much that you see it as not only the solution to everything, but a necessary first step.

Your mental faculties have actually decreased as a result. As a newbie you would just solve the problem directly, you wouldn’t even waste time on this, and the outcome would be better. What else can this be called besides brain-rot?

A paradigm made of straw

Some readers may accuse me of making up a straw man and that this has nothing to do with OOP, but I disagree. The two are very much intertwined. While you can certainly use objects and interfaces in reasonable ways, that’s not how OOP is taught and practiced. When I was in university and learning OOP, object-oriented design was always at the forefront.

From the very beginning we were learning how Cat and Dog are subclasses of Mammal which in turn is a subclass of Animal. Though maybe they should be subclasses of Feline and Canine respectively? What about Carnivore? Do we use multiple-inheritance?

While the exercise was meant to explain inheritance, I really need to stress out how it was entirely disconnected from solving an actual problem. The same pattern repeated later when we were modeling “Customers” and “Employees” as subclasses of “Person”. Once again, a taxonomy that did not serve any actual purpose in solving a problem. All a customer really was in the end was an ID in a database.

This is what students are being evaluated on, and it’s how they learn to think about problems: in terms of object-oriented design. That the first step to creating a software solution is to model a taxonomy of concepts as a class hierarchy. This is a detrimental and unproductive way of thinking. It creates a rigid structure in the codebase that does not reflect the actual needs of the codebase, making it a pain to work with.

As an example, consider how people turn their noses at “Utils” or “Manager” classes even though just having some extra functions laying around is often the simplest and most natural approach. What issues are such classes actually causing, beyond going against some nonsensical object-oriented design ideal5?

Later those same students go to a job interview and are questioned on their knowledge of design patterns, another entirely wrong way to look at problems. You don’t need the Visitor Pattern, you need sum types (even C can do them).

So a combo-wombo of an entirely wrong way to think about problems, leading to poor program structure, plus rote memorization of patterns meant to solve the structural issues caused by the paradigm.

If you’re worried about maintainability, which is what OOP is supposedly good at6, then making a class-hierarchy based on a real-world taxonomy is the worst thing you can possibly do. Even serious real-world taxonomies need to be changed from time to time. Your partial taxonomy is going to have to change and the rigid structure it imposed on your codebase won’t make it easy.

Reset your brain and focus on the problem. You have data that needs to be transformed and/or moved around. How to accomplish that in a simple and performant manner? That’s all you need to care about.

The so called “Borrow Checker” is a part of Rust whose job is to enforce a pair of rules at compile-time:

1. Any borrow must last for a scope no greater than that of the owner.

2. You may have any number of immutable borrows to the same data, or a single mutable borrow to that data, but not both at the same time. 1 writer xor N readers.

But what’s a “borrow”? What’s an “owner”? Why those two rules? Is Rule 2 necessary if we don’t care about multithreading?

I’m not going to explain the rust borrow checker. You can read the Rust documentation if you want a very detailed explanation. Rather, the goal of this post is to naturally arrive at a similar solution by tackling the same problem: memory safety through compile-time checks.

There are two main vulnerabilities we’re trying to prevent: Double-Free (alongside memory leaks) and Use-After-Free. The nastiest vulnerability, Out-of-Bounds Access, is trivial to prevent with minimal runtime support3.

We’ll start from a simple C-like language and add compiler checks (our own “borrow checker” of sorts) to make it memory safe, and hopefully show you how you could have arrived at a similar solution in the process.

Did you know C# has something like a “borrow checker” too4?

Meet Cnile

We’re going to turn a made-up C-like language memory safe. We’ll call it Cnile5.

Cnile is very similar to C with a few minor differences: In Cnile the declaration syntax puts all relevant information next to the type, e.g. int[] x instead of int x[]. All data is initialized to 0. “Constness” is a property of a binding and not of a type, and is transitive.

The most important change is that it supports slices. Arrays don’t decay to pointers, they implicitly cast to slices, and slices track their length:

int[] x = {1, 2, 3, 4};

int example(int[..] y) {
  // will crash the program
  return y[5];
}

example(x);

Slices are equivalent to the following anonymous struct:

{ T* data; size_t length; } 

The T[..] syntax was proposed for C by Walter Bright back in 2009. Works well enough. Note that an array parameter would have a different meaning.

The above takes care of bounds checking. If the bounds check is too expensive for whatever reason, just grab the internal pointer and index that.

Memory Safety at Runtime without GC

If we ignore memory leaks, there’s actually a pretty easy way to also handle both double-free and use-after-free at runtime.

If all allocations are stored in arenas/regions, and the smallest arena size is at least one memory page in size, you can just use operating system features to protect the relevant pages after an arena is cleared. Any attempt to access that memory will cause a segmentation fault. There’s barely any performance cost to doing this6.

With a 64-bit address space it’ll take a very very long time before any page needs to be reused, and the chances that an invalid pointer to such an old page survived without being dereferenced is extremely low.

This only works for bulk allocations. If you want memory safety with individual malloc/free style memory management, you can do so with memory tagging.

Instead of a raw pointer, a safe_malloc would return a special reference type, which is equivalent to the following anonymous struct:

{ size_t tag; void* ptr }

The Vale programming language calls these generational references. Each allocation generates and stores a non-zero 64-bit random number tag at the top of the allocation. A simple xorshift generator works great for this.

When dereferencing one of these tagged references, the tag stored in the reference and the tag stored in the allocation are compared. If they don’t match then we’ve detected a use-after-free.

There’s around a 0.0000000000000002% chance of collision. Extremely unlikely. Your odds of winning a lottery that would make you a multi-millionaire are over a billion times higher.

A safe_free function would only accept references like the above, not pointers, and would zero-out the tag in the allocation.

As with bounds checks the check for the tag does not need to be done on every access. As soon as you know the reference is valid, you can use the pointer directly until you call a function that might free the reference.

But we don’t want any runtime costs, we want to do everything at compile-time.

Double-Frees and Memory Leaks

Consider the following data structure:

struct DynArray {
  int* data;
  size_t length;
  size_t capacity;
}

void push(DynArray* a, int i) { 
  if (a->length == a->capacity) {
    a->capacity = (a->capacity + 1) * 2;
    a->data = unsafe_realloc(...);
  }
  a->data[a->length] = i;
  a->length += 1;
}

void delete(DynArray* a) {
  unsafe_free(a->data);
  a->data = null;
  a->length = 0;
  a->capacity = 0;
}

DynArray a = {};
push(&a, 1);
delete(&a);

Our first goal is to ensure that delete is called one time for each DynArray, thereby preventing memory leaks and double-free-related vulnerabilities.

Calling delete on “a” twice is actually perfectly safe, the problem only occurs if we copy “a”. This would result in a shallow copy, as the internal pointer of both arrays would point to the same memory.

Deleting both “a” and a shallow copy of “a” results in a Double-Free. We need to prevent DynArray’s from being shallow copied7. Automatically performing a deep copy would be an option, but Cnile is a systems programming language, we don’t want any stinky hidden operations like that.

First Attempt: No Implicit Copies

Let us add a special annotation to DynArray, which tells the compiler we don’t want any implicit copies of it happening:

[no_copy]
struct DynArray {
  ...
}

DynArray a = {};
push(&a, 1);
DynArray b = a; // error
a = {}; // ok but now we're in trouble
delete(&a) // not freeing the right "a"

We’ve forbidden the compiler from inserting implicit copies of DynArrays. The only way to pass a DynArray around now is through a pointer. We would have to go out of our way to make a function that shallow copies DynArrays to get into trouble again8.

Note: dereferencing a DynArray pointer would create a shallow copy so that is not allowed either, only the individual members can be accessed. If we were bothering with access control, the internal pointer would be private.

We designed delete to be safe to call multiple times, so double-free cannot occur without “cheating” and messing with the pointer inside the array directly, but there are major problems with this solution:

• We can’t return a DynArray from a function directly, we have to use an out parameter pointer.

• Nothing actually forces us to call delete, so memory leaks are not solved.

• We can assign a new value to a DynArray, making the original impossible to delete (another source of memory leaks).

If we forget to call delete, once a DynArray goes out of scope we have a memory leak. We need to ensure delete always gets called.

Second Attempt: Single-Use Types

Let us change our perspective a little bit. Instead of thinking about preventing copies, we enforce instead that a DynArray must be used (without going through a pointer) exactly 1 time. In Computer Science these single-use types are called Linear Types, so we’ll call the annotation linear:

[linear]
struct DynArray {
  ...
}

DynArray a = {};
DynArray b = a; // ok, 1 use
printf("%d\n", a.length) // error, a is used up
// error, we never used (e.g. deleted) b

This might not seem like much of a change, but we’ve gotten quite a lot from it. The line DynArray b = a is now allowed: it shallow copies “a” into “b” but because “a” cannot be used afterwards, there’s no issue, we are only able to call delete on “b”. And we have to either call delete on b, or assign it to something else.

In Rust this is called a “move” but there’s no need to think of it as “moving”. It’s just a side-effect of how the single-use rule works.

To make a.length work without using up “a”, it can be treated as if it were actually:

(&a)->length

Which would be allowed since accessing “a” through a pointer doesn’t consume it.

To make delete actually consume a DynArray, we just need to change it to take a DynArray by value:

void delete(DynArray a) {
  unsafe_free(a.data);
  unsafe_drop(a);
}

Passing a DynArray to delete will shallow-copy it, consuming it in the process and making it inaccessible. Inside delete we only need to free the pointer, no need to zero-out other data since it is no longer valid and cannot be accessed.

The unsafe_drop builtin is used to get rid of “a” inside of delete, since here too we must use it once.

This solution, if we ignore aliasing through pointers, is actually enough to achieve memory safety. The language ParaSail, which lacks pointers, achieves memory safety via a similar approach (though it inserts implicit deep copies and destructor calls).

A “Borrow Checker” is only necessary to retain memory safety in the presence of aliasing. Aliasing is where the challenge is. For example:

DynArray a;
push(&a, 1);
int* x = &a.data[0];
delete(a);
printf("%d\n", *x) // Use-After-Free

We need to prevent the above somehow.

Use-After-Free and Aliasing

We’ve already partially solved Use-After-Free with the previous solution. As long as we don’t actually store a pointer to a DynArray anywhere, there is no way to use it after it has been deleted. The single-use rule takes care of that.

Aliasing due to pointers in function parameters also does not introduce any problems. The pointers are scoped to the function so they’ll be gone by the time the function returns. Returning a pointer from a function is also ok as long as it is immediately passed as an argument to another function, not stored in a variable.

Pointers that are restricted to being used in this manner are called “second-class references” and are a perfectly reasonable and simple solution to the problem. The Hylo language works like this for example.

This is not the approach we will use, we want a bit more control. Before that though there’s still one problem that we need to tackle. Remember how Rust had that weird Rule 2 enforcing 1 writer xor N reader “borrows” (note: borrow=reference)? It’s important regardless of multithreading. Consider:

void concat(DynArray* a, int[..] b) {
  for (int i = 0; i < b.length; i++) {
    push(&a, b[i]);
  }
}

DynArray x = ...;
concat(&x, x.data[0..2]); // Kaboom

If you’ve done serious C or C++ programming the above should immediately trigger PTSD. The v[i..j] syntax is how we create a slice from existing data in Cnile: we’re creating a slice of the first 3 elements of x. So the idea is for the concat call above to append the first 3 elements of x to its end.

The problem is that the push call inside concat might need to reallocate, which may then perform a new allocation, copy the data to it, and free the old allocation. If this happens, the slice is now pointing to freed memory, causing a use-after-free.

Forbidding internal pointers could be an option for a more higher level language design. If all pointers point to the “source”, which is only ever invalidated by a call to a function like delete, then we avoid the issue. Many languages (e.g. Java) lack internal pointers. Not an option for us since we’re Cnile.

There is one more advantage to the 1 writer xor N reader rule (outside of multithreading): it prevents iterator invalidation bugs. Without it we must either allow iterator invalidation bugs (like C++), or catch them at runtime (like Java).

We’ll go with 1 writer xor N readers since it prevents lots of bugs, but this is an interesting design space and I encourage you to think about it.

Safe References

We’ll keep pointers as actual pointers and leave them as “unsafe” constructs in Cnile. As a safe alternative, we’ll provide references that enforce the “borrow checking” rules. Slices will be made safe references as well.

Let us consider concat above, and make a safe version of it with references:

void concat(DynArray& a, int[..] b) {
  for (int i = 0; i < b.length; i++) {
    push(&a, b[i]);
  }
}

DynArray x = ...;
concat(&x, x.data[0..2]); // error

All we’ve done is change * (pointer) to & (safe reference). But now the concat line fails because of the 1 writer xor N reader rule. How do we check this though? Well, internally the compiler is tracking extra information:

void concat(DynArray& <?p1> a, int[..] <?p2> b) {
  for (int i = 0; i < b.length; i++) {
    push(&a, b[i]);
  }
}

DynArray x = ...;
concat(&x <x>, x.data[0..2] <x.data>); // error

Every reference carries alongside it a hidden “path” to the data it refers to. Unlike Rust, but like C#, the user never writes these out themselves, they have no syntax.

The compiler checks the hidden reference paths for overlap. Overlapping paths that have more than 1 writer or a writer and a reader are rejected.

Function parameters have “variables” (marked with ?) for the paths, that are assigned when the function is called. Within concat the two inputs are assumed to be independent, the check happens on each callsite.

The same paths can be used to detect use-after-free:

DynArray  a = {}; // available: {a}
DynArray& x = &a; // available: {a, x <a>}
DynArray  b = a;  // available: {x <a>, b}
concat(x <a>, {1, 2, 3});  // error, a no longer available

Things get trickier when we want to return references from functions, what path should the output reference have in the following example:

int[..] <?> shortest(int[..] <?p1> a, int[..] <?p2> b);

The shortest function returns the slice with the smallest length from the two inputs, meaning the return reference path <?> could end up being either p1 or p2.

In these situations all we can do is a safe approximation, and record both paths:

int[..] <?p1 & ?p2> 
shortest(int[..] a <?p1>, int[..] <?p2> b);

When using this function, the output will only last for as long as either of the inputs lasts, regardless of the actual result:

int[..] x = a[0..2]; // x <a>
int[..] y = b[0..3]; // y <b>
int[..] z = shortest(x, y); // z is x, but has path <a & b>
delete(b);
printf("%d/n", z[0])  // error, b is no longer available

This is the unavoidable limitation with compile-time checking, some things will only be known at runtime, so we need to consider all situations and assume the worst.

Path Precision

For the example above the most specific we can be is the intersection of the two input paths, but the compiler will do this even for cases where it isn’t necessary:

int[..] <?p1 & ?p2> 
find_subslice(int[..] a <?p1>, int[..] <?p2> b);

This function returns a sub-slice of “a” that is equal to “b” (if any), or just returns an empty slice. The result will never point to “b” but the compiler has no way to tell this from the function’s declaration. So we need to provide more information.

In Rust this would be done with lifetime annotations. In C# this is done by explicitly saying which inputs cannot be part of the output. We’ll follow the C# solution:

int[..] find_subslice(int[..] a, [in] int[..] b);

The [in] annotation means that “b” will not be part of the output. In C# the keyword scoped is used instead, but the meaning is the same.

This gets us most of the power of Rust without introducing lifetime annotations. There are two major capabilities that we lose by not having them:

• We have no way to state in a function or struct declaration that two references must have overlapping paths. Paths are always allowed to be disjoint.

• We have no way to state in a function declaration with a struct that stores multiple references as input, which reference paths will be part of the output.

To make the second case clear, let us talk about reference structs.

Reference Structs

In C#, a struct that contains a “ref” type (like Span) must itself be a “ref” type. In Cnile, we have the same constraint:

[reference]
struct SlicePair {
  int[..] a;
  int[..] b;
}

Structs that contain reference types must themselves be annotated as being references, so the compiler knows it needs to track paths for them.

The struct SlicePair above stores two slices together. What is its path?

int[..] x   = ... // x has path <foo>
int[..] y   = ... // y has path <bar>
SlicePair p = {x, y} // p has path <?>

If we treat a SlicePair as a unit, then “p” must have path <foo & bar>. The fields p.a and p.b, however, could each track individual paths.

Field-level tracking depends on how much information the compiler carries around for the type. For a rather messy example:

int[..] get_x(SlicePair& p);
int[..] get_y(SlicePair& p);

What are the paths inferred by the compiler for get_x and get_y above? It’s a bit weird at first glance:

int[..] <?p1 & ?p2> get_x(SlicePair <?p2> & <?p1> p);
int[..] <?p1 & ?p2> get_y(SlicePair <?p2> & <?p1> p);

There are two paths to consider for “p”. The path to the SlicePair itself (p1) and the path that the SlicePair references (p2).

There’s no way for the compiler to know which of the two paths matters for the output, so if the SlicePair goes out of scope, the output of get_x and get_y is no longer valid even if the actual slices stored in the SlicePair are perfectly valid.

If we had explicit lifetime annotations, the user could be fully specific in this case:

int[..]<'a> get_x(SlicePair<'a,'b> &<'c> p);
int[..]<'b> get_y(SlicePair<'a,'b> &<'c> p);

This is how it works in Rust. Many times the compiler can figure out the lifetime annotations for you, but not always. When it can’t, as in the example above, you better understand how lifetime annotations work. You also always need to write them out by hand for struct declarations, which gets pretty annoying.

By lacking any syntax for lifetime annotations we avoid functions with complex declarations like the above, but are also more limited in the programs we can express. Personally I prefer to avoid the complexity that lifetime annotations introduce.

Conclusion

Well, there you have it, a memory safe C-like language through the use of compile-time checks (except for bounds checks and escape hatches like raw pointers of course).

Note that we didn’t really mention “ownership”, “moves” and “borrowing” except when contrasting with Rust terminology. We have no destructors9 nor lifetime annotations. There’s quite a bit of design space to work with.

Single-use (linear) types alongside scoped references with implicit path tracking (as done in C#) provide a simpler alternative to Rust’s borrow checker implementation.

Hopefully this clarifies how and why a “borrow checker” works the way it does and the options one has when designing one. I find Vale’s idea of mixing single-use (linear) types with generational references to be a very compelling alternative to borrow checking with minimal runtime overhead.

Note that a default constructor achieves the same purpose as ZII1 (albeit less efficiently), the idea of returning a safe default on error is the bit I’m concerned with here, the zeroing itself is not relevant.

I think this is a silly way to handle errors, mostly stemming from C-language deficiencies. Zero being a useful default is a good idea, specially in C, that’s not what I have issue with. My problem is specifically with returning a default, “harmless” value (that just happens to be “zero” in this case) as the output of every fallible function, and continuing to do useless work with that value.

In Object-Oriented languages this would be an instance of the NullObjectPattern. Same idea, different coat of paint. I don’t like it.

Dealing with errors doesn’t require checking for them everywhere, nor to keep doing useless computations with a “harmless default” when an error occurs, even in C. There are situations where “null objects” are the most convenient solution to the problem, but much like exceptions using them everywhere just leads to a mess.

Zero being Useful != Errors being Useful

To make things a bit clearer I’ll use an example from the Our Machinery blog, which is where I first saw the idea of applying ZII to error handling. A lot of the article is about how much nicer using 0 as a default is in C, which I 100% agree with, so let us skip to the part I have a beef with. The example uses the following function:

uint32_t find_bone(const char *name);

This function returns the index of a data structure with information on a particular human bone, stored in a global array, given that bone’s name. So passing in “femur” would give you the index where the information on femur is stored.

If I pass in the name “fenur”, which is not a valid name for a human bone, the function has to do something about it. The suggestion in the blog is to return index 0, which happens to contain a special “no bone” case. That “no bone” is a “null object” to use OOP terminology. That means I can do:

uint32_t femur_id = find_bone("fenur");
float femur_length = bone_length(femur_id);

Getting the bone length will always work no matter what the output of find_bone is. In this case it’ll return 0, which is the length of “no bone”. Program doesn’t crash, control flow is kept simple, job’s done right? Not really no.

Garbage In, Garbage Out

Lets assume that find_bone does nothing but return 0 (i.e., “no bone”) when a bone is missing, because if it does anything else (like record the error somewhere) then that’s the actual error handling part. We’ll cover that case later. For now, assume find_bone does nothing besides return index 0, aka “no bone”, in case of an error.

This special “no bone” (a null object) is a valid bone in the sense that every operation on bones works on “no bone”. I can get the length of a “no bone”, I can get the position of a “no bone”, I can get the name of a “no bone”, I can render a “no bone”, whatever.

But I wanted the femur. The find_bone function “handled” the error by returning garbage, and the rest of the program kept on going like nothing happened.

Garbage! From King Garbage! Of the Garbage Dynasty! Stupid dog, always bringing garbage into the house.

— Eustace Bagge

If I get the length of a “no bone”, which is not the length of a femur, that length is also garbage. Any calculation involving that length is also garbage. And so on and so forth. Some people think that because “no bone” is a valid bone, this somehow is not garbage data. I don’t understand their reasoning. NaNs are valid floats, but they’re unwanted garbage nearly all of the time.

If we assume that the job of the application was to render a human skeleton, then the resulting display would have oddly short and deformed legs. Some people consider this an acceptable outcome, they’ll talk about displaying a giant red ERROR2 texture or similar instead but it’s the same thing really.

You didn’t handle the error, you ignored it. Your program “works” in the presence of an error by producing results the user didn’t ask for. Sometimes a half-working state can be beneficial, a lot of the time it is not.

Deal with Errors, but Later.

While some people will unironically claim “null objects” as the be-all-end-all solution to error handling, most people are using them as part of a larger strategy. For an example of the latter see this article by Ryan Fleury: The Easiest Way To Handle Errors Is To Not Have Them3.

I mentioned before that the find_bone function could have recorded the error somehow. Instead of just returning “no bone”, find_bone will additionally store the information that “fenur” doesn’t exist in some error log.

Some other part of the program will later go through this log and display the error to the user. This allows you to write your whole program without adding error checks everywhere, and still deal with the error. The result is a program with much simpler control flow, which in turn makes the code a lot easier to follow.

For C in particular, because all the code always executes, that means also all the resource management code executes, which is extremely valuable4.

I can definitely accept that reasoning, but that is a workaround for a language deficiency. What you wanted to do is abort the “task” that caused the error, reverting the program to the point right before the task started.

The “work” the program did meanwhile didn’t really achieve anything useful for the user. Best case scenario it amounted to a lot of “NOP”-ing around, worst case there is now garbage state all over the place the user will have to cleanup somehow.

Avoiding the latter takes discipline. It’s not enough to just return “null objects” everywhere and pray it all works out, that results in a bad user experience.

My suggestion if you do this is to record the error right where it occurs, let the null object propagate until the end of the function, then check if an error occurred. Don’t let it propagate further than that. You can extend “a function” to a larger task but make sure no null objects escape that task by getting stored somewhere.

Additionally, make sure you add checks or asserts for these null objects in every IO function. You don’t want to be sending them over the network and such. Remember, we’re talking about the case where they’re “errors”, not just a nice default.

What are “errors” to begin with?

There are two major sources of “errors”: bugs (unexpected errors) and partial functions (expected errors). You can consider program state as an additional input to a function for the latter case, to keep things simple.

I have another article on the fault (bug) → error → failure pipeline5, but the basic idea is that a bug is one or more erroneous lines in the program. You can’t do anything about a bug after you’ve already launched the program. Those lines, when executed, will cause an error, which is some invalid data or state. You can’t do anything about that directly either, since you didn’t expect it. That data or state will propagate and transform until it manifests as some user visible glitch or trigger some assertion in the code, resulting in a failure.

Can’t do anything about glitches (the user will have to report them). The only thing you can do with a failure is a generic “crash handler” that attempts to restore the program to a previous valid state and records as much information as it can to help debug the source of the failure (i.e., to figure out the actual bug that led to it)6.

Unexpected Errors → Crash Handler. Moving on.

Partial Functions

A partial function is a function that is only defined for a subset of its input set. Division is the classic example, its result is undefined when 0 is given as the divisor.

On a computer we can’t just leave things “undefined” like that, we need to decide what happens when 0 is passed in. CPUs do different things for integer and floating point division, and as programmers we too have different options when dealing with any partial function:

1. We can restrict the function’s input set, for example by only allowing non-zero integers as input, and relying on the type system to prevent compilation if 0 is passed in7. Non-nullable pointers are a great example of this approach.

2. We can check if the input is within the subset for which the function is defined as a precondition or assertion, triggering a “panic” if a value outside of that subset is given. We’re treating a particular input as being a bug in this case, meaning the actual error handling will be done as with any other bug, with a crash handler.

3. We can expand the output set of the function and map the inputs for which the function is not normally defined to this larger set. This is your Optional<T> or Result<T, E> style solution, or multiple return values as in Go8.

4. We can map inputs for which the function is not normally defined to one or more existing members of its output set, preferably unused ones. Division by 0 could return 0 for example9, I think Elm does this.

Continuing with division as an example, these are the options in pseudo-code:

1. int div(int a, nz_int b) { ... }
2. int div(int a, int b) { requires(b != 0); ... }
3. int? div(int a, int b) { if (b == 0) return NONE; else ... }
4. int div(int a, int b) { if (b == 0) return 0; else ... }

All options are valid, there is no option that’s universally better, but some options are better than others in different situations.

Depending on the language you’re using some options might be rather cumbersome. Options 1 and 3 are pretty terrible in C for example. I guess that’s a good reason to go for 4, but that doesn’t make it a good thing, it just makes C bad.

Error Handling the Whinesalot Way

My suggestion is to do the following: Pick Option 1 (restrict the input set) if it’s easy enough to do. The typechecker is your friend, use it. If that just isn’t practical, pick between Options 2 (assertion) and 3 (extended output set) depending on the case. Division by 0 is invariably a bug (so 2), but I might pass a non-existent key to a hash-table to check if it is stored there (so 3). Option 4 should only be used for languages that can’t do 3 properly (like C), and it should be used as if it were 3. Meaning “no bone” is an error case and it should be handled immediately.

What do I mean by dealing with an error “immediately”? Does that mean showing an error message right away? No, it means aborting the erroneous task.

First, should find_bone follow option 2 (assertion) or 3 (expanded output set)? It depends. Where are its inputs coming from? Is it only called in one place? Or in multiple places for different purposes?

Consider for example that it’s used to get data for some fixed set of bones in a help page. The bones are hardcoded and written by the programmer, if one is missing it’s almost certainly a bug. In this situation it’s more convenient for find_bone to do option 2 since needing to check if a bone I know is there is actually there is just noise.

But what if the name of the bone is given in a search box by the user? They can make a typo, or pass in a real bone name that just so happens to not be stored in the application’s “bone database”. In this situation it’s more convenient for find_bone to do option 3, so I’m reminded that I need to handle the situation of the bone being missing and provide the user with an appropriate message (thanks typechecker!).

You see this duality in many APIs, for example Python has both dict[key], which throws an exception if the key is missing (option 2) and dict.get(key), which returns None if the key is missing (option 3).

Since Option 2 is always dealt with the same way, what matters is how to program with partial functions that use Option 3.

I (know/don’t care/don’t know) if I’m right

Sometimes an API only provides Option 3, but you “know” you’re passing in a valid value. In that case use whatever solution the language provides to assert this. You can use .unwrap() in Rust for example, or try! in Swift. I’m not a fan of these noisy methods/operators, I much rather have two separate functions, one that uses Option 2 (meaning crash on wrong input) and one for Option 3, which returns an error value.

Sometimes if something is missing, you don’t really care, as it can be safely replaced by a default. An example would be filling in a default avatar if a user doesn’t provide one. Hopefully you’re using a language that makes this easy to do, like C#:

var avatar = user.GetAvatar() ?? new DefaultAvatar();

Note that the default avatar might be a generated image, rather than the same image for every user. In other cases what makes a good “default” might be different, even for the same function, when used in different parts of the program. Having “zero” be the default is not necessarily the answer here.

The big red “ERROR” texture can also apply in this case, but only if that is an acceptable default texture when a texture is missing, it is not ok as a result if the texture was actually necessary for the task, as in the bone example.

The last case is where you are hoping the function succeeds, but you can’t know if it will a priori. This is the interesting one, sorry it took so long to get here!

Error Values vs Null Objects

If a function may return an error result, and that result cannot simply be replaced with a useful default (not a useless “null object”), then you need to do the following:

1. Abort the “task” being attempted, whatever that may be. A “task” may or may not be a larger unit than a single function. This doesn’t mean crashing the program, it means returning to the last working state.

2. Inform the user of the error, either immediately, or by recording it in some error set that will later be displayed to the user in bulk.

As an example, consider a type checker, which is actually one of the best fit problems for the “null object” solution10, since we typically want to collect multiple errors rather than just stopping at the first one (at least in an IDE setting).

To make the difference clear between my suggested error handling approach and the “null object” approach, consider typechecking the add operator with implicit conversions between floats and integers. We start with the “null object” version:

Type top(Ctx* c, Type a, Type b) {
  // needed to prevent recording a mismatched type error for ERRORs
  if (a == ERROR || b == ERROR) { return ERROR; }
  if (a == b) { return a; }
  if (a == INT && b == FLOAT) { return b; }
  if (a == FLOAT && b == INT) { return a; }
  record_mismatched_type_error(c, a, b);
  return ERROR;
}

Type typecheck_add(Ctx* c, ASTNode* l, ASTNode* r) {
  Type left_t = typecheck(c, l);
  Type right_t = typecheck(c, r);
  Type result_t = top(c, left_t, right_t);
  // this check is needed to avoid redundant error messages
  if (!result_t) { return ERROR; }
  if (!(result_t == INT || result_t == FLOAT)) {
    record_expected_type_error(c, result_t, "a numeric type");
  }
  return result_t;
}

Note how we must check anyway for ERROR to ensure we don’t keep recording error messages that ERROR is not a numeric type or whatever. That check could be done on the various “record” functions as well (and should be there instead), but it has to happen somewhere, and we need to remember to do so.

Either way, if we naively use null objects without any checks, the compiler output will be full of useless errors! With error values, we’d do the following:

OptionalType top(Ctx* c, Type a, Type b) {
  if (a == b) { return some(a); }
  if (a == INT && b == FLOAT) { return some(b); }
  if (a == FLOAT && b == INT) { return some(a); }
  record_mismatched_type_error(c, a, b);
  return error();
}

OptionalType typecheck_add(Ctx* c, ASTNode* l, ASTNode* r) {
  OptionalType left_t = typecheck(c, l);
  OptionalType right_t = typecheck(c, r);
  return_if_error(left_t, right_t);
  OptionalType result_t = top(c, left_t.value, right_t.value);
  return_if_error(result_t);
  if (!(result_t.value == INT || result_t.value == FLOAT)) {
    record_expected_type_error(c, result_t.value, "a numeric type");
  }
  return result_t;
}

In this case we never do computation with OptionalType, all functions take Type as input, OptionalType is only ever used as a return type. Because of this we need to turn OptionalType into Type before using it. The typechecker makes sure we never forget to do this. If not for needing to record multiple errors, I wouldn’t even bother with OptionalType, I’d just exit() the moment I hit an error.

Additionally, no unnecessary computation happens. The top function is never called if either the left or the right node failed to typecheck. In this small example it makes no difference, but it can have an impact in a larger application. Imagine instead of a typechecker we were writing a JSON parser, what use is there to keep going?

The important point is that we got additional type safety for little additional effort, and most of it is due to C not helping what so ever. Consider Rust instead:

fn top(c: &Ctx, a: Type, b: Type) -> Option<Type> {
  if a == b { return Some(a); }
  if a == INT && b == FLOAT { return Some(b); }
  if a == FLOAT && b == INT { return Some(a); }
  record_mismatched_type_error(c, a, b);
  return None;
}

fn typecheck_add(c: &Ctx, a: &ASTNode, b: &ASTNode) -> Option<Type> {
  let left_t = typecheck(c, l);
  let right_t = typecheck(c, r);
  let result_t = top(c, left_t?, right_t?);
  if (!(result_t? == INT || result_t? == FLOAT)) {
    record_expected_type_error(c, result_t?, "a numeric type");
  }
  return result_t;
}

Other than some extra noise with Some and ?, this is just as simple as the “null object” case where the check for ERROR is hidden in the record functions (except here we don’t even need to do so). We’ll never end up with a “mismatched types: ERROR and INT”, we’ve structured the code in a way that makes it impossible! We cannot forget to handle the error, and neither can the intern.

If the language allowed implicit conversions between Type and Option<Type>, then we could get rid of those noisy and unnecessary Some(x), and if it implicitly added the “?” operator when attempting to convert Option<Type> to Type, then the code would be completely clean11.

The “null object” solution is not some novel discovery, it’s a workaround for poor error handling in your language of choice. If you use it, use it properly.

Well, the latter is not exactly true. C++, Rust and Ada all have reference counted pointers in their standard library, and reference counting is a type of garbage collection. Both Swift and Nim use automatic reference counting as their default memory management strategy.

Interestingly Swift’s predecessor Objective-C, older versions of Nim, and early versions of Rust all supported tracing garbage collection! Yet they all abandoned it.

There are many reasons as to why, but I think it boils down to a few key points:

• Reference counting, at least in its simplest form, is deterministic and plays well with scope-based resource management. Objects are freed immediately after the last reference to them is dropped. A typical tracing garbage collector is non-deterministic, you don’t know when it will kick in and free things.

• Reference counting, at least in its simplest form, is entirely local. The less you use of it, the less of an impact it has in your program. A typical tracing GC needs a hefty runtime, safepoints inserted into functions, etc., even if it is barely used.

• Reference counting, at least in its simplest form, plays well with C, and therefore with anything that talks to C (i.e., it is good for FFI). A typical tracing GC needs its runtime to be set up by the host, or for a copy of it to be bundled with every FFI-exported library.

Note that I specifically mentioned reference counting in its simplest form, which happens to be the reference counting implementation used by all of these systems programming languages. They all have plenty of compile-time optimizations, sure, but the runtime component is bog standard reference counting. No fancy concurrent, coalesced, deferred reference counting or anything like that, because it would have the same issues as a tracing GC.

According to A Unified Theory of Garbage Collection, reference counting and tracing garbage collection are just two ends of a spectrum. The more you optimize one or the other the more they meet in the middle. As far as “systems programming languages” are concerned, those optimizations have an unacceptable cost.

But that got me thinking. If modern tracing garbage collectors are “in the middle of the spectrum”, and that is bad, and simple reference counting is “on one end of the spectrum”, and it is ok… what about the other end? Can we make “the simplest form” of tracing garbage collection work for systems programming?

Back to Basics

With how fancy modern tracing garbage collectors are, we tend to forget that the original algorithms were extremely simple.

Mark-and-Sweep is just graph reachability. Starting from a set of roots, find all reachable nodes and mark them. Go through the graph again and free every node that isn’t marked.

It’s not particularly efficient, but it’s extremely simple to implement. So simple in fact that you could just implement your own mark-and-sweep garbage collector for whatever graph-like data structures you use.

The Yices2 SMT solver, written in C, does this for example. It has a mark-and-sweep garbage collector you can invoke at any time to free no longer used expression nodes. Note that the collector is never implicitly called, there’s no runtime, it’s just a function that Yices makes available to the host.

For the set of roots it doesn’t need to do any fancy stack scanning. Any expression stored in a model is a root, and any root expressions not stored in a model can be passed in as an extra argument.

We’re so blinded by these fancy incremental, concurrent, generational garbage collectors that we forget just how simple tracing garbage collection really is.

Yices2 proves that tracing garbage collection can be used in a systems programming context. The host calls the GC if and when it wants to, no runtime.

But can we generalize this? Having each library implement their own custom mark-and-sweep tracing GC is fine and all, but I’m wondering how this could work as a general memory management strategy.

Just taking an existing, modern garbage collector and making it cleanup only when explicitly called does not work very well, they’re incremental for a reason (in fact pretty much every tracing GC-ed language tells you not to do this). But we don’t need any of that fancy-schmancy stuff, we’re going back to 1970.

Garbage Collected Arenas

While the languages mentioned previously all use a mix of scope-based resource management and reference counting, in C that’s actually not so common since it has to be done manually. Needing to manually increment and decrement reference counts is not only pretty annoying it is also very error prone.

The more common memory management approach is to use Arenas, also known as Bump Allocators. You throw a bunch of allocations into a big bucket, and then you free them all in one go. Allocation and deallocation are both extremely fast, but deallocation is delayed to when it is convenient for the program, rather than immediately when an object becomes inaccessible.

You know, a bit like how a tracing GC works.

Yup, that’s what we’re doing. Alongside free(&arena) we’re adding a new collect(&arena, &roots) function in the style of Yices. Collect does the same thing as free, but it copies all data accessible through the roots to a new bucket (adjusting the pointers in the process). This is called a moving collector, which has various advantages compared to mark-and-sweep:

• Allocating is just a pointer bump on the arena, as fast as can be.

• Freeing is just resetting the arena pointer. That means inaccessible data, no matter how much or how big, has zero impact on the garbage collector2.

• Data also gets compacted when copied, avoiding fragmentation and improving memory locality.

• The particular algorithm we’ll use, Cheney’s Algorithm, is iterative. It uses no recursion what so ever.

So what’s the catch? Well, copying can be slow, but beyond that not much. Having to pass the roots manually is a bit annoying but Yices shows that it can work. Not an issue for a custom language that uses this as its memory management strategy, since it can easily compute the roots.

There’s an extra cost per allocation, however, a small header has to be prepended with the following information:

• The size of the allocation.

• A pointer to a “tracing/moving” function or to a data structure with information about the pointers contained in the allocation.

• A forwarding pointer for the case where the same data is reachable from multiple roots but has already been copied.

The forwarding pointer is only needed during collection, so with some bit fiddling we can get away with an 8 byte header. Reference counting needs 8 bytes for the reference count, another 8 if there are weak references, plus however much is used by the malloc header.

The tracing/moving function is equivalent to the destructor in the reference counting case, with the advantage that it is not recursive.

Lastly, the set of roots has to be stored somewhere to be passed to the collect function. In a custom language, this can be implemented very easily following the approach outlined in this paper: Accurate garbage collection in an uncooperative environment.

Basically, you keep a shadow stack of pointers to garbage collected data. If the language has an effect system, you can restrict this shadow stack to be manipulated only by functions that directly or indirectly call collect. You could, for example, only allow applications (not libraries) to ever call collect3.

Putting it all together

Start from a language like Zig or Odin where every function that allocates takes an allocator as input (implicitly or explicitly). From the function’s point of view, this allocator is just a bog standard arena that asks for some weird metadata. You could just as well pass in a normal, non-garbage collected arena to this function if desired, and you can have as many distinct arenas as makes sense for your application.

But, if you don’t need that level of control, you can also just pass in the default arena. This arena acts just like a regular arena, except it stores some extra metadata alongside each allocation. Again, from the function’s point of view, there’s no difference beyond needing to pass in a pointer to the metadata (can be compiler generated or written by hand, doesn’t matter).

Lastly, the application (not libraries!) can call collect at any point to clean up the default GC arena. The compiler needs to maintain a shadow-stack for every function in collect’s call stack. This can be easily tracked with a simple effect system. In the case of another language like C serving as the host, it can just pass the root set explicitly to collect.

And we’re done. The only cost is the extra header 8 byte header per allocation and a tracing function per type that stores GC pointers. Otherwise the application can choose to never call collect at all and just call free directly and everything will still work. The garbage collection is (almost) entirely optional4.

This honestly sounds too good to be true, so I’m wondering why it hasn’t been done before. The closest attempt I can think of is Cone. Maybe I’ll have to be the one to put it to the test in my own programming language. For now, here’s a GC Arena implementation in C (still needs improvement).

Update: This article was discussed on hackernews

Error handling and resource management in C can be a total pain in the butt. Consider for example a program that needs to split a log file into three separate files for “info”, “warning” and “error” logs respectively:

FILE* log_file = fopen("log.txt", "r");
if (!log_file) {
  return -1;
}

FILE* info_log_file = fopen("info_log.txt", "w");
if (!info_log_file) {
  fclose(log_file);
  return -1;
}

FILE* warning_log_file = fopen("warning_log.txt", "w");
if (!warning_log_file) {
  fclose(info_log_file);
  fclose(log_file);
  return -1;
}

FILE* error_log_file = fopen("error_log.txt", "w");
if (!error_log_file) {
  fclose(warning_log_file);
  fclose(info_log_file);
  fclose(log_file);
  return -1;
}

// ...
// code to actually read and write the files goes here.
// if any operation fails we also need to "cleanup" there.
// ...

fclose(error_log_file);
fclose(warning_log_file);
fclose(info_log_file);
fclose(log_file);
return 0;

Copy-pasting those fclose calls around everywhere is not only annoying but extremely error prone. The above is the simplest example. Imagine we only want to create the various output files if one of the corresponding log type lines appears in the original file, or only create and write to one of the files if a certain configuration option is set. There might be memory management (malloc/free) happening in between all of this for whatever reason.

In this post I’m going to show you a really neat macro trick that makes dealing with resource management as pleasant as it can get in standard C1.

What do other languages do?

Most programming languages provide a built-in mechanism for resource management.

C++ and Rust have scope-based resource management, where each type may have an associated “destructor” (called drop in rust) that is implicitly inserted by the compiler at the end of the scope where the variable storing the resource resides. When the scope ends resources are cleaned up, no matter if it happens “naturally”, due to an exception, or a return statement.2

This solution is particularly nice because there is nothing for the consumer of the API to worry about, it just works, but it requires that object lifetimes be tied to scopes.

Languages like C#, Java and Python have special resource handling blocks: using statements in C#, try-with-resource statements in Java, and with statements in python3, which implicitly call a cleanup function at the end of their scope. In C# it’s also possible to have using statements not introduce a new scope, tying the resource to the current scope instead.

Instead of calling the destructor, a specific method from the language’s “cleanup protocol” is called: Dispose() in C#, close() in Java and __exit__() in Python.

This solution has the disadvantage that the consumer of the API must remember to use the resource handling statement for cleanup to occur, but it allows regular objects to be managed in a non-scope oriented manner.

Languages like Go, Zig, Odin, and C3 support a defer4 statement that allows to specify the cleanup action, rather than implicitly invoking some standard cleanup protocol method or destructors:

// Go
log_file, err := os.Open("log.txt")
if err != nil {
  return nil, err
}
defer log_file.Close()

info_log_file, err := os.Open("info_log.txt", "w")
if err != nil {
  return err
}
defer info_log_file.Close()

warning_log_file, err := os.Open("warning_log.txt", "w")
if err != nil {
  return err
}
defer warning_log_file.Close()

error_log_file, err := os.Open("error_log.txt", "w")
if err != nil {
  return err
}
defer error_log_file.Close()

// ...

return nil

This solution has the disadvantage that the user of the API must not only remember to use defer, but also to know which specific function to call.

An important thing to note is that Go’s defer works differently from the other languages mentioned. The defer of the other languages is very simple: it can be understood as “copy-pasting” the cleanup code right before the scope ends (including if it is exited by a return statement).

Go’s defer is more of an imperative command. The block of code it will execute works like a closure, capturing the values of the referenced variables at the time of the defer statement’s execution (not at cleanup!). It also runs the cleanup code at function end rather than scope end, so it requires extra storage. For example, a defer statement executed in a loop 10 times will add 10 cleanup instructions to the “cleanup stack”.

Go’s style of defer is less efficient and its behavior can be surprising5.

The advantage of defer is that it is fully explicit and gives the user the most control.

Fully explicit? Most control? Sounds like C to me. There are already some existing implementations of defer for C but I’m not happy with any of them:

• The reference implementation of the N2895 defer proposal is needlessly complicated, to the point of requiring an extra preprocessor. It relies on GCC extensions and setjmp + longjmp. It needs all this complexity to support features nobody asked for, IMHO. It is “Go style”.

• ceraii’s implementation is also quite complicated (though not to the same extent) and relies on setjmp + longjmp. It is “Go style”.

• moon-chilled’s Defer macro is the simplest but also requires either GCC extensions or setjmp + longjmp, plus a 32 element buffer at the start of each function to store the defers. It is “Go style”.

I don’t want any of this, I want a simple and efficient scoped defer like Zig, Odin, C3 and D have, but in pure standard C. I want the scoped defer of the N3199 proposal.

So until that gets accepted (i.e. never), it is time to get our hands dirty.

Goto Fail

The classic solution to do resource management in C is to (ab)use goto statements to organize things a little6:

int result = 0;
FILE* log_file = fopen("log.txt", "r");
if (!log_file) {
  result = -1;
  goto DEFER_0;
}

FILE* info_log_file = fopen("info_log.txt", "w");
if (!info_log_file) {
  result = -1;
  goto DEFER_1;
}

FILE* warning_log_file = fopen("warning_log.txt", "w");
if (!warning_log_file) {
  result = -1;
  goto DEFER_2;
}

FILE* error_log_file = fopen("error_log.txt", "w");
if (!error_log_file) {
  result = -1;
  goto DEFER_3;
}

// ...

DEFER_4:
  fclose(error_log_file);
DEFER_3:
  fclose(warning_log_file);
DEFER_2:
  fclose(info_log_file);
DEFER_1:
  fclose(log_file);
DEFER_0:
  return result;

We avoid the copy-paste problem, but this has its own issues. The resource cleanup has to be explicitly written in last-in-first-out order at the end of the function. Instead of using return, it’s now necessary to use goto to jump to the correct cleanup point. Things gets even more complicated when certain resources are allocated and released within inner scopes. This technique is also partially responsible for goto fail.

A first attempt

Having only one resource cleanup call and the ability to jump around opens up some options. Lets reorganize the code a little:

int result = 0;
FILE* log_file = fopen("log.txt", "r");
if (!log_file) {
  result = -1;
  goto DEFER_0;
}
if (0) {
  DEFER_1:
    fclose(log_file);
    goto DEFER_0;
}

FILE* info_log_file = fopen("info_log.txt", "w");
if (!info_log_file) {
  result = -1;
  goto DEFER_1;
}
if (0) {
  DEFER_2:
    fclose(info_log_file);
    goto DEFER_1;
}

FILE* warning_log_file = fopen("warning_log.txt", "w");
if (!warning_log_file) {
  result = -1;
  goto DEFER_2;
}
if (0) {
  DEFER_3:
    fclose(warning_log_file);
    goto DEFER_2;
}

FILE* error_log_file = fopen("error_log.txt", "w");
if (!error_log_file) {
  result = -1;
  goto DEFER_3;
}
if (0) {
  DEFER_4:
    fclose(error_log_file);
    goto DEFER_3;
}

goto DEFER_4;
DEFER_0:
  return result;

Yikes that looks even worse! But this was an important change, now the cleanup code is right next to the initialization code, right where defer would be.

Lets try making some macros:

#define CAT(A, B) A##B
#define defer_return(I, V) do {goto CAT(DEFER_, I)} while(0)
#define defer(I, F, P) if(0){CAT(DEFER_, I): F; goto CAT(DEFER_, P)}
#define run_deferred(I) goto CAT(DEFER_, I); DEFER_0: return result;

int result = 0;
FILE* log_file = fopen("log.txt", "r");
if (!log_file) {
  defer_return(0, -1);
}
defer(1, fclose(log_file), 0);

FILE* info_log_file = fopen("info_log.txt", "w");
if (!info_log_file) {
  defer_return(1, -1);
}
defer(2, fclose(info_log_file), 1);

FILE* warning_log_file = fopen("warning_log.txt", "w");
if (!warning_log_file) {
  defer_return(2, -1);
}
defer(3, fclose(warning_log_file), 2);

FILE* error_log_file = fopen("error_log.txt", "w");
if (!error_log_file) {
  defer_return(3, -1);
}
defer(4, fclose(error_log_file), 3);

run_deferred(4);

Better, but not good enough. Having to manually count the defers like this is really annoying. We need to get fancier.

Duff’s device to the rescue

There’s no good way to “count” in the C preprocessor to avoid the issue above. The closest thing we have is the __LINE__ macro which stores the current line, but there’s no way to refer to the “last line” in which we deferred.

Rather than trying to do everything at the preprocessor level, we’ll shift some of the work to runtime7. Let’s rework the example above to use a duff’s device-like switch statement instead of straight gotos:

#define defer_return do {goto _d_start;} while(0)
#define defer(I, F) {_d=I; if(0){case I: _d--; F; goto _d_start;}}
#define defer_block int _d = -1; _d_start: switch(_d){case -1:
#define run_deferred() goto _d_start;}

int result = 0;
defer_block {
  FILE* log_file = fopen("log.txt", "r");
  if (!log_file) {
    result = -1; defer_return;
  }
  defer(1, fclose(log_file));

  FILE* info_log_file = fopen("info_log.txt", "w");
  if (!info_log_file) {
    result = -1; defer_return;
  }
  defer(2, fclose(info_log_file));

  FILE* warning_log_file = fopen("warning_log.txt", "w");
  if (!warning_log_file) {
    result = -1; defer_return;
  }
  defer(3, fclose(warning_log_file));

  FILE* error_log_file = fopen("error_log.txt", "w");
  if (!error_log_file) {
    result = -1; defer_return;
  }
  defer(4, fclose(error_log_file));
} run_deferred();
return result;

Much better, now we only need 1 count in the defers. The basic idea is that each call to defer sets the defer count (_d) equal to I. It also adds an if(0) block that has the corresponding switch case label in its body. Inside the body it executes the action, decrements _d, and jumps to the switch again.

So once the first jump back to the switch occurs, the defer blocks get executed in reverse order, until _d becomes 0 which isn’t a valid case and the switch stops.

The syntax:

defer_block {...} run_deferred();

is modeled after:

do {...} while(...);

Which I personally find makes it quite easy to understand. But this is still not good enough! If a user writes the wrong value in the defer count, all hell breaks loose. We’re also missing inner scope support for proper scoped defer.

Rather than track the current defer by incrementing and decrementing, we’ll track where the defers occur by storing their __LINE__, and we’ll track the previous defer through an auxiliary variable called _d_p:

#define defer(F) {int _d_p = _d; \ 
  _d = __LINE__; if(0){case __LINE__: _d = _d_p; F; goto _d_start;}}

The _d_p (defer previous) variable is declared inside a new scope, such that it shadows any previous usage of the variable. It remembers the last value of _d, before _d is set to the current line. Now we can use the line as the case constant, avoiding explicit defer counting. Instead of decrementing, we set _d back to its previous value through _d_p. The last thing we need to ensure is that when the first _d_p is set, it points to “nowhere” so the switch ends, we do this by setting _d right after the switch starts to an impossible line:

#define defer_block int _d = 0; \ 
  _d_start: switch(_d){case 0: _d = -1;

We’ve now gotten a single level of defer working with no manual counting.

Scoped defer

The last thing we want to support are nested scopes, where you can execute a subset of defers for a particular scope without running the whole thing. We also need defer_return to run every defer from the current scope all the way to the top level defer block.

The first thing to do is introduce a way to open a new defer scope. We’ll do this with the most complicated trick so far:

#define _DEFER_SNAPSHOT(A) {int _d_p = _d; \ 
  _d = __LINE__; if(0){case __LINE__: _d = _d_p; A;}}
#define defer(F) _DEFER_SNAPSHOT(F; goto _d_start)
#define defer_scope
  do { _DEFER_SNAPSHOT(if(_d_r){goto _d_start;} else {continue;})

We factor out the logic of defer into a _DEFER_SNAPSHOT macro which is shared with defer_scope. The idea is that defer_scope behaves like defer, in that it creates a “jump point”, but rather than executing an action, it makes a decision based on a new _d_r (defer return) variable:

• If _d_r is true, it continues the “unwinding” process.

• If _d_r is false, it jumps out of its do {} while(0) loop with continue.

This variable is set by the updated defer_return. We also add a defer_break alternative which is used to end only the current scope:

#define defer_break { goto _d_start; }
#define defer_return { _d_r = 1; goto _d_start; }

We make a small change to run_deferred, such that it works for both the top level defer block and the inner defer scopes:

#define run_deferred() goto _d_start; } while (0)

For the inner scopes, the while(0) ends the do loop. For the top level block, it’s just an extra while(0) that does nothing.

Lastly, we need to initialize the _d_r variable when we start the defer_block:

#define defer_block int _d_r = 0; int _d = 0; \ 
  _d_start: switch(_d){case 0: _d = -1;

And we’re done, scoped defer:

int result = 0;
defer_block {
  FILE* log_file = fopen("log.txt", "r");
  if (!log_file) {
    result = -1; defer_return;
  }
  defer(fclose(log_file));

  defer_scope {
    FILE* info_log_file = fopen("info_log.txt", "w");
    if (!info_log_file) {
      result = -1; defer_return;
    }
    defer(fclose(info_log_file));
    
    FILE* warning_log_file = fopen("warning_log.txt", "w");
    if (!warning_log_file) {
      result = -1; defer_return;
    }
    defer(fclose(warning_log_file));

  } run_deferred();

  FILE* error_log_file = fopen("error_log.txt", "w");
  if (!error_log_file) {
    result = -1; defer_return;
  }
  defer(fclose(error_log_file));
} run_deferred();
return result;

Conclusion

Unlike the little macro I used to implement generators in C, this one is a bit… much. But it’s also substantially simpler and more efficient than the longjmp based approaches, so if you want to have defer in C, try this out.

If you are ok with sticking to GCC (or clang), then __attribute__((cleanup)) is the way to go, it just works, even if it only allows calling a function with a specific prototype rather than an arbitrary block of code.

for i := 0; i < 100; i++ { 
  mutex.Lock()   
  defer mutex.Unlock()
  ...
}

FILE* f1 = fopen(...);
if (!f1) {
  goto fail;
}
FILE* f2 = fopen(...);
if (!f2) {
  goto fail;
}
fail:
  if (f2) fclose(f2);
  if (f1) fclose(f1);

We, as a nation, have the ability – and the responsibility – to reduce the attack surface in cyberspace and prevent entire classes of security bugs from entering the digital ecosystem but that means we need to tackle the hard problem of moving to memory safe programming languages.

— National Cyber Director Harry Coker.

I’m honestly surprised it took this long for government to start getting involved, considering how long memory safety issues and the resulting security vulnerabilities have made a mess of things.

Some of the most infamous cyber events in history – the Morris worm of 1988, the Slammer worm of 2003, the Heartbleed vulnerability in 2014, the Trident exploit of 2016, the Blastpass exploit of 2023 – were headline-grabbing cyberattacks that caused real-world damage to the systems that society relies on every day. Underlying all of them is a common root cause: memory safety vulnerabilities.

— Anjana Rajan, Assistant National Cyber Director for Technology Security.

The online discourse quickly devolved into “just rewrite it in Rust bro!”1 which I find a bit unfortunate because it is a relevant report and the use of memory safe languages is just the main recommendation, other possibilities are given. I urge everyone to actually read the report and not just immediately jump to conclusions.

One of the most important statements in the report, I feel, is the following:

However, even if every known vulnerability were to be fixed, the prevalence of undiscovered vulnerabilities across the software ecosystem would still present additional risk. A proactive approach that focuses on eliminating entire classes of vulnerabilities reduces the potential attack surface and results in more reliable code, less downtime, and more predictable systems.

Extensive testing is not going to cut it, we need to build our software and hardware systems such that certain subsets of security vulnerabilities simply cannot happen.

But ultimately it is we (meaning software developers at large) that need to do this. It’s not better tools that will do it, it is we. The report suggests tools that can help us achieve this goal, but it is all for nothing if the tools are misused or ignored.

It all comes down to convenience

Humans are lazy creatures, we tend to take the path of least resistance. Most memory safe languages (including every language recommended in the report) have escape hatches that allow you to make as many memory unsafe horrors as you want. But that way of doing things is less convenient than the memory safe way of doing things in those languages, and as such is rarely an issue.

You can have 🔥 blazing 🚀 fast memory vulnerabilities in 100% safe rust 🦀.2

Here, have a double free in C# without even using the unsafe keyword:

static void Main() {
    IntPtr ptr = Marshal.AllocHGlobal(4096);
    Marshal.FreeHGlobal(ptr);
    Marshal.FreeHGlobal(ptr);
}

Nobody in their right mind would do this, but they can.

So it’s not so much the availability of memory unsafe features that is the problem, but how convenient they are compared to safe alternatives. The safe route can sometimes get painful in Rust due to the borrow checker, but Rust’s community has a very strong culture of avoiding unsafe code unless absolutely necessary, going so far as pestering library developers about it on github.

That culture is what keeps Rust developers from taking the path of least resistance and reaching for unsafe3 after losing a single “fight” to the borrow checker.

C and C++ are not “memory unsafe” languages, they’re languages that lack built-in memory safe ways of doing the same job in a more convenient manner.

There’s not really much of a reason for this to be the case, beyond not enough people pushing for it versus the number of people that not only don’t care about security but are actively hostile towards even small safety-related improvements.

Do you even C bro?

All the way back in the prehistoric times of 2009, Walter Bright, creator of the D programming language and the Digital Mars C and C++ compilers, wrote an excellent article titled C’s Biggest Mistake.

That mistake, he argues, was conflating arrays with pointers, or more specifically arrays decaying into pointers when passed to functions, losing their size information in the process. And oh boy was he right.

Buffer Overflows are one of the most common exploits out there, and C and C++ are pretty much the only languages actually vulnerable to them in any serious capacity. C++ has taken steps to mitigate this problem with the introduction of std::string_view (C++17) and std::span (C++20), but these should have really been part of C++114.

Better late then never I suppose… Except that std::span’s index operator ([]) is unsafe, and its .at() method (which has been in std::vector and std::string since forever) is only coming in C++26. How did the entire committee forget the .at() method? It boggles the mind. So many talks about safety at cppcon and then they do this.

What about C then? Well, all you really need to protect against buffer overflows is something like the following:

typedef struct slice_i {
  int* data;
  size_t len;
} slice_i;

#define idx(s, i) (i >= 0 && i < s.len ? s.data : (abort(), s.data))[i]

Just copy paste slice_i around and change the _i and the type of the pointer as needed. Wouldn’t it be nice if you didn’t have to write these structs out by hand or use that hideous macro? If only a very smart gentlemen had suggested a minor syntactical addition to C all the way back in 2009 that could be used to make slices:

int a[..]

Simple no? If you try to index one of these things you get bounds checking. If you need extra performance you can always just cast it to a pointer. So what did C devs think of Walter’s suggestion? Lets check out a reddit thread from 2020:

I would have said overloading the 'break' keyword.

All other complaints about C are just, "Why do we need to breath oxygen?" It's just part of the landscape.

— which_spartacus

Ah yes, needing break in switch statements is definitely not just part of the landscape.

... *rolls eyes*

Fat pointers are pointless. If you want a fat pointer.. *gasp* make a struct of an integer and a pointer!

— okovko

The point flew so hard over this person’s head that Walter got a home run.

Another problem that can only be solved by writing good code.

— p0k3t0

It was so simple all along.

have to disagree. Since C has no implicit bounds checking (for performance reasons), there's no point in having the compiler know the size of an array / pointer. If you, the programmer, need that information, you can just pass the length explicitly.

— BioHackedGamerGirl

If only there was a convenient way to choose between having bounds checks or no bounds checks depending on the performance requirements of a particular piece of code. Many Buffer Overflows happen on non-performance-critical codepaths.

Next up ... Assembly Language's Biggest Mistake!

— nahnah2017

I think you get the point. 11 years after that article came out, this is the attitude. What do you think will happen if these developers are forced to use Rust? They’ll just use unsafe everywhere, completely defeating the point.

These people are why we’ve had like 5 attempts at a “strxcpy” function over the years instead of proper string handling functions in the standard library.

Even Linux kernel developers, whom you’d think would mainly consist of the absolute top tier of C developers, and would be expected to be highly security conscious, apparently don’t know how to implement a string buffer.5

What about Use-After-Free?

Safety and security are a spectrum, one where reaching 100% is sadly next to impossible. Preventing buffer overflows in C and C++ is almost trivial and would have avoided ~20000 known vulnerabilities.

Before worrying about trickier memory management concerns, don’t you think that should be priorities 1 through 20?6

Sadly, after that White House report, we’re already seeing discussions about adding substructural typing a la Rust to C. One of the silliest cases of putting the cart before the horse in recent years.

I’m not saying adding an owner pointer annotation to C wouldn’t be a good thing (I’d love that actually), I’m saying that’s not what you should be worrying about for now. If you can’t even add a slice type to the language how do you expect substructural typing to get accepted by the security unconscious jagoffs exemplified above?

There are also other effective ways of defending against Use-After-Free. Hardened memory allocators like SlimGuard are a thing. They leave a lot of performance on the table to also protect against buffer overflows, which they wouldn’t need to do if buffer overflows weren’t such a major issue in the first place!

Would hardened allocators like SlimGuard be more popular if the performance penalty was reduced from no longer needing overflow protection? Perhaps.

Safe Arenas and Pools

Thing is, if you’re coding in C, you shouldn’t be mallocing and freeing that much in the first place. You should be using Arenas to group multiple lifetimes together and freeing everything in one go. The less frees in your codebase, the less chances of a use-after-free. Is it 100% safe and applicable in all cases? No, but would you rather have ~20000 vulnerabilities or ~2000?

You can make Arenas nearly 100% safe by taking advantage of memory page protection features and only reusing old pages after exhausting the entire 64-bit address space. The chance of an attacker being able to exploit one of those page reuses is basically 0%.

On macOS, for example, you can do:

madvise(arena_ptr, size, MADV_FREE_REUSABLE);
mprotect(arena_ptr, size, PROT_NONE);

On Linux you’d probably use MADV_REMOVE and on Windows you’d use:

VirtualFree(arena_ptr, size, MEM_DECOMMIT);

Unlike with malloc the performance penalty of doing this on an arena is minuscule.

You can also use memory pools with generational handles if you need to allocate and free various objects of the same type with varying lifetimes in a 100% safe manner. Is it applicable in all cases? No, but would you rather have ~2000 vulnerabilities or ~200?

You see the point? Why are arenas and pools not part of the standard library if they would already help so much with the problem?

If you had slices, hardened arenas and generational pools, you could make C almost as “safe” as Rust7 through linter rules forbidding various unsafe features like pointer arithmetic, “decayed” arrays and direct usage of malloc/free without first setting #pragma unsafe or whatever.

Memory Safety != Security

PHP is a memory safe language. For many years PHP-based websites were a security nightmare due to SQL injection vulnerabilities because developers were concatenating SQL strings with unsanitized user input.

No amount of language safety features can protect against that, what can is providing safe alternatives that ensure the SQL query is built in such a way that user input cannot possibly affect it. That’s what every language (including PHP) has these days.

Java, another memory safe language, was a security nightmare in the Java Applet days. Recently there was the Log4Shell vulnerability, which involved a combination of very bad design decisions resulting in (quoting wikipedia):

The vulnerability's disclosure received strong reactions from cybersecurity experts. Cybersecurity company Tenable said the exploit was "the single biggest, most critical vulnerability ever,"^[18] Ars Technica called it "arguably the most severe vulnerability ever"^[19] and The Washington Post said that descriptions by security professionals "border on the apocalyptic."^[8]

Nothing can protect against a logging framework connecting to remote servers based on a format string beyond a culture of everyone and their mother screaming their lungs out against the addition of such a feature.

That’s not to say memory safety isn’t important, it is. But lacking security is ultimately an attitude problem. If developers don’t care about it, forcing onto them a memory safe language will not accomplish anything. They’ll screw up some other way.

First make them care, then make it easy for them to do the right thing.

While C does not natively support generators, we can implement them in a somewhat clever way by abusing a quirk of how the switch statement works1.

Why would you want to implement generators in C? I can think of a few reasons:

• You want to have a better understanding of how generators (or coroutines in general) work.

• You use C as a compilation target for your programming language and want to add support for yield.

• They can be really convenient!

Fibonacci Numbers

As a running example we’ll be turning a function that prints the fibonacci numbers into a generator function that yields the fibonacci numbers on demand. Here’s our starting point:

void fib(void) {
  int a = 0;
  int b = 1;
  int n = 1;
  while (1) {
    printf("%d\n", a);
    a = b;
    b = n;
    n = a + b;
  }
}

This is a simple iterative fibonacci implementation that prints out an “infinite”2 stream of the fibonacci numbers. Hopefully nothing too surprising.

Instead of having fib print the numbers out directly (or store them in a list or whatever else), we want it to lazily produce the next number in the sequence on demand, with the caller then doing whatever they want with it.

To achieve this we’ll turn fib into a coroutine, meaning a function that can be suspended and resumed3.

Stack Frames

First thing we need to do is store fib’s local variables outside of it. We can’t have fib’s local variables in registers or the regular C stack because when we suspend it we’re returning back to the caller, which will trample all over those registers and the stack before we get to resume fib.

The way to do this in C is to manually put all those local variables into a separate struct, reifying fib’s stack frame, and storing it elsewhere4:

typedef struct {
  int a;
  int b;
  int n;
} fib_frame;

void fib(fib_frame* f) {
  f->a = 0;
  f->b = 1;
  f->n = 1;
  while (1) {
    printf("%d\n", f->a);
    f->a = f->b;
    f->b = f->n;
    f->n = f->a + f->b;
  }
}

// usage
fib_frame f = {};
fib(&f);

All we’ve managed to do so far is make fib less efficient and more annoying to call, but now we have fib’s stack frame stored outside of the machine’s stack, which we will need to implement suspension.

Suspending a Function

In order to suspend fib, we need to store some additional information in its stack frame: the equivalent of the program counter. We need to know in which “instruction” fib stopped at after the yield:

typedef struct {
  // program counter (sort of)
  int _pc;
  int a;
  int b;
  int n;
} fib_frame;

We’ll also change fib so that instead of printing numbers it returns them:

int fib(fib_frame *f) {
  f->a = 0;
  f->b = 1;
  f->n = 1;
  while (1) {
    return f->a;
    // we want to resume here somehow
    f->a = f->b;
    f->b = f->n;
    f->n = f->a + f->b;
  }
  return -1; // we'll never get here
}

We need to store the address of the instruction after the return, and jump directly there the next time we call fib. If you’re thinking goto, you’re on the right track. For example, we can do the following:

int fib(fib_frame *f) {
  if (f->_pc == 1) { goto resume_1; }
  f->a = 0;
  f->b = 1;
  f->n = 1;
  while (1) {
    // store the resumption point before returning
    f->_pc = 1;
    return f->a;
  resume_1:
    f->a = f->b;
    f->b = f->n;
    f->n = f->a + f->b;
  }
  return -1; // we'll never get here
}

And then we can use fib as follows:

int main() {
  fib_frame f = {};
  for (int i = 0; i < 10; i++) {
    printf("%d\n", fib(&f));
  }
  return 0;
}

The fib function just keeps producing the next number in the sequence each time we call it, so if we call it 10 times, we get the first 10 fibonacci numbers.

Labels in C aren’t first class, so there is no way to store resume_1 directly in _pc, and goto in C cannot be used to jump to a numerical address5, hence the conditional at the start translating the number 1 to the appropriate jump.

If we had more yield points, the same pattern would repeat:

if (f->_pc == X) { goto resume_X; }
if (f->_pc == Y) { goto resume_Y; }
...
f->_pc = X; return something; resume_X:  // yield
...
f->_pc = Y; return whatever;  resume_Y:  // yield

Those ifs could be a switch statement instead:

switch (f->_pc) { 
  case X: goto resume_X;
  case Y: goto resume_Y;
}

And those yields are asking for a macro:

#define yield(PC, X, R) PC = X; return R; resume_##X:

// usage
while (1) {
  yield(f->_pc, 1, f->a);
  ...

If we standardize on the name of the frame parameter (f) and its program counter field (_pc), we can remove the first argument:

#define yield(X, R) f->_pc = X; return R; resume_##X:

Much better, here’s the full example now:

int fib(fib_frame *f) {
  switch (f->_pc) { 
    case 1: goto resume_1;
  }
  f->a = 0;
  f->b = 1;
  f->n = 1;
  while (1) {
    yield(1, f->a);
    f->a = f->b;
    f->b = f->n;
    f->n = f->a + f->b;
  }
  return -1; // we'll never get here
}

Still, needing to spell out the jumps at the start of the function and explicitly numbering the yield points is a bit annoying. Good enough for a compiler backend, but annoying for direct usage in C, the explicit stack frame is bad enough… However, we can do better by taking advantage of a feature that’s usually considered a wart in C.

Switch Fall-Through

Most languages, including newer members of the C-family, have their equivalent of a switch statement work like this:

switch (x) {
  case 1 {
    // do this for case 1
  }
  case 2 {
    // do this for case 2
  }
  case 3 {
    // do this for case 3
  }
}

C’s switch statement doesn’t work like this, instead it works like this:

switch (x) {
  case 1:
    // do this for case 1
  case 2:
    // do this also for case 1
    // do this for case 2
  case 3:
    // do this also for case 1
    // do this also for case 2
    // do this for case 3
}

This is called fall-through and is a common beginner trap and source of bugs for sleep deprived programmers that forget to put a break at the end of each case. C# retains the same syntax but always requires an explicit break to avoid this issue.

Basically, those cases work like our resume_X label from before. You can even put a case inside a block from another case and C won’t bat an eye:

// this works, what the hell C!
switch (x) {
  case 1: while (1) {
    case 2: if (0) {
      case 3: 
        ..
    }
  }
}

Nasty, but we can take advantage of this. Instead of the switch at the start of the function mapping a number to a goto statement, we can use it to jump directly:

int fib(fib_frame *f) {
  switch (f->_pc) { 
  case 0:
    f->a = 0;
    f->b = 1;
    f->n = 1;
    while (1) {
      f->_pc = 1;
      return f->a;
    case 1:
      f->a = f->b;
      f->b = f->n;
      f->n = f->a + f->b;
    }
  }
  return -1; // we'll never get here
}

Might seem like a small change, but the start of the function is now always the same, regardless of the number of yields:

switch (f->_pc) { 
  case 0:

This not only reduces the amount of boilerplate, but it also enables us to handle the numbering of the yield points automatically6. Add another pair of convenience macros and we’re done7:

#define generator_init() switch (f->_pc) { case 0:
#define generator_done(R) } return R
#define yield(R) f->_pc = __LINE__; return R; case __LINE__:

typedef struct {
  int _pc;
  int a;
  int b;
  int n;
} fib_frame;

int fib(fib_frame *f) {
  generator_init();
  f->a = 0;
  f->b = 1;
  f->n = 1;
  while (1) {
    yield(f->a);
    f->a = f->b;
    f->b = f->n;
    f->n = f->a + f->b;
  }
  generator_done(-1);
}

This blog post could stop here but sadly I have no self control.

Bonus: Iterators

The above is a perfectly serviceable generator function but it doesn’t work quite the same way as generator functions in Python or C# do. If our generator function has any input arguments, we need to initialize them directly in the frame structure, which is a bit unusual to say the least. Easy fix, split the function in two, and lets rename frame to iterator while we’re at it:

typedef struct {
  // same fields as before, new name
} fib_iterator;

int fib_next(fib_iterator *f) {
  // same as our old fib
}

fib_iterator fib(void) {
  // no arguments so zero initialized
  // should have used a different example...
  return (fib_iterator){};
}

// no need to initialize the struct manually anymore
fib_iterator f = fib();
for(int i = 0; i < 10; i++) {
  printf("%d\n", fib_next(&f));
}

Now it looks more familiar, but it’s the same thing really.

Bonus: Foreach

Other languages also have foreach loops and various ways of combining iterators. Right now we don’t have a standardized way for an iterator to indicate that it has stopped, and our running example (fib) never terminates, so we need a new one:

#define generator_init() switch (i->_pc) { case 0:
#define generator_done() } return false
#define yield(R) i->_pc = __LINE__; *r = R; return true; case __LINE__:

typedef struct {
  int *array;
  int len;
  int idx;
  int _pc;
} rev_iterator;

rev_iterator rev(int *array, int len) {
  return (rev_iterator){array, len};
}

bool rev_next(rev_iterator *i, int *r) {
  generator_init();
  for (i->idx = i->len-1; i->idx >= 0; i->idx--) {
    yield(i->array[i->idx]);
  }
  generator_done();
}

The new example is a simple reverse iterator, yielding the elements of an array starting from the end and going backwards.

The next function now returns a boolean indicating if the iterator produced a value, while storing the actual output in an out parameter. We also adjust the macros accordingly. Now we can make a “foreach” loop:

rev_iterator i = rev((int[]){1, 2, 3}, 3);
for(int v; rev_next(&i, &v);) {
  printf("%d\n", v);
}

Bonus: Itertools

Ok ok, I swear this is the last one. Other languages let us combine iterators with fancy generic functions like zip, take_while, etc. We can also do those in C… somewhat.

First change is that we need to have a shared interface for our iterators:

typedef struct iterator iterator;
struct iterator {
  int _pc;
  bool (*next)(iterator*, void*);
};

typedef struct {
  iterator base;
  int *array;
  int len;
  int idx;
} rev_iterator;

Yup, we’re making objects.

We also need to update the constructor function8 and the macros. The rev_next function from before stays the same:

#define generator_init() switch (i->base._pc) { case 0:
#define generator_done() } return 0
#define yield(R) i->base._pc = __LINE__; *r = R; return 1; case __LINE__:

rev_iterator rev(int *array, int len) {
  return (rev_iterator){
    .base.next = (bool (*)(iterator*, void*))rev_next,
    .array = array, 
    .len = len,
  };
}

This shared interface allows us to implement a yield_from macro:

#define iter_next(I, R) ((iterator*)(I))->next((iterator*)(I), (R))
#define yield_from(I) while(iter_next((I), r)) \ 
  { i->base._pc = __LINE__; return 1; case __LINE__:; }

And with it, we can implement a sequence iterator:

typedef struct {
  iterator base;
  iterator* fst;
  iterator* snd;
} seq_iterator;

bool seq_next(seq_iterator* i, void* r) {
  generator_init();
  yield_from(i->fst); 
  yield_from(i->snd);
  generator_done();
}

seq_iterator seq(iterator* fst, iterator* snd) {
  return (seq_iterator) {
    .base.next = (bool (*)(iterator*, void*))seq_next,
    .fst = fst,
    .snd = snd,
  }
}

Now we can pretend we’re a modern programming language with streams:

rev_iterator r1 = rev(int[]{1, 2, 3}, 3);
rev_iterator r2 = rev(int[]{4, 5, 6}, 3);
seq_iterator s = seq((iterator*)&r2, (iterator*)&r1);
for(int v; seq_next(&s, &v)) {
  printf("%d\n", v);
}

If we change the generator constructor functions to heap allocate and return opaque pointers9, we’ve reached parity with all these new language whippersnappers:

iterator* r1 = rev(int[]{1, 2, 3}, 3);
iterator* r2 = rev(int[]{4, 5, 6}, 3);
iterator* s = seq(r2, r1);
for(int v; iter_next(s, &v)) {
  printf("%d\n", v);
}

But in all seriousness this is getting out of hand 😬.

Conclusion

Generators in C with only mild macro abuse. This was quite a fun post to write. I don’t think I’d bother with generators in C most of the time (certainly not with fancy combinators), but it is nice the capability is there if I need it.

I’ll definitely use this for my toy language backend though, generators are awesome.

iterator* seq(iterator* fst, iterator* snd) {
  seq_iterator* i = malloc(sizeof(seq_iterator));
  *i = (seq_iterator){
    .base.next = (bool (*)(iterator*, void*))seq_next,
    .fst = fst,
    .snd = snd,
  };
  return (iterator*)i;
}

From a computational perspective, it’s about equivalent to the following:

Loud and unspecific. Well, at least with a baby there’s a limited set of reasons for crying, so fixing the issue amounts to tackling each possible cause until something works. Good luck debugging the error message above though.

That got me thinking, how does one end up with a useless error message like the above? What facilities should programming languages provide to make it easy for developers to handle errors properly and conveniently?

Originally this post was going to be a survey on error handling approaches, similar to my post on memory management approaches, but I decided against it in the end because I think looking at this problem from “outside the box” is important. Maybe someone can come up with a better way to “handle errors”.

Faults, Errors and Failures.

Lets starts from the beginning. What do we mean by error? Unfortunately the term “error” is not used consistently in the literature, but we can use 3 related, commonly used terms from the study of fault tolerance to distinguish different meanings for the word error1:

• A fault (known colloquially as a bug), is static defect in software, meaning there’s some line (or lines) of code that are incorrect and will likely result in…

• An error, which is an unobserved, incorrect internal state that will likely result in…

• A failure, which is observed, incorrect behavior with respect to the software’s expected behavior.

For example, consider the following piece of C# code:

var a = new int[]{1,2,3,4,5};
for (int i = 0; i <= 5; i++) {
  Console.WriteLine(a[i]);
}

There’s a fault in the loop stop condition (it should be i < 5), which will result in an error state of i = 5, which will trigger a failure when used to index the vector (throwing an IndexOutOfRange exception).

The error is not the exception (failure) nor is it the incorrect loop condition (fault).

It is critical to understand this distinction for the rest of the article to make sense. Before tackling error handling, I’ll need to talk about error prevention, as the two often get mixed up, and it’ll help clarify what I mean.

Error (Fault) Prevention

Testing, static type systems, model checking, sanitizers, fuzzers, and even certain language features like foreach loops (which would trivially avoid the problem above) are ways to indirectly prevent errors by either detecting or preventing faults.

I’m 100% pro all of the above, they’re all great. If you’re not using them and you could, then you should. Your users will thank you. Your colleagues will thank you. Your future self will thank you.

Errors are almost always the result of faults. Barring cosmic rays, hardware issues or really unusual race conditions between the application and the operating system, if an error occurs it is because the programmer screwed up and introduced a bug.

Some languages like Rust and Haskell have a reputation that “if the code compiles, it works”. They get this reputation because they excel at preventing common faults through a combination of a powerful static type system plus a culture of modeling function domains and codomains as accurately as possible.

Consider the following function declaration (in Rust syntax):

fn head(v: Vec<i64>) -> i64

The head function takes as input a vector2 of signed 64-bit integers and returns the first integer in that vector.

This function’s domain is the set of all vectors of signed 64-bit integers and its codomain is the set of signed 64-bit integers. But this function declaration is “lying”, either about its domain, or about its codomain, depending on the point of view.

There is a hidden pre-condition that v is not empty. If v is empty, the function will panic3.

A “truthful” head function would have a different domain or codomain:

fn head(v: NonEmptyVec<i64>) -> i64
fn head(v: Vec<i64>) -> Option<i64>

In the first case, we tighten the domain, in the second case we loosen the codomain. Either way, we’ve made the function less likely to result in faults — by reminding the programmer of special cases they must take into account — at the cost of making it more annoying to use.

In the first case the caller must prove they have a NonEmptyVec by calling some explicit conversion function, while in the latter case the caller must always handle the “None” case even if they know for a fact that the vector is not empty.

If multiple properties are desired at the same time (e.g., expecting a non-empty even-length vector) the “truthful domain” approach quickly collapses without access to much more powerful type system features like dependent types or refinement types4, which in turn add a massive amount of complexity to the language and are, IMO, not worth it5.

The codomain can always be loosened with a single additional “invalid input” case, and various language facilities can be added to conveniently deal with this extra case, making it the better solution most of the time. For example, in C# nullable types have a great deal of features to make them as convenient to use as possible:

• flow typing, which propagates the result of null checks (i.e. an int? variable becomes int for as long as the result of the check is valid)

• null-coalescing (?? and ??=) operators that make it convenient to replace a null value with an alternative value from the non-nullable type.

• null-conditional (?. and ?[]) operators that allow “flat-mapping” operations from the non-nullable type into the nullable type.

The idea is to keep the advantage of truthful codomains (reminding the programmer to handle special cases) while mitigating the disadvantages (inconvenience).

Error (Failure) Handling

Even with as many fault prevention measures in place as possible, errors will always happen. Fault prevention amounts to ensuring known pre-and-post-conditions are properly handled, it can’t help with unexpected logic errors or lacking requirements.

Unfortunately once an error occurs, it can’t be handled directly. Remember, an error is an unobserved incorrect state. The moment you observe an error, it has already turned into a failure.

Consider for example a set of additions and subtractions applied to a variable where intermediate computations result in overflow but the final result does not. The intermediate overflowed values are errors, but there is no resulting failure.

If you actually checked for overflow each operation, you’d detect the error, triggering a failure and causing a panic or equivalent. And if, instead, you checked all the inputs to make sure they wouldn’t ever overflow, you’d be preventing a fault. Conflating faults and failures, I believe, has led to some really shitty language features that cause more faults and failures than they solve (e.g., exceptions, more on those in a bit)

So errors cannot be handled, only failures can, and a failure is an observed incorrect behavior of the program. If the program is behaving incorrectly, what can you do about it?

First, how does the program know it is behaving incorrectly? If the program can know it is behaving incorrectly, couldn’t the fault be prevented in the first place? Yes. But for various reasons the cost of doing so may be too high.

An example would be needing to check after every arithmetic operation for overflow. I don’t mean the compiler inserting checks and triggering a failure, I mean the programmer explicitly checking for overflow after every arithmetic operation and handling that “special case” each and every time. Extremely annoying.

Alternatively, the overflow-related faults could be prevented by using arbitrary-precision arithmetic, which would instead have a computational cost and may result in a different kind of failure (out-of-memory).

Compiler (or programmer) inserted checks are a means of computationally observing failures caused by unlikely (but otherwise expected) errors, in turn caused by faults that would be excessively costly to prevent. The most common example is array bounds checking.

Note that these checks are not failure handling, they are a necessary step to detect the failure but actually handling it comes afterwards.

Exceptions Suck

The most common response to observing a failure is to throw an exception. Exceptions unwind the stack until they hit a programmer-specified handler (i.e., a try/catch block) or a default handler that crashes the program and usually prints out a stack trace to help debug the problem.

The second most common approach is to abort the program, by sending it a signal that more often than not is caught by a default signal-handler, which will terminate the program and output a “useful” message like Segmentation Fault (which is really a type of failure as defined above, don’t you hate inconsistent naming conventions?)

Rust panics can be set to use an exception-like mechanism, to terminate the current thread, or to abort the program (sending it a signal as above).

You’ll notice that all of the possibilities just crash the program by default. Before I discuss why, I need to point out that the following is not failure handling:

try {
  File f = new File("filename.txt");
  ...
}
catch (FileNotFoundException e) {
  ...
}

In my opinion this is just a weird looking conditional for one particular “output value” of the File constructor. The “truthful” codomain of the File constructor includes additional cases that are “returned” as exceptions.

Would you ever write code like the above to handle an array out of bounds situation? Right after you tried to index an array? What about division by zero? No, right6?

Forgetting to handle a missing file is a fault, and catching that “output” is not having that fault. Using exceptions instead of properly modeling the function’s codomain has increased the likelihood of a fault and its corresponding failure by not reminding the programmer to handle a common failure case.

I really dislike exceptions for mixing up these unrelated concerns7.

If you allow encoding different “error types” into your failure handling feature you’ve probably already screwed up. Let me explain.

How to handle Errors (i.e., Failures)

Let me put it nice and clear:

Remember the pipeline: Fault → Error → Failure. A fault is a bug in the program’s source code which causes the program to enter an incorrect unobserved state, which can then lead to a failure, which is observed incorrect behavior.

The job of a failure handler is to get rid of the error. Note that it’s not handling the error, what is being handled is the failure, the error that caused it is unknown. But the goal is, nonetheless, to get rid of the error, somehow.

Consider an IndexOutOfBounds exception. If one happens, it’s because there is a bug in the program that resulted in some variable being set to an incorrect value, which then resulted in the observed incorrect behavior of trying to index an array out of bounds. What should the failure handler do?

First, a true failure handler won’t be anywhere near the actual index out of bounds situation, because if it was, you were just handling one of the possible “return values” of the indexing operation (preventing a fault), not actually handling a failure.

In a language with algebraic effect handlers, the handler for the index out of bounds situation could “fix” the failure by resuming the program with a made up value for that index. Terrible idea, essentially replacing a failure with a new error. With exceptions you can’t even do that.

No, a true failure handler is a piece of code that a programmer hopes never actually has to run! It’s the last line of defense. All that the failure handler knows is that some variable (no idea which) got set to a wrong value at some point (no idea where) that was then used to wrongly index an array (knows where, but can’t do anything about it).

Given the above, IndexOutOfBounds might as well have been PoopExplosion90008 as far as the failure handler is concerned. The information is useful for the programmer, as is the stack trace, but it could just as well have been a text string in the assertion error message. The actual type of the exception is utterly useless for the purpose of failure handling (not for ghetto codomains, but you shouldn’t use exceptions for that in the first place).

While you can only handle failures (because only the failure is observed) getting rid of the failure by itself doesn’t do much good, there’s still the unobserved invalid state that led to it in the first place. But you can’t do anything about that invalid state directly, since you have no idea what it is or where it originated from.

This is why the default handler for any sort of panic mechanism (exceptions, signals, etc.) is a full-on crash that spits out as much information as it can for the programmer to debug with. There’s nothing else it can do.

The only thing you can do is turn it off and on again.

Every failure handler that’s any better than the default is ultimately just decreasing the scope of what’s getting restarted or improving the usefulness of the info-dump.

Restarting for the Greater Good

The choice of panic mechanism (exceptions, signals, terminating threads) essentially dictates the scope of what can be restarted. If restarting wasn’t the goal, then there would be no need for any mechanism beyond terminating the program outright.

Exceptions let you restart at an arbitrary point in a function. This is less useful than it sounds because the error state may have occurred outside the handler’s restart point. You can only safely restart pure functions or those that work like transactions.

Restarting threads (particularly worker threads) is one of the best approaches if the threads don’t share mutable state. Erlang (and by extension Elixir) is built entirely around this idea, using supervision trees. Because Erlang was designed with failure handling in mind, it is an excellent fit for high-reliability systems.

The last case, sending a signal to a program, may sound like it doesn’t leave much room for “restarting” but that depends heavily on how the software works. If the program backs-up the users work every second to disk, then you can go back to a “known, correct state” by restarting the whole program and instructing it to load the user’s backed up work. You can also use this approach in a multi-process architecture.

Another software architecture that works well for restarting is the Elm Architecture, since each “update” step can be cancelled. Alas, Elm itself lacks a nice failure handling mechanism to take advantage of this. Trying to avoid failures at all costs results in this sort of nonsense.

In all cases the hope is that the triggering of the fault is an uncommon occurrence, otherwise the program will end up in a pointless restart loop.

Unfortunately errors due to logic bugs may not result in a computationally observable failure. Think glitched out physics simulations for example. Just gotta wait for the user complaints to show up.

Wrapping Up

Faults are bugs in the source code that lead to errors (unobserved invalid states) that wreck havoc until they trigger a failure (observed invalid behavior).

Many faults are easily preventable mistakes while others are too costly or annoying to prevent. Languages and tools that avoid or detect such mistakes are good.

You can’t handle errors directly, you can only handle their corresponding failures.

Handling a failure always means restarting (part of) the program to get rid of the error, otherwise it’s not really handling a failure, it’s just working with an extra known “return value” of a function.

You should architect your software in a way that allows for proper failure handling. Some languages (like Erlang) have excellent support for this. Others like Elm think it isn’t necessary (I disagree).

Exceptions are used as both a way to model extra “return values” of functions and as a failure handling mechanism, leading them to be lousy at both. Exceptions suck.

Side-Note: I didn’t mention it within the text because it’s already too big and an incoherent mess, but there’s another nice approach to preventing faults than expanding a codomain with a “must handle invalid result” case to remind the programmer of the potential fault.

You can also expand the codomain with an extra “harmless” result. A terrible idea for a library since it can’t possible know what such a harmless result would look like for the caller, but a perfectly valid approach for an application.

For example, lets say your program loads up a bunch of textures at the start. Instead of the texture loading function returning an Option<Texture> that you then need to deal with everywhere, it could return just Texture, but use a special “Missing Texture” for textures that failed to load.

The function would log somewhere which textures were missing, and just let the program proceed as normal, still letting the user do some work (if they didn’t need those textures). The program can check that errors were logged at some point and display them to the user in a separate codepath, while fixing the missing textures can be done by the user some other time. Check this post by Ryan Fleury on the subject.

Where do you store the types?

Some CS professor is probably already screaming “in the symbol table dummy!”, but that’s not what I’m talking about. The job of the symbol table is to map identifiers in the various scopes of the program to their types or other relevant information, but it’s usually an ephemeral thing. Once the semantic analysis phase is over, the symbol table is gone. Parts of it don’t even last more than the scope where they are needed.

But what if you need to know the types of various expressions and such in the later phases? For example, what if you need to go from implicit casts to explicit casts? What what if you have type inference and need to remember what the actual computed type was? What if you need to know which overloaded function got selected? How do you store this information? Well, there are 4 ways I can think of:

• The Mutable AST

• The Typed AST

• The Generic AST

• The Relational AST

The Mutable AST

This one is the easiest, but mutation can get awkward depending on the language that you are using to implement the compiler. I’ll use Python here to keep things simple. I’m also not going to show a full blown AST, just little bits to give the idea:

@dataclass
class Expr:
  span: Span
  e_type: Optional[Type] = field(default=None, init=False)

@dataclass
class Binary(Expr):
  left: Expr
  op: Token
  right Expr

Lets say we’re parsing an expression like 3 + 4.0. The parser has no idea about types, so it just does the following1:

def parse_binary(self, left: Expr) -> Expr:
  op = self.parse_token()
  right = self.parse_expression()
  span = left.span + right.span
  return Binary(span, left, op, right)

The span represents the area of the source code that the expression covers, important to show some red squiggles later 😉. Then in the typechecker we fill in the e_type:

def check_binary(self, b: Binary):
  self.check_expr(b.left)
  self.check_expr(b.right)
  if b.op.id in ("+", "-", "*", "/"):
    self.check_if_numeric(b.left)
    self.check_if_numeric(b.right)
    b.e_type = self.common_supertype(b.left, b.right)
  elif b.op.id in ("==", "!="):
    ...

def check_if_numeric(self, e: Expr):
  assert(e.e_type is not None)
  if not isinstance(e.e_type, (IntType, FloatType)):
    raise TypeCheckError(
      f"expected a numeric type, got {e.e_type}"),
      e.span)

Something like this. We fill in e_type as we recursively go through the AST in the typechecker, then we see if the e_types of the sub-expressions are of the type we want.

Note that you want your checking function to return “nothing”, because this forces you to grab the e_type in the caller, in turn making sure you don’t forget to set it.

• The Good: As simple as it gets. You use the same AST you already created during parsing so no extra allocations necessary, nor any duplication in the source code.

• The Bad: Depending on the language that you are using, that Optional[T] can get annoying. In Rust it would mean a lot of .unwrap() for example. Additionally, the structure of the AST classes/structs that makes the most sense for parsing may be awkward to work with in later stages2.

• The Ugly: Your AST classes/structs reference “types”, which technically come from a later stage in the compilation. Not a major issue by any stretch of the imagination but it is a bit awkward from a code-organization standpoint.

The Typed AST

This one is a lot more work. Here your type checker produces an entirely new structure from the AST, lets call it a Typed-AST or TAST for short. For example:

# AST
@dataclass
class Expr:
  span: Span

@dataclass
class Binary(Expr):
  left: Expr
  op: Token
  right Expr

# TAST
@dataclass
class TExpr:
  e_type: Type

@dataclass
class TBinary(TExpr):
  left: TExpr
  op: BinaryOp
  right TExpr

Your typechecking functions would take ASTs as input and spit out TASTs as output.

def check_binary(self, b: Binary) -> TBinary:
  left = self.check_expr(b.left)
  right = self.check_expr(b.right)
  ...
  return TBinary(e_type, left, op, right)

• The Good: This approach has the most freedom. You can have the AST be as convenient as possible for parsing/typechecking and the TAST be as convenient as possible for typechecking/code-generation. You can’t forget to set anything, the act of building the TAST forces you to fill in the types. No Optional in sight.

• The Bad: You need a whole separate class/struct hierarchy! That’s quite a bit of extra work compared to reusing the same AST. You need to maintain both as you add or change features. This approach also requires more memory allocations.

• The Ugly: “TExpr”, “TBinary”, etc. 🤢. I ended up calling the AST ones “ExprNode” and the TAST ones “Expr” instead.

The Generic AST

This one is sort of a middle ground between the two approaches above. Instead of storing an Optional type and mutating the AST, we store a generic metadata field:

# AST
@dataclass
class Expr(Generic[T]):
  metadata: T

@dataclass
class Binary(Expr[T]):
  left: Expr[T]
  op: Token
  right Expr[T]

The parser outputs Expr[Span] while the type checker outputs Expr[Type], for example. You can make the metadata as rich as you’d like between stages.

• The Good: Like the mutable AST there is only one class/struct hierarchy, easy to maintain. Like the typed AST there are no annoying Optionals relating to the type information.

• The Bad: Like the mutable AST the structure may not be the best fit for every compilation stage. Like the typed AST this needs extra memory allocations.

• The Ugly: The moment you need more than 1 generic parameter you’ll regret ever picking this option.

The Relational AST

The last option is to not store the extra type information in the AST at all, but instead to do things the relational way: each AST node has a unique ID that serves as the primary key for tables of whatever extra data you need.

@dataclass
class CompilerDB:
  expr_types: Dict[int, Type]
  expr_spans: Dict[int, Span]
  ...

@dataclass
class Expr:
  id: int

You can improve type-safety by creating distinct “ID” types for different things, instead of using int everywhere. You can also take the relational model further3:

@dataclass
class CompilerDB:
  expressions: List[Expr]
  expr_types: Dict[Id[Expr], Type]
  expr_spans: Dict[Id[Expr], Span]
  ...

@dataclass
class Expr:
  id: Id[Expr]

@dataclass
class Binary:
  left: Id[Expr]
  op: Token
  right: Id[Expr]

In Python this approach doesn’t make a lot of sense, but in a more high-performance language like C++, Rust or even C# this can lead to some nice performance gains. Not only can the IDs be smaller than a pointer, saving memory, but if your parser builds the list of expressions in the CompilerDB bottom-up, then in the typechecker you can navigate the list linearly instead of using recursion4.

So what’s the catch? Unfortunately mainstream programming languages don’t support relational programming in any meaningful way, so you need to do everything in a rather cumbersome manner:

def check_binary(self, b: Binary):
  b_span = self.db.expr_spans[b.id]
  l_type = self.db.expr_types[b.left]
  l_span = self.db.expr_spans[b.left]
  r_type = self.db.expr_types[b.right]
  r_span = self.db.expr_spans[b.right]
  self.check_numeric(l_type, l_span)
  self.check_numeric(r_type, r_span)
  e_type = self.compute_super_type(l_type, r_type, b_span)
  self.db.expr_types[b.id] = e_type

• The Good: Very efficient in both speed and memory usage. Extremely adaptable. Basically all the benefits of the relational model.

• The Bad: The CompilerDB needs to be dragged around everywhere. You can’t implement .str() in the AST classes themselves, the function needs access to the CompilerDB. Boilerplate-heavy. Easy to make mistakes.

• The Ugly: It reminded me of how cool the relational model is and how mainstream programming languages don’t even know it exists beyond lousy ORM libraries for SQL databases. I don’t like SQL and I don’t like ORMs.

Conclusion

Ok, four options, which one should you use? If you’re building a small, simple compiler that doesn’t need a lot of extra processing, use the Mutable AST. If your compiler is single pass then it’s a no-brainer since the “annoying optional” problem doesn’t even exist, just fill the type in right there and then.

If you’re building a more complicated compiler, you may want to use either the Typed AST or the Relational AST, depending on your needs.

Most computer-science papers use some variant of the Typed AST approach, usually translating between multiple different intermediate languages. It’s the nicer approach if you want to do lots of analyses and you want to make sure you got them right.

If you want the best performance then the Relational AST is the way to go. You can avoid recursion, you save memory by using IDs instead of pointers, you can move data to different tables depending on usage patterns. It can be very data-oriented.

The Generic AST is not worth it, the other approaches are just better. Any other approach I don’t know about? Let me know in the comments below.

Anyway, that’s not to undermine the incredible achievement that is ChatGPT or other Large Language Models (LLMs), nor other neural network-based solutions for image or audio generation and such. They’re game changing pieces of tech, no doubt about it.

But I cannot help but be sad that the focus is all in on neural networks, they’re not the only approach to AI. Not only are there many other forms of Machine Learning (ML)1, but there is also Machine Reasoning (MR), the focus of this post.

Machine Reasoning? What’s that?

If you’ve never heard the term Machine Reasoning, I don’t blame you, I’m convinced it’s something a former manager of mine came up with to distinguish what we were doing from Machine Learning (which was already getting hyped at the time).

You won’t find the term on Wikipedia, instead you’ll find the idea split across many different pages like “automated reasoning”, “knowledge representation and reasoning”, “constraint satisfaction problem”, and more. A complete mess to follow.

To make it clear what I mean: a Machine Learning system is a piece of software that derives rules from facts (an inductive process). In contrast, a Machine Reasoning system is a piece of software that derives facts from rules (a deductive process).

Expert Systems you mean?

When I mention Machine Reasoning to people, the first thing that usually comes to mind for them are Expert Systems — piles of “IF-THEN” rules painstakingly hand-written by domain experts. It’s no surprise those things fell out of favor against machine learning approaches that just figure things out from a big pile of data.

Expert systems are a form of machine reasoning yes but one of the least interesting IMO. Just as linear regression is not the first thing that comes to mind when talking about Machine Learning, neither should expert systems be the first thing that comes to mind when talking about Machine Reasoning.

For me the truly interesting Machine Reasoning systems are model generators for which the constraint satisfaction problem page is the closest but incomplete.

What’s a model generator you ask? It’s a piece of software that given a set of variables and a set of constraints on those variables, produces an assignment of those variables (a model) that is consistent with the constraints.

On top of that basic idea you can add:

• Objective functions — where we’re interested in finding an optimal model (in regards to the objective) rather than any arbitrary model.

• Soft constraints — where we’re interested in finding a model that satisfies the most constraints simultaneously.

When the hard constraints have one or more conflicts, it’s also possible to compute Minimum Unsatisfiable Subsets (MUS) — meaning the minimal set of constraints that if removed would make the problem have a solution.

The search for an optimal solution can be done incrementally depending on the algorithm used (producing better and better solutions over time), and we can also ask some systems to produce all possible models for the problem, rather than just one.

That’s nice and all, but why not ML instead?

Machine Learning (ML) and Machine Reasoning (MR) overlap for some problems but you’d normally use them for different purposes.

There are two properties of ML that make it a poor fit for some domains:

• ML requires a massive amount of training data to work well. Some domains don’t have such a quantity of data available.

• ML is statistical in nature and ultimately a stochastic process. The results aren’t very predictable, hence “hallucinations” and hands with 7 fingers.

ML excels at problems without a clear set of rules and where the solution does not have to be 100% accurate. For example, I have no idea what rules I would give to some system for it to generate a picture of a dog, and it is usually ok if the picture looks more like a drawing or 3D render than a perfectly lit photo of a dog.

MR, on the other hand, is a better choice for well defined problems that require precise solutions. Examples include: electronic design automation (EDA), software verification, test generation, network optimization, routing, resource allocation, scheduling, supply chain management, etc.

One example of what MR can do is Simcenter Studio, which is able to generate all topological variants of a physical system, simulate them, and then rank them, given simple high-level specification of the system’s components, their ports, and any relevant connection constraints.

It was used, for example, to find all 4-speed 2-stage transmission designs that can possibly exist (turns out there are only 12). It proved there are no other solutions in around 8 hours. It took decades for the first 4-speed 2-stage transmission design to be patented after 2-stage transmissions were first introduced.

An ML system trained on pre-existing 3-speed 2-stage transmission designs might be able to accidentally produce a 4-speed 2-stage transmission design, but there’s no guarantee. It certainly cannot prove there are only 12 solutions.

Many package managers use MR tech to handle package dependency resolution. A friend of mine is using MR for planning wedding seats.

Machine Reasoning Technologies

There are many different technologies that fit the umbrella of “Machine Reasoning” or model generators as I’ve defined it above, here I’ll briefly cover the ones I find the most interesting or relevant: Boolean Satisfiability (SAT), Satisfiability Modulo Theories (SMT), Constraint Programming (CP), (Mixed) Integer Programming (MIP) and Answer Set Programming (ASP).

Boolean Satisfiability (SAT)

Boolean Satisfiability (SAT), is sort of the assembly language of model generation. SAT is restricted to boolean variables, and boolean constraints (clauses) in Conjunctive Normal Form (CNF). This may sound limiting at first but then you remember that computers work in 0s and 1s. You can model logic gates in CNF using a Tseytin transformation, and with logic gates you can model arithmetic.

The algorithm at the heart of nearly every SAT solver is called Conflict-Driven Clause Learning (CDCL). CDCL is a backtracking algorithm that records extra conflict clauses every time it hits a dead-end branch in the search space, preventing similar paths from being explored in the future.

CDCL (plus many fancy heuristics and preprocessing tricks) makes SAT solvers absurdly efficient. Modern SAT solvers can handle problems with millions of variables and millions of constraints. They’re a core piece of tech in the EDA domain.

There’s a yearly SAT competition where many different SAT solvers compete to solve as many problems as possible in as little time as possible. The standard input format for SAT solvers is DIMACS. Example SAT solvers include Kissat and CryptoMiniSAT.

Satisfiability Modulo Theories (SMT)

Satisfiability Modulo Theories (SMT), builds on top of SAT by combining a SAT solver with one or more theory solvers. Examples of theories include integer arithmetic, real arithmetic, uninterpreted functions, strings, etc. SMT solvers are an indispensable piece of technology in the domains of software verification and formal proofs.

Most SMT solvers use the CDCL(T) algorithm, but there are other approaches like translating the problem entirely into SAT (only possible for some theories like the theory of bit-vectors or floating point numbers).

Like with SAT there’s a yearly competition where many different SMT solvers compete to solve as many problems as possible in as little time as possible. The standard input format for SMT solvers is SMTLIB2. SMT solvers include: cvc5, Z3, Yices 2, MathSAT 5, Bitwuzla, etc.

Constraint Programming (CP)

Unlike SMT which is a direct descendant of SAT, Constraint Programming (CP) developed in parallel, though these days many CP solvers employ SAT solvers internally and work in a similar manner to SMT solvers.

In CP the focus is on finite-domain integer variables2 and global constraints: powerful high-level constraints that can, through specialized algorithms, effectively and efficiently reduce the domains of the variables involved in the constraint. The most famous global constraint is alldifferent, which constrains every variable in its input array to take a different value.

The naive encoding for alldifferent would require pairwise inequalities between every constrained variable, which is inefficient to solve. There are substantially more efficient ways of decomposing or propagating alldifferent. The global constraint has more information than a set of equivalent local constraints, and because of this can be solved more efficiently. That’s the basic idea of CP.

CP solvers are an indispensable piece of technology in operations research, excelling at solving complex scheduling and routing problems.

There are two main algorithms used these days for solving CP problems: Maintaining Arc-Consistency (MAC) and Lazy Clause Generation (LCG). MAC works by employing specialized algorithms for each global constraint alongside an arc-consistency algorithm like AC-3, whereas LCG works similarly to an SMT-solver, with each global constraint behaving as a sort of theory solver that produces SAT clauses representing reductions in the domain of the variables.

Like SAT and SMT, CP has yearly competitions, one for each of the two standard input formats: MiniZinc and XCSP3. CP solvers include: CP-SAT, Gecode, Choco, etc.

(Mixed) Integer Programming (MIP)

This section is technically encompassing many related technologies.

Linear programming (LP) focuses on optimization problems involving continuous variables and linear inequality constraints on those variables, plus an objective function. Integer linear programming (ILP) is the same but with integer variables. If the solver supports both continuous and integer variables, it’s called Mixed Integer Programming (MIP). If the variables are integer and the constraints non-linear, it’s called Integer Programming (IP). If the variables are mixed and the constraints non-linear, it’s called Mixed-Integer Non-Linear Programming (MINLP). And then there are special cases of non-linear programming like Quadratic Programming (QP) that have dedicated algorithms. Phew… got all that?

(Mixed) Integer Programming (MIP) in particular has the same “computational power” as the other approaches above. They can all handle NP-complete problems.

The focus here is clearly on optimization. If you need to find the best solution to a problem that is mostly continuous plus some discrete variables, this is the tech to go for. Some SMT solvers and most CP solvers can also handle optimization, but MIP solvers tend to be much better at it. So much better in fact that commercial MIP solvers can sell for thousands of dollars.

The main algorithm used by most modern MIP solvers is called Branch & Cut, which combines the Simplex algorithm from linear programming with backtracking search and cutting planes to reduce the search space when the search hits a dead-end.

Unlike SAT, SMT and CP, I’m not aware of any official competition for MIP, but there is a benchmark that compares some open source and commercial solvers. There isn’t a standard input format for MIP, but nl, LP and MPS are the most commonly supported formats. MIP solvers include: CBC, SCIP, HiGHS, Gurobi, CPLEX, etc.

Answer Set Programming (ASP)

One way to think of Answer Set Programming (ASP) is as a middle ground between Datalog and Prolog. Not turing-complete like Prolog, but more expressive than Datalog (Datalog cannot handle NP-complete problems, but ASP can).

Constraints in ASP (unlike the other approaches) are mainly universally quantified rather than existentially quantified. That is:

ancestor(A, B) :- parent(A, B). 
ancestor(A, C) :- parent(A, B), ancestor(B, C).

Is equivalent to the first order logic formula

∀a, b. Parent(a, b) → Ancestor(a, b)
∀a, b, c. Parent(a, b) ∧ Ancestor(b, c) → Ancestor(a, c)

ASP solvers cannot handle universally quantified formulas like that directly, they first need to be grounded (i.e., every variable expanded with every possible value). That’s handled by a separate component called a grounder. ASP software is usually a pair of a solver and a grounder.

Modern ASP solvers use an adaptation of CDCL from SAT as their core algorithm and can often invoke the grounder component incrementally.

Like with SAT, SMT and CP, there is a competition, but it does not happen as often. The standard input formats for ASP solvers are AnsProlog and ASP-Core-2. ASP solvers include: clingo (clasp + gringo) and DLV (wasp + I-DLV)

Concluding Remarks

That was a long but very incomplete look at a field that is dear to my heart. It saddens me to see it relegated to the sidelines while deep-learning steals the spotlight. There is so much more to cover, I didn’t even cover local-search-based approaches.

It’s not like Machine Reasoning and Machine Learning are at odds either, they can be combined! You can, for example, train a ReLU-based neural network on the operational data of some real-world hardware system to create a simulation model, translate that neural-network to a set of linear equations, and then use a MIP Solver to optimize some aspect of that system.

Or you could have an LLM generate constraint satisfaction problems and invoke an existing solver instead of trying to solve arithmetic problems directly. Or combine the texture synthesis capabilities of wave function collapse with the image generation capabilities of diffusion models.

Don’t leave everything to megacorps with massive datasets.

A function like malloc may appear simple at first glance but most modern malloc implementations, like jemalloc or tcmalloc, are actually incredibly sophisticated. From avoiding memory fragmentation to scalable concurrency, there’s a lot of hardcore engineering work going on under the hood.

But while malloc implementations automatically manage memory pages, they’re considered a form of “manual memory management” because it is still up to the programmer to use malloc and free correctly. Managing memory manually, with unstructured usage of malloc and free, is highly error prone, with double-free and use-after-free bugs possibly resulting in severe security vulnerabilities.

Less severe, but also problematic, are memory leaks resulting from forgetting to free some allocations. A long running program with recurring memory leaks will keep increasing its memory usage until the process has to be terminated.

All three issues stem from “free”. If you forget to call it, you get a leak. If you call it more than once, you get a double-free. If you call it while some part of the program is still using that memory, you get a use-after-free. Ouch 😬.

This post is a survey on the various approaches that exist to avoid these issues, from semi-automatic solutions that greatly reduce the memory management effort to fully-automatic solutions that remove the need for “freeing memory” entirely.

I’ll cover the following in this blog post:

• Regions (aka Arenas, aka Bump Allocators)

• Memory Pools

• Reference Counting

• Tracing Garbage Collection

• Substructural Type Systems

• Lifetime Annotations

Regions (aka Arenas, aka Bump Allocators)

I mentioned previously that modern malloc and free implementations are highly sophisticated, so it may come as a surprise to you that one of the best ways to manage memory in C is to avoid malloc and use the simplest allocator imaginable instead: a bump allocator (also known as a region, or arena).

A bump allocator is called that because it just “bumps” a pointer forward on each allocation. The simplest bump allocator does little more than the following:

void* alloc(bump_allocator* a, size_t size) {
  if (a->pointer + size >= a->end) {
    return null;
  }
  void* result = a->pointer;
  a->pointer += size;
  return result;
}

Instead of returning null, the bump allocator may also request additional pages from the operating system, switching the pointer over to the new location.

If the bump allocator is only tracking a single pointer, how does the program free individual allocations? Well, the secret is that it doesn’t. You can only reset the allocator’s pointer back to the beginning1.

The standard malloc has to be as general as possible to cover all imaginable use cases. The bump allocator on the other hand can only be used to allocate blocks of memory that share a lifetime. Thankfully having many blocks of memory sharing a lifetime is rather common.

Consider a web server: each request can use its own bump allocator, allocating freely, followed by a single free when the request is complete. Or a video game: any data that only lasts for a frame can be allocated freely and then freed in bulk.

While a bump allocator cannot handle all memory allocation patterns, the patterns it can handle become almost trivial and therefore substantially less error prone. Going from 1 free per allocation to 1 free per frame makes a huge difference! It’s a lot easier to get that single free right. As bonus, bump allocation is also ridiculously efficient, so you get a massive performance improvement as the cherry on top.

The Zig and Odin programming languages are designed with the usage of bump allocators for temporary allocations in mind.

Having said that, while double-frees are extremely unlikely with bump allocation, use-after-free can still happen. It’s possible to defend against use after-free by combining regions with the various approaches below. For example, C3 uses scoped regions for temporary allocations.

Despite their simplicity, going in depth about regions would require a blog post on its own, as each combination with the other approaches creates a unique approach to memory management.

Memory Pools

While a bump allocator manages many allocations of different sizes all sharing the same lifetime, a pool allocator manages many allocations of the same size but with different lifetimes. The basic idea of a pool allocator is an array that tracks if the data at a particular index has changed.

A pool allocator does not return a raw pointer to each allocation, but instead it returns a handle with a mix of a generation ID and the index into the array where the allocation resides:

struct PoolHandle {
  uint32_t generation;
  uint32_t index;
}

The number of bits dedicated to the generation or the index can be changed as needed. The generation ID is used to track when a particular index in the pool is freed and/or reused. If the generation ID of a handle and the generation ID of the current allocation at that index don’t match, you have detected a use-after-free. Double-free and leaks are not possible with pools, beyond the “leak” of keeping a certain number of slots in the pool always available even if they are not in use. You can theoretically still have use-after-free and double-free bugs with the pool pointer itself, but since it is only 1 pointer for many allocations, the likelihood of error is greatly reduced.

Pools are somewhat inconvenient to use because to access their data you need both a pointer to the pool and the respective handle, you cannot access the data without having access to the pool. You also need different pools for different datatypes or allocation sizes at least. Much like bump allocators, they cover certain common memory usage patterns, but not all.

Note that unlike bump allocation which is semi-automatic (1 free for many allocations) pool allocation is manual, data is explicitly allocated and freed. The advantage comes from the added safety.

The Vale language extends the idea of generational handles to generational references, safeguarding every memory allocation with the same idea.

The two approaches above are handled by the programmer and are language independent: you can use bump allocation in any language with raw pointers, and pool allocation in every language with mutable arrays. The remaining approaches require the compiler to get involved.

Scope-Based Resource Management (aka RAII)

One of the main improvements C++ brought to the table over C was scope-based resource management, typically called by the awful name RAII2 (Resource Acquisition is Initialization), as that was the name originally given to the approach by Bjarne Stroustrup, the creator of C++. He’s apologized for the name ever since3.

The core idea of scope-based resource management comes from the realization that nobody ever complains about managing memory on the stack, only on the heap. Variables on the stack are automatically pushed and popped alongside the scopes in which they reside, so everything just works.

So why not tether heap memory to the stack variable that “owns”4 it, allocating it when the variable is initialized, and freeing it when the variable’s scope ends? Well, that’s exactly how it works in C++:

void example() {
  string s("Hello World");
  vector<string> v;
  // v's destructor is implicitly called here
  // s's destructor is implicitly called here
}

In the case of containers, like vectors, memory is freed recursively: first every element of the vector, then the vector itself. In C++ this is handled by giving every type a constructor member function and a destructor member function (sometimes even automatically generated by the compiler), and implicitly inserting calls to the type’s destructor at the end of the scope.

This approach of tying resources to stack variables is useful for more than just memory, so many languages provide scoped lifetimes (e.g. with in python) or a defer statement to execute some cleanup code at the end of the scope or function, even if they use a different approach to memory management.

If this was all that C++ supported, it would be a memory safe language (not counting features it inherited from C of course), but things are a bit more complicated.

One major issue with this approach is the pervasive copying. Just as normal stack allocation copies data freely, so must this emulation of the stack with heap memory. Copying small integers on the stack is no big deal, but copying large arrays is:

void example() {
  vector v1{...}; // some huge vector
  vector v2 = v1; // copy :(
}

To work around this issue, C++ has references and move semantics. References are just like pointers except they can’t be null (plus some other confusing and mostly irrelevant constraints). Move semantics use a special kind of reference (an rvalue reference) to indicate that the previous location of the value won’t be used anymore:

void example() {
  vector v1{...};
  vector v2 = move(v1); // no copy :)
  // v1 is in an undefined state after the move
  std::cout << v1.at(0) << "\n"; // *boom*
}

The move function doesn’t actually do anything, it is just a cast to an rvalue (&&) reference, the actual “move” occurs in the vector’s move assignment operator:

vector<T>& operator=(vector<T>&& other)

The operator is responsible for “consuming” the data from “other” (v1 above) and storing it in the vector (v2 above). After the assignment, “other” (meaning v1) is left in a safe but “unspecified” state, and should not be used anymore.

C++ does not provide any safety mechanisms for references and only minimal safety for moved-from values. You can’t get a double-free (because the compiler inserts the destructor for you, though the destructor could be buggy) but you can easily have use-after-free bugs with references.

Rust took this idea (including references and moves), and made it 100% safe. We’ll cover the approaches Rust uses to ensure safety later. For now, the important thing to keep in mind is that Scope-Based Resource Management (without references on top) enforces a tree-shaped structure on memory.

C++, Rust, and Ada all use Scope-Based Resource Management.

Reference Counting

The idea of reference counting is simple and effective. When you allocate some data on the heap, a “reference count” is stored alongside it. When a new pointer to that data is created, the reference count is increased. When a pointer’s lifetime ends (e.g. it was stored in a variable that’s now out of scope), the reference count is decreased. When the reference count reaches 0, the data gets freed.

Reference Counting is the simplest form of garbage collection and combines very well with Scope-Based Resource Management, since the latter can handle the increments and decrements of the reference count in the constructors and destructors of the pointers. Because of this languages like C++ and Rust can provide reference counted pointers as a library (std::shared_ptr in C++ and Rc and Arc in Rust).

There are a few issues with reference counting:

• The first is that there are a lot of increment and decrement operations happening, and they need to hit main memory to update the reference count, which negatively affects cache usage. This can be mitigated by detecting “moves” which are an increment immediately followed by a decrement, and just skipping them.

• The second is that in a multithreaded environment, these increment and decrement operations need to be atomic, which further slows things down. This can be mitigated by detecting when reference counted pointers cross thread boundaries.

• The last is that reference counting cannot handle cycles. If A stores a reference to B and B stores a reference to A, the reference count of both is 1 and they’ll never get freed, causing a leak. This must be worked around by either forbidding cycles (constraint on the memory layout), using weak references to break cycles (manual labor and error prone), or applying some form of cycle collection (i.e. tracing).

Python, Swift and Nim all use reference counting. Python has cycle collection, Swift does not, and Nim’s is optional. Without a workaround for the cycle issue, reference counting imposes a DAG-like structure on memory.

Tracing Garbage Collection

Of the approaches we’ve seen, one requires that all allocations share a lifetime, one is still manual but protects against use-after-free and double-free, one enforces a tree-like memory layout, and one enforces a DAG-like memory layout.

What if we don’t want to worry about memory management at all5? That’s where tracing garbage collection comes in. Unlike every other approach covered so far, tracing garbage collection can handle arbitrary graphs of references.

The concept is simple: First, determine all root pointers, i.e., pointers stored in registers, stack variables and global variables that point to the heap. Mark every allocation pointed to by a root pointer. For every pointer stored in a marked allocation, recursively mark those allocations. Once every reachable allocation is marked, free all others as they are unreachable and therefore garbage data.

There are different ways to implement the above idea. The most direct is a mark-and-sweep garbage collector which marks and then sweeps (frees) exactly as described above. But another way to achieve the same idea is with a copying semi-space collector. The basic idea is as follows:

Set up two bump allocators, the “from” space and the “to” space. Allocate everything in the from space until it fills up. Once full, go through all the pointers as in a mark-and-sweep collection, but also copy each marked allocation to the “to” space. Once finished, “sweep” by simply resetting the “from” allocator pointer. The “to” space becomes the new “from” space and the old “from” space becomes the new “to” space.

Neat stuff. So if we have these algorithms that can handle arbitrary graph-like structures, why did I even bother going through the other more limited approaches above? Everything has tradeoffs:

• The first issue is that it is mostly unpredictable when a tracing garbage collector will get triggered. The algorithm is not cheap so you don’t want to run it often.

• The second, is that to make matters worse, when the garbage collector is triggered, it needs to stop the whole program while it does its job. This is terrible for applications that need to remain highly responsive like video games.

• The third issue is that to work around the above issues, modern tracing garbage collectors are extremely complex, and any solution to reduce the length of the pauses inevitably requires doing more work, and therefore lowers throughput.

• The fourth and final issue, is that there is a lot of infrastructure that is necessary for the tracing garbage collector to work (i.e., how does it know about all the pointers in the program? how does it stop every thread in the program?), I go over that in my article The Hidden Costs of Tracing Garbage Collection.

Java, Javascript, C#, Go, Haskell, etc. all use tracing garbage collection, it is by far the most popular approach despite the drawbacks.

Substructural Type Systems

I mentioned before that I’d cover the approaches used by Rust to ensure complete safety for scope-based resource management with moves and references. First lets have a look at “moving”.

“Moving” in C++ happens when a move constructor or move assignment operator is executed, it is otherwise not tracked by the type system. The moved-from data must be set by the move operation to some safe but unspecified value such that the destructor can later check for that value to prevent a double-free.

Rust tracks ownership of values at the type system level with affine types, a kind of substructural type system. A variable of an affine type must be used at most once.

fn example() {
  let x = vec![...];
  let y = x; // move
  println!("{}", x); // compile error, x was moved
  // destructor of y implicitly called here
}

The “x” variable was “used up” in the second line of the function, and as such cannot be used anymore. It’s not so much that rust tracks “moves” but more so “uses”. If a variable is never used, rust will implicitly insert a destructor for it at the end of the scope, but from the type system’s perspective the destructor does not matter.

As an alternative to affine types, there are also linear types that enforce each variable is used exactly once. In a linear type system the programmer would be expected to explicitly call a destructor function (which is just any function that consumes the value and does not return a new one).

Using affine or linear types can get quite cumbersome, as any transformation requires storing the result in a new variable:

let x = f();
let x1 = g(x);
let x2 = h(x1);
...

To make substructural types a bit more palatable there’s the idea of “borrowing”. In Rust that is handled with references, which complicates things, so the example below uses an explicit made up “borrow” keyword:

fn increment(borrow x: Vec<i64>) {
  // increment all values of x
}
let y = vec![...];
increment(y);

Is theoretically equivalent to:

fn increment(x: Vec<i64>) -> Vec<i64> {
  let x1 = x;
  // increment all values of x1
  return x1;
}
let y = vec![...];
let y = increment(y) // new variable also called y

Though of course the compiler can just do an in-place mutation.

Rust, Clean, Mercury and Austral all use substructural types, but only Rust and Austral use them for memory management. An added benefit of substructural types is that they also prevent data races at compile time.

Lifetime Annotations

Finally we get to how references can be made safe. The easiest solution is to just remove them from the language entirely, as ParaSail and Hylo do. You can get surprisingly far without first-class references.

But what if we really want references plus safety, what can be done? Rust solves this issue with lifetime annotations. The idea of tracking lifetimes came from attempts at making region-based memory management safe to use, but in Rust they’re generalized to work with any allocation.

The basic idea is simple, when you create a reference to some allocation, add an extra tag to the type of the reference with the lifetime of the allocation, for example:

let x = vec![...];
let y = &x;

The types in the example above are inferred, but if we write them out, we see that there’s an extra type parameter to the reference, a lifetime annotation:

let x: Vec<i64> = vec![...];
let y: &'a Vec<i64> = &x;

That little ‘a is tracking y’s lifetime. It doesn’t actually store anything, it’s just a type variable that gets unified in the type system. For example, a function like:

fn get<'a>(array: &'a [Stuff], index: usize) -> &'a Stuff {
  ...
}

The type variable ‘a is tracking the lifetime of the slice, and records in the type of the returned Stuff reference that it has the same lifetime. This is achieved through unification, the same algorithm used to handle generic type variables.

Rust’s syntax is not the easiest to follow when it comes to lifetimes, the above function wouldn’t normally even have that ‘a explicitly written out since Rust can figure out the annotations on its own for cases like the above with only 1 reference.

So instead we can look at Austral since its references are always explicit:

let buf: ByteBuffer := allocateBuffer(100, 'a');
let len: Index := length(&buf);
destroyBuffer(buf);

The above looks very similar to Rust and works the same except for the explicit destructor. But if we look at the length function, things become a bit clearer:

generic [R: Region]
function length(buf: &[ByteBuffer, R]): Index is
  return !(buf->size);
end;

The length function is generic over a region called R, and every reference in Austral must have an associated region6. Inside of it we use the path operator (→) to get a reference to the size field of the buffer (of type &[Index, R]) which we then convert into an Index with the dereference operator (!).

But what if we want to store a reference to the ByteBuffer in another variable?

let bufref: &[ByteBuffer, ?] := &buf;

What do we fill in for the “?” there? In Rust, an implicit “region” is created (so to say) to fill that in, but in Austral we have to explicitly borrow buf:

borrow buf as bufref in R1 do
  -- bufref is of type &[ByteBuffer, R1]
end borrow;

The region is only valid in that scope, if we tried to store the reference outside of the scope, we’ll get an error:

let outerref: &[ByteBuffer, R1]; -- error: unknown region R1
borrow buf as bufref in R1 do
  outerref := bufref -- impossible since they cannot have the same type
end borrow;

What rust is doing is effectively the same as the above, only with a lot more syntax sugar to avoid needing to explicitly write out the lifetimes.

Conclusion

Ouff, that was a long article, but hopefully I covered all the main approaches to memory management. If there are any that I missed do let me know in the comments. I’ve heard of ASAP which is a sort of compile-time inlined tracing garbage collector but unfortunately the one implementation of the idea showed mediocre performance (which makes sense, a tracing GC is not something you want to run often, even in little spread out bits).

Personally, I think if we just gave up on first class references we’d have a really good memory management story that doesn’t require the massive sledgehammer that is a tracing garbage collector or nasty lifetime annotations.

If you’re developing a component library, to be used by other people developing web applications, then your choice of framework will limit your users to that framework (my favorite component library is Mantine, but it is only available for React). Most frameworks don’t play nice with each other, web component-based libraries being the sole exception2.

But many web component libraries (like Shoelace) depend on Lit, which is a framework, which means that if the app developer is not using Lit, then their app has two frameworks now.

So this post is an attempt at making sense of the features natively provided by the browser, and how well and how far one can take them in order to develop a component library, without needing to rely on a framework of some kind.

Web Components

The browser-native approach for developing component libraries are web components. However, technically, web components aren’t their own thing, rather they are a mix of three related but ultimately separate standards: Custom Elements, HTML Templates, and Shadow DOM.

When I first heard about web components, I was expecting them to be something like the following:

<template name="my-split-panel">
  <style>...</style>
  <script>...</script>
  <div>
    <slot name="left-panel">...</slot>
    <div id="splitter">...</div>
    <slot name="right-panel">...</slot>
  </div>
</template>

Which you’d then use (assuming they were written in a separate file) like the following:

<html>
  <head>
    <link rel="import" href="/__u/btmc.substack.com/components/my-split-panel.html">
  </head>
  <body>
    <my-split-panel>
      <div slot="left-panel">...</div>
      <div slot="right-panel">...</div>
    </my-split-panel>
  </body>
</html>

Pretty simple right?

Right?…

Boy was I wrong! Oh so wrong…

Unfortunately things are quite a bit different in reality. The HTML import spec never got accepted so that’s out. Not a major issue, you could just use some sort of bundler to join the files into one.

The bigger issue are the templates: You can’t instantiate HTML templates from within HTML. You can’t actually name them. You can give them an ID, and find them and instantiate them from JavaScript, but not from HTML.

So if I wanted to make something like a reusable “card” or “avatar”, or “navigation” component, which has no associated behavior but merely styles some divs a certain way, I gotta use JavaScript. A “split button” component that reuses builtin browser behavior? Sorry, JavaScript it is. Forget “server side rendering” too.

Ok so we need to create a “Custom Element” in JavaScript to achieve that custom <my-split-panel/> tag. What does that look like?

const template = document.createElement("template");
template.innerHTML = `
  <template>
    <style>...</style>
    <div>
      <slot name="left-panel"></slot>
      <div id="splitter">...</div>
      <slot name="right-panel"></slot>
    </div>
  </template>
`;

class MySplitPanel extends HTMLElement {
  constructor() {
    super();
    const shadowRoot = this.attachShadow({ mode: "open" });
    shadowRoot.appendChild(template.content.cloneNode(true));
    ...
  }
  connectedCallback() {
    // got attached to the DOM
    ...
  }
}

customElements.define("my-split-panel", MySplitPanel);

We could get the template out of the HTML file using getElementById, but actually putting the template into an HTML file wouldn’t make any sense for a reusable component library (can’t import HTML remember?), so we need to put it into a string in the JS file. Nobody thought this was dumb? Just me?

So Custom Elements, other than being a pure JS thing instead of the HTML+CSS+JS hybrid they should be, are pretty reasonable. They don’t handle any kind of reactivity as you’d expect from a modern javascript framework, and they need some annoying boilerplate, but otherwise they’re fine.

Or they would be, if not for that “shadowRoot” thing, which is part of the Shadow DOM spec.

What’s a Shadow DOM?

Shadow DOM is a way to encapsulate the internals of a component from the rest of the page, meaning CSS cannot directly affect a web component’s internal nodes and any selector queries won’t find those nodes either. Quoting mdn:

Without the encapsulation provided by shadow DOM, custom elements would be impossibly fragile. It would be too easy for a page to accidentally break a custom element's behavior or layout by running some page JavaScript or CSS. As a custom element developer, you'd never know whether the selectors applicable inside your custom element conflicted with those that applied in a page that chose to use your custom element.

Funny, somehow this was never a problem for Angular, or React, or Vue, or Svelte, or any other framework, but Custom Elements without shadow DOM would be impossibly fragile? I’m sorry but I don’t see how. Even without frameworks and browser support, web “components” have been a thing since forever!

I can plop a CodeMirror instance into my webpage and not worry about blowing up their styles. Why would I? They’re all namespaced as “cm-”. Perhaps they should have used a longer namespace with less of a chance of collision, sure, but that’s neither here nor there. CSS modules work via name mangling and are redundant with this whole shadow DOM approach. CSS variables? Better remember to namespace them as shadow DOM or not they can conflict. Selectors? Just don’t write lousy selectors! When has that ever been a problem?

Shadow DOM is a misguided idea. The ability for JavaScript and CSS to dig down and modify components made by other people is a feature, not a bug. The author of a web component has to put a lot of effort into making it possible to style their component from the outside. Shadow DOM does not play well at all with external stylesheets (e.g. Bootstrap) or even Tailwind. It sucks IMO, and I’m not alone in this sentiment.

Well, Shadow DOM is technically a separate spec right? We can simply not use it, right? … Right? Unfortunately template slots require a shadow DOM.

So we can avoid this shadow DOM business so long as we don’t use slots, meaning we can’t place sub-elements provided by the consumer of our component in the right place without using a shadow DOM (we can’t use an internal <style/> block either but CSS modules are a much better solution for that anyway so it doesn’t matter).

Well, technically we can place the elements in the right place, just not using slots. There’s nothing preventing us from doing the following to set the components in the right place:

const splitPanelTemplate = document.createElement("template");
splitPanelTemplate.innerHTML = `
  <div>
    <template id="my-split-panel-left"></template>
    <div id="my-split-panel-splitter">...</div>
    <template id="my-split-panel-right"></template>
  </div>
`;

class MySplitPanel extends HTMLElement {
  constructor() {
    super();
    this.template = splitPanelTemplate.content.cloneNode(true);
    this.left = this.template.getElementById("my-split-panel-left");
    this.right = this.template.getElementById("my-split-panel-right");
    ...
  }

  connectedCallback() {
    let left = this.querySelector("[data-slot='left']");
    let right = this.querySelector("[data-slot='right']");
    this.left.replaceWith(left);
    this.right.replaceWith(right);
    this.innerHTML = this.template.firstElementChild.innerHTML;
  }
}

// make sure to initialize the component after its children
window.onload = () => {
  customElements.define("my-split-panel", MySplitPanel);
}

There are some important things to note here. First, we’re emulating slots with data attributes, by replacing the child nodes of the cloned template node with the child nodes with the corresponding data-slot annotation.

Second, The connectedCallback() method is not necessarily called when the component’s children have already been created, but when the browser knows about the component and sees its tag. We work around that by only defining the custom element on window.onload, guaranteeing the child elements are in place first.

Lastly, this does not handle the case where an element in a slot is changed programmatically. For that, we need to set up a MutationObserver. Not a big deal, but it would be a lot nicer to have the slotchanged event available.

My point with the above is not to tell you this is the right way to do things, it’s to show you that I can emulate shadow DOM slots without a lot of ceremony, meaning there’s no reason the browser couldn’t do it too. It’s an unnecessary restriction.

Conclusion

Boy what a mess. I do not like this standard, not one bit. That said, the Custom Element part of it can be salvaged, and also templates up to a point. For the example above I wouldn’t bother with a template at all, I’d just ask the user to provide the div for the splitter as well. One extra line of code from the user for more control on their part and a much simpler component on my part? Win win. These web components that just add behavior to a set of user-defined light DOM elements are being called “HTML Web Components” by some other light DOM loving folk out there.

I hate that name, but I can’t come up with a better one. I love the idea though, they work great, are very efficient, and fit in well with the existing web. The only issue is that they require the user to provide all the internal divs themselves, which depending on the component may defeat the whole point. Maybe one day I’ll write a micro compiler from my imaginary HTML+CSS+JS Web Component standard to a light DOM-based Custom Element as the one I showed above. Sort of like a micro Svelte (without reactivity or any other fancy features).

Do let me know in the comments if I’m an idiot for not understanding the brilliance of the shadow DOM. As for me, I prefer the light.