The moment I wrote my first program, I knew I wanted to build a programming language. But how are languages made? How did these carefully arranged magic spells become instructions a machine could execute?

For a long time, compiler construction was my main technical obsession — and it still is. I devoured anything I could find about compilers, intermediate representations, parsers, and LLVM. I worked through the Kaleidoscope tutorials and spent hours refining the syntax of my soon-to-be language. At the start of my third year of college, that obsession led me to an internship with Intel's Compilers and Languages team. I was suddenly being paid to work on real programming languages used by real developers.

Eight years later, I know much more about compilers, CPUs, GPUs, and the systems around them. I also have more questions.

Why are production compilers so complicated? Which parts of that complexity are inherent to the problem, and which are consequences of accumulated design decisions? Is the architecture we use today the only reasonable one? Do I really understand how compilers work well enough to derive one from the machine upward?

Puzzled by those questions, and deeply inspired by John Regehr's Future Directions for Optimizing Compilers, I decided to build a compiler from scratch. Not by pretending that the last fifty years of compiler research never happened, and not by reproducing an existing compiler feature for feature. Instead, I want to treat familiar design choices as hypotheses rather than axioms. I want to begin with the semantics of the machine, derive the requirements of each layer, and justify every abstraction as it appears.

TIR is my attempt to do that. It is the target-independent intermediate representation at the center of an end-to-end compiler infrastructure: a C front end, optimization passes, and back ends for multiple processor architectures. This series will document how it is designed, where it succeeds, and where its assumptions break down.

What Counts as a Compiler?

Okay, so I've talked about my passion for compiler construction probably a bit too much. But what are compilers? If you took a foundations of programming class in college, the lecturer probably explained to you that a compiler is a tool that turns your program in some programming language (say, C or Rust) into assembly — another language that a CPU can understand. That definition is useful, but narrower than the one I will use in this series.

Personally, I define compilers as tools that lower the level of abstractions. A script that reads JSON and generates pre-filled C structures is a compiler. A tool that translates a small domain-specific language into configuration or executable code is also a compiler. GCC and Clang are simply larger and more familiar examples.

For the purposes of this series, by "compiler" I am going to refer to something LLVM-shaped. If you have never heard of LLVM, it is a framework for compiler construction. It provides building blocks for programming language authors: target-independent intermediate representation, a set of optimizations, target definitions for x86, ARM, RISC-V and other ISAs. The broader LLVM project includes C and C++ frontends, a Fortran frontend, as well as linkers, debuggers, runtimes, and related tooling.

And that is roughly what I am building — an end-to-end modular infrastructure for constructing compilers. C frontend, target-independent IR, a set of optimizations, and multiple backends to run on a bunch of different CPUs. But I want to do it differently. Unlike a traditional C-compiler-in-a-weekend kind of tutorial, I want my thing be real enough to compile a wide range of real-world codebases. Competitive performance is a stretch goal, not an assumption here. The measurements (and the failures) are part of the experiment.

What "From First Principles" Means

Just twenty years ago every major chip manufacturer would provide their users with a suite of compilers (typically C/C++ and Fortran). Microsoft, Intel, IBM, AMD — all of them. Often they would take money for you to actually use it and in exchange you would get excellent performance out of the box.

Over time, much of the industry converged on LLVM-based toolchains. Intel, IBM, Arm, AMD, and many other vendors now build substantial parts of their compiler offerings on LLVM. This happened for good reasons: LLVM offers permissive licensing, reusable infrastructure, support for many languages and targets, and a mature surrounding ecosystem.

Its success also means that many new compiler projects inherit LLVM's architectural assumptions before asking whether those assumptions fit the problem they are trying to solve. TIR will use existing compilers as evidence and reference points, but not as specifications.

Building from first principles does not mean blindly ignoring established compiler theory. It means starting from the requirements imposed by programs and machines, then deriving the design from those requirements. Familiar techniques remain available, but they must earn their place.

Compiler architecture is often divided into three broad layers:

  • Frontend. Parses source text, checks its meaning, and translates it into an intermediate representation.
  • Middle-end. Analyzes and transforms that representation, mostly independently of the eventual target architecture.
  • Backend. Lowers the representation to machine instructions for a particular target and emits assembly or object code.

TIR will retain this broad decomposition where it proves useful, but the details inside each layer are open to question. What information should the IR preserve? Which transformations belong in the middle end, and which depend on the target? Which abstractions make optimization easier, and which merely move complexity somewhere else?

Those are the questions this project is meant to explore.

The Plan

We will build an intuition for what an instruction is, how hardware and compilers represent it differently, and whether a better abstraction can bridge that gap. From there, we will derive the requirements for an intermediate representation, build analyses and optimization passes, and add code generation for multiple targets.

In parallel with building the core infrastructure for middle end and backend, we will work on a simple C frontend and experiment with various design choices. Compilers bridge software and hardware. One does not make sense without the other. The design therefore has to evolve in both directions: real source programs must shape the low-level abstractions just as real hardware constraints shape the language and optimization layers above them.

The first substantial milestone here will be compiling and running SQLite (although, I'm sure there will be stop gaps where we play with simpler programs). Success will not be measured only by whether the program runs, but also by language coverage, correctness, compiler and generated code performance and quality, and the ease with which the system can be understood and extended.

With each new post we will uncover a new layer of this mystery. It won't always be a big win. It won't always be a win at all. But failure is part of the process. Stay tuned.