Analysis of Build Systems: The Civ Perspective

Following some engagement on the permacomputing mailing list regarding the architecture of personal or permacomputing-focused build systems, I wanted to provide a deep dive into the ../lua/civ build system. I will also examine other existing approaches and share some evolving thoughts on a potential package manager for civ.

Rather than offering targeted critiques, I prefer to outline the fundamental requirements of an effective build system. While I will reference Bazel (Google's massive, incremental build tool) and Make, my goal is to define the "ground truth" of build logic.

The Fundamental Question

To understand build systems, we must first define their purpose. I recall a conversation with a recent graduate who was struggling with CAD tooling. He spent nearly ten minutes detailing the minutiae of his scripts and the frustrations of the software's representation.

Despite my lack of CAD expertise, I asked one clarifying question:

"What are your inputs, and what are your outputs?"

This simple prompt forced him to strip away the noise, revealing a much simpler technical path than he had previously imagined.

The Civ Definition

In the context of civ, a build system is a program that maps inputs to outputs:

$\text{Inputs (Source, Binaries, Config)} \xrightarrow{\text{Build System}} \text{Outputs (Binaries, Files, Structure)}$

Consequently, the First Principle of a quality build system is: $\text{All inputs and outputs for every build phase must be explicitly defined.}$

---

Architectural Principles

The second major challenge is how a user defines their build goals and how disparate elements (scripts, libraries, configs) are integrated.

The Failure of Make

Make fails here because it is essentially a collection of shell scripts wrapped in a platform-specific, obtuse syntax. While it defines rules via inputs and outputs, it lacks a cohesive vision for stitching those rules into a larger whole.

The Second Principle

A build system must allow for the programmatic configuration of inputs and outputs to ensure adaptability across different target systems. It should provide an abstraction layer between the raw source and the final executable—keeping it as simple as possible, but no simpler.

The Civ Solution: The Build Library

Civ handles this by moving the "where" and "how" of the build into a centrally configurable build library. This library uses a minimal API with internals that can be swapped based on the platform.

Example Workflow:

1. The user defines a requirement in cc.luk (a sandboxed Lua configuration): cc{hrd='foo.h'}.
2. The logic in cc.luk converts this into an object.
3. This object is passed to sys/cc/build.lua, which executes the actual build.

Benefits of this Decoupling:

• Source Flexibility: Users can organize source code however they like while still producing a structure that supports system-level flags (e.g., LD_LIBRARY_PATH).
• Scalability: Because the specification is separate from the execution, the process can be scaled across multiple threads or even distributed across machines.

Comparison: Civ vs. Bazel

---

Towards Package Management

The final hallmark of a great build system is how easily it integrates with a package manager. This is currently in the design phase for civ.

Target Definition

In civ (and similarly in Bazel), targets are defined explicitly:

P.mylib = cc { 
    lib = 'mylib', 
    hdr = 'mylib.h', 
    src = 'mylib.c', 
    dep = { 'someHub:other/lib:otherlib' }, 
}

The Hub System

When running civ init, a ~/.config/civ.luk file is created to map "hubs." For example:

C.hubs = { 
    -- The civboot tech stack and build tool
    civ = "/home/rett/projects/civstack/", 
    -- System-specific build rules
    sys = "/home/rett/projects/civstack/sys/", 
}

Currently, these point to local directories, but they could point to remote repositories. To turn this into a full package manager, I propose the hubrepo abstraction.

The Hubrepo Concept

A hubrepo would be a git repository structured as follows:

• user/ directory: Organized by username (e.g., user/re/tt/rett.kv)
  • Contains: Public symmetric keys and ban status.
• hub/ directory: Organized by hub name (e.g., ci/vb/oo/civboot.kv)
  • Contains: Owner info, hub location (GitHub, etc.), and version control type (git, pvc).

This is just a dream $\rightarrow$ This is the blueprint for a decentralized, lightweight dependency system.