🚀 NixOS-DGX-Spark: Bringing Nix to the DGX Spark

This project enables the use of Nix and NixOS on the NVIDIA DGX Spark and the Asus Ascent GX10. Whether you want to keep your current OS and use Nix for development or dive into a full NixOS installation, this repository provides the necessary USB images and NixOS modules.

[!IMPORTANT] For a high-level overview, check out the 5-minute lightning talk from Planet Nix: Watch here.

---

🛠️ Option 1: Using Nix on DGX OS (Ubuntu)

You don't need to replace your entire operating system to benefit from Nix. You can run playbooks and dev shells directly on the factory NVIDIA DGX OS.

Setup Checklist

• Install Nix: Use one of the following methods:
  • Official: sh ( curl -L https://nixos.org/nix/install ) --daemon
  • Determinate: curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
• Enable Experimental Features: Add the following to /etc/nix/nix.conf: experimental-features = nix-command flakes
• Configure Caching: Enable the graham33 Cachix cache.

Handling GPU Drivers

On Ubuntu, Nix-built CUDA apps cannot natively find host GPU drivers. To solve this, the project uses nix-gl-host.

• Automatic: Container-based playbooks don't need it; Nix-native playbooks wrap commands automatically.
• Manual: You can prefix your commands like so:

  nix develop . # cuda nixglhost deviceQuery
  

---

💿 Option 2: Full NixOS Installation

Warning: The factory firmware is designed to boot only DGX OS.

Installation Process

1. Flash the Image: Use dd to write the ISO to your USB drive:

   sudo dd if=$(echo result/iso/*.iso) of=/dev/your_usb_disk_device bs=1M status=progress sync
   

2. Boot: Enter the BIOS, Enable Secure Boot $\rightarrow$ Disable Secure Boot, and boot from the USB.
3. Configure: Follow the official NixOS installation manual.

Kernel Selection

When booting, the GRUB menu offers two distinct paths:

Kernel Option	Description	Status
NixOS (Default)	Specialized NVIDIA kernel for DGX Spark	✅ Full GPU & Ethernet support
NixOS (Standard)	Standard NixOS 6.17 kernel	⚠️ Ethernet works perfectly Networking issues

---

📦 The DGX Spark Module

The provided module allows for easy hardware configuration.

Basic Configuration

To enable support, add this to your config:

dgx-spark = { 
  enable = true; # Activate DGX Spark hardware support
  useNvidiaKernel = true; # Use the optimized NVIDIA kernel (default)
};

Features of the NVIDIA Kernel

The NVIDIA kernel is a custom build tailored for this hardware. It includes:

• Optimized Performance: Based on NVIDIA's Debian annotations.
• System Monitoring: Enables the DGX Dashboard web interface at http://localhost:11000 for real-time GPU telemetry.

---

⚙️ Kernel Configuration Management

The project uses a streamlined process to maintain the kernel config, stored in kernel-configs/nvidia-dgx-spark-version.nix.

The Generation Pipeline

The process of creating the "terse" configuration is visualized below:

Efficiency & Maintenance

By only storing options that differ from the NixOS defaults, the verbosity is reduced significantly. We can represent the configuration size reduction as: $\text{Reduction} \approx 82\%$

Regeneration is required when:

1. The NVIDIA kernel version is updated (update kernel-configs/nvidia-kernel-source.nix).
2. The nixpkgs common-config changes.

Manual Generation Command: nix run . # generate-kernel-config (Use -- --kernel-source /path/to/NV-Kernels for local dev).

---

🚀 Integration & Quick Start

Importing the Flake

Other projects can integrate this module by adding the input:

{
  inputs.dgx-spark.url = "github:graham33/nixos-dgx-spark";

  outputs = { nixpkgs, dgx-spark, ... }: {
    # Your system configuration
    imports = [
      # ... other modules
      {
        imports = [ dgx-spark.nixosModules.dgx-spark ];
        dgx-spark.enable = true; 
        # dgx-spark.useNvidiaKernel = false; # Optional: use standard kernel
      }
    ];
  };
}

Using the Template

For a turnkey experience, initialize a new configuration using the provided template:

# 1. Setup directory
mkdir my-dgx-spark-config && cd my-dgx-spark-config

# 2. Initialize from template
nix flake init -t github:graham33/nixos-dgx-spark#dgx-spark

This generates a flake.nix and configuration.nix pre-configured for the DGX Spark.