NixOS install doc — Anywhere eval, DTB notes, install approach, boot management #6
1 changed files with 248 additions and 0 deletions
248
docs/nixos-install.md
Normal file
248
docs/nixos-install.md
Normal file
|
|
@ -0,0 +1,248 @@
|
|||
# NixOS Install — Orange Pi 3B (Cyberdeck Multi-Boot)
|
||||
|
||||
**Goal:** NixOS on eMMC partition p4 (root_nixos), booting via shared extlinux.conf alongside Orange Pi OS Arch and Armbian.
|
||||
|
||||
## Context
|
||||
|
||||
The cyberdeck's eMMC has a four-partition layout with a shared /boot on p1:
|
||||
|
||||
```
|
||||
mmcblk0 (233GB eMMC user data area)
|
||||
├── mmcblk0p1 FAT32 1GB /boot (shared — kernels, DTBs, extlinux.conf)
|
||||
├── mmcblk0p2 ext4 85GB root_opios (Orange Pi OS Arch — live)
|
||||
├── mmcblk0p3 ext4 85GB root_armbian (Armbian — live or planned)
|
||||
└── mmcblk0p4 ext4 ~84GB root_nixos (NixOS — target)
|
||||
```
|
||||
|
||||
Boot is Rockchip-standard user-data-area: PARTITION_CONFIG=0x78, idbloader at sector 64, u-boot.itb at sector 16384, SPI flash erased. u-boot reads `extlinux.conf` from the FAT32 /boot partition for the boot menu.
|
||||
|
||||
Full walkthrough of the boot architecture and partition setup: `docs/emmc-multiboot.md`.
|
||||
Existing second-OS install pattern: `docs/armbian-install.md`.
|
||||
|
||||
## NixOS Anywhere — Evaluated, Not a Fit
|
||||
|
||||
[NixOS Anywhere](https://github.com/nix-community/nixos-anywhere) is an SSH-based tool that installs NixOS onto a remote machine. It uses `kexec` to boot a temporary NixOS installer in RAM, then uses `disko` to partition and format the disk. It's a great tool for Hetzner servers and fresh-install scenarios, but it doesn't work for our multiboot setup for three reasons:
|
||||
|
||||
1. **Disko is destructive by default.** Default mode is `destroy,format,mount` — it wipes the entire disk and repartitions from scratch. There is no supported flow to say "install into partition 4 and leave the rest alone." [GitHub issue #274](https://github.com/nix-community/nixos-anywhere/issues/274) documents this gap; the [disko dual-boot discourse thread](https://discourse.nixos.org/t/disko-dualbooting-windows-and-nixos/61665) shows someone whose existing partitions got eaten.
|
||||
|
||||
2. **Shared /boot is incompatible with NixOS's boot generation model.** NixOS auto-generates boot entries. Every `nixos-rebuild switch` or `nixos-rebuild boot` writes to /boot and would overwrite our extlinux.conf, removing the Arch and Armbian entries.
|
||||
|
||||
3. **kexec on ARM adds complexity without solving anything.** Aarch64 kexec images exist (from `nix-community/nixos-images`) but even once you kexec in, you're back to the disko problem.
|
||||
|
||||
**Verdict:** NixOS Anywhere is built for clean-slate installations on blank disks. Our shared-extlinux.conf multiboot setup needs a manual approach.
|
||||
|
||||
## Device Tree Blob (DTB) — Same One Works
|
||||
|
||||
The DTB describes the Orange Pi 3B's hardware — the RK3566 SoC, its memory map, its peripherals. It's not OS-specific. The same `rk3566-orangepi-3b.dtb` that Orange Pi OS Arch uses will boot NixOS.
|
||||
|
||||
The only caveat: DTBs are compiled from the kernel source tree, and kernel bindings evolve. A DTB from kernel 6.6 might not have everything a kernel 6.12 expects. But in practice, for Rockchip boards, they're backward-compatible within a major version range.
|
||||
|
||||
**Recommendation:** Point the NixOS extlinux.conf `FDT` directive at the DTB file already on the shared /boot partition. If it doesn't work (kernel panic about missing clocks or an undetectable PHY), extract a matching DTB from the NixOS closure instead.
|
||||
|
||||
## NixOS-Specific Challenges
|
||||
|
||||
NixOS differs from Arch and Debian in how it handles kernels and initrds. These three wrinkles make installation more involved than `rsync` and an extlinux.conf entry.
|
||||
|
||||
### 1. Kernel + DTB Are Inside the Nix Store
|
||||
|
||||
Unlike Arch (where `/boot/vmlinuz-linux` is a standalone file) or Debian (where `/boot/vmlinuz-*` is installed by the kernel package), NixOS bundles the kernel and DTB inside the Nix store at paths like:
|
||||
|
||||
```
|
||||
/nix/store/<hash>-linux-<version>/Image
|
||||
/nix/store/<hash>-linux-<version>/dtbs/rockchip/rk3566-orangepi-3b.dtb
|
||||
```
|
||||
|
||||
These are not on a traditional /boot partition. They're symlinked into the NixOS /boot by `systemd-boot` or the extlinux generator, but the actual files live in the Nix store. You can't just copy them — you need to build the closure on a machine with Nix installed, then extract the boot artifacts.
|
||||
|
||||
### 2. Initrd Is Generated From Configuration
|
||||
|
||||
NixOS generates the initrd at build time from the system configuration. It doesn't exist as a standalone file until `nixos-rebuild` produces it. Like the kernel, it lives in the Nix store:
|
||||
|
||||
```
|
||||
/nix/store/<hash>-initrd-linux-<version>/initrd
|
||||
```
|
||||
|
||||
You need to extract it from a pre-built closure and place it on the shared /boot manually.
|
||||
|
||||
### 3. NixOS Wants to Manage /boot
|
||||
|
||||
By default, NixOS installs a bootloader (systemd-boot on aarch64 UEFI, or extlinux on non-UEFI). Once running, `nixos-rebuild switch` attempts to write to /boot — which would overwrite our shared extlinux.conf and break the other OSes.
|
||||
|
||||
**Fix:** Configure NixOS to manage only the kernel and initrd on /boot, not the bootloader menu. Set:
|
||||
|
||||
```nix
|
||||
boot.loader.grub.enable = false;
|
||||
boot.loader.generic-extlinux-compatible.enable = true;
|
||||
# OR, for manual-only:
|
||||
boot.loader.generic-extlinux-compatible.enable = false;
|
||||
```
|
||||
|
||||
With `generic-extlinux-compatible` enabled, NixOS writes a `boot.txt` or `extlinux.conf` to /boot, but will overwrite our shared version. With it disabled, NixOS doesn't touch /boot at all — you manage the extlinux.conf entry yourself on every kernel update.
|
||||
|
||||
## Recommended Install Approach
|
||||
|
||||
The pattern is: **pre-build on a machine with Nix, extract boot artifacts, copy to eMMC.**
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- A machine with Nix installed (Lucy, Mermaid Man, or BarnacleBoy if Nix is available)
|
||||
- An SD card to transfer files to the Orange Pi 3B
|
||||
- The Orange Pi 3B booted into Orange Pi OS Arch (or any working OS on eMMC)
|
||||
|
||||
### Step 1: Build a NixOS closure (on Lucy or Mermaid Man)
|
||||
|
||||
Create a Nix flake for the Orange Pi 3B configuration. At minimum:
|
||||
|
||||
```nix
|
||||
{
|
||||
description = "NixOS for Orange Pi 3B";
|
||||
inputs = {
|
||||
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
|
||||
};
|
||||
outputs = { self, nixpkgs }: {
|
||||
nixosConfigurations.orangepi3b = nixpkgs.lib.nixosSystem {
|
||||
system = "aarch64-linux";
|
||||
modules = [
|
||||
({ pkgs, ... }: {
|
||||
# Minimal config for first boot
|
||||
boot.loader.grub.enable = false;
|
||||
boot.loader.generic-extlinux-compatible.enable = false;
|
||||
boot.kernelParams = [ "console=tty1" "console=ttyS2,1500000n8" ];
|
||||
system.stateVersion = "24.11";
|
||||
# Root on p4 — set the UUID or /dev path
|
||||
fileSystems."/" = {
|
||||
device = "/dev/mmcblk0p4";
|
||||
fsType = "ext4";
|
||||
};
|
||||
# Do NOT mount /boot — it's managed externally
|
||||
# fileSystems."/boot" = ...; /* leave this out */
|
||||
users.users.root.initialPassword = "changeme";
|
||||
services.openssh.enable = true;
|
||||
})
|
||||
];
|
||||
};
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Build it:
|
||||
|
||||
```bash
|
||||
nix build .#nixosConfigurations.orangepi3b.config.system.build.toplevel
|
||||
```
|
||||
|
||||
### Step 2: Extract boot artifacts
|
||||
|
||||
From the built closure, extract the kernel, initrd, and DTB:
|
||||
|
||||
```bash
|
||||
# Find the closure path
|
||||
ls -la result/
|
||||
|
||||
# Kernel
|
||||
cp result/boot/Image /tmp/nixos-kernel
|
||||
|
||||
# Initrd
|
||||
cp result/boot/initrd /tmp/nixos-initrd
|
||||
|
||||
# DTB — find it under the kernel package
|
||||
find result/ -name "rk3566-orangepi-3b.dtb" -exec cp {} /tmp/ \;
|
||||
|
||||
# The entire Nix store closure (for copying to p4)
|
||||
nix copy --to ./result /tmp/nixos-closure
|
||||
```
|
||||
|
||||
Getting the full rootfs onto p4 is the trickiest part. Options:
|
||||
|
||||
- **Option A:** `nix copy` the closure to a path on the SD card, then `nix-store --import` on the OPi3B (if Nix is installed)
|
||||
- **Option B:** Use `nixos-enter` or manual chroot manipulation
|
||||
- **Option C:** Build a tarball of the rootfs and extract it on the target
|
||||
|
||||
### Step 3: Copy to the Orange Pi 3B
|
||||
|
||||
Put the kernel, initrd, DTB, and rootfs archive on an SD card. Boot the OPi3B into Orange Pi OS Arch.
|
||||
|
||||
```bash
|
||||
# Mount the SD card
|
||||
sudo mount /dev/mmcblk1p1 /mnt/sd
|
||||
|
||||
# Mount the target partitions
|
||||
sudo mount /dev/mmcblk0p4 /mnt/nixos_root
|
||||
sudo mount /dev/mmcblk0p1 /mnt/nixos_root/boot
|
||||
|
||||
# Copy kernel, initrd, DTB to shared /boot
|
||||
sudo cp /mnt/sd/nixos-kernel /mnt/nixos_root/boot/
|
||||
sudo cp /mnt/sd/nixos-initrd /mnt/nixos_root/boot/
|
||||
sudo cp /mnt/sd/rk3566-orangepi-3b.dtb /mnt/nixos_root/boot/
|
||||
|
||||
# Extract rootfs to p4
|
||||
sudo tar -xzf /mnt/sd/nixos-rootfs.tar.gz -C /mnt/nixos_root/
|
||||
```
|
||||
|
||||
### Step 4: Add extlinux.conf entry
|
||||
|
||||
Edit `/mnt/nixos_root/boot/extlinux/extlinux.conf` (the shared one) and add:
|
||||
|
||||
```
|
||||
LABEL NixOS
|
||||
LINUX /nixos-kernel
|
||||
INITRD /nixos-initrd
|
||||
FDT /rk3566-orangepi-3b.dtb
|
||||
APPEND root=UUID=<uuid-of-p4> ro console=tty1 console=ttyS2,1500000n8
|
||||
```
|
||||
|
||||
Get the UUID:
|
||||
|
||||
```bash
|
||||
sudo blkid /dev/mmcblk0p4
|
||||
```
|
||||
|
||||
### Step 5: Boot and finish setup
|
||||
|
||||
```bash
|
||||
sudo umount /mnt/nixos_root/boot /mnt/nixos_root
|
||||
sudo sync
|
||||
```
|
||||
|
||||
Reboot, pick "NixOS" from the u-boot menu. Once booted:
|
||||
|
||||
```bash
|
||||
# Login as root with the password set in config
|
||||
# Set up SSH keys, add a user, etc.
|
||||
# Do NOT run nixos-rebuild switch until /boot management is understood
|
||||
```
|
||||
|
||||
## Boot Management After Install
|
||||
|
||||
Once NixOS is running, kernel updates become a manual process because NixOS isn't managing /boot:
|
||||
|
||||
1. On the build machine, build a new closure with the updated kernel
|
||||
2. Extract the new kernel + initrd + DTB
|
||||
3. Copy them to the OPi3B's shared /boot via SD card or SSH+scp
|
||||
4. Update the extlinux.conf `LINUX` / `INITRD` paths if filenames changed
|
||||
5. Reboot
|
||||
|
||||
Alternatively, if the kernel and initrd paths stay constant (same filenames), you can overwrite in place and the existing extlinux.conf entry keeps working.
|
||||
|
||||
## NixOS on ARM — Board Support
|
||||
|
||||
The Orange Pi 3B doesn't have a dedicated NixOS wiki page yet. The [NixOS on ARM wiki](https://wiki.nixos.org/wiki/NixOS_on_ARM) confirms AArch64 has full upstream support. The generic aarch64 images from Hydra should work on the OPi3B with the correct DTB. The Orange Pi 5 (RK3588) wiki page at <https://wiki.nixos.org/wiki/NixOS_on_ARM/Orange_Pi_5_Plus> serves as a reference — the approach is the same, just different SoC.
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
- [ ] NixOS boots from u-boot menu
|
||||
- [ ] Network works (the kernel needs the right DTB for the Ethernet PHY)
|
||||
- [ ] SSH access (configured in the NixOS flake)
|
||||
- [ ] extlinux.conf entries for all three OSes still present after NixOS boots
|
||||
- [ ] Kernel update works (build, extract, copy, reboot cycle)
|
||||
|
||||
## References
|
||||
|
||||
- [NixOS Anywhere](https://github.com/nix-community/nixos-anywhere) — evaluated, not suitable (see above)
|
||||
- [Disko](https://github.com/nix-community/disko) — declarative partitioning, destructive by default
|
||||
- [NixOS on ARM wiki](https://wiki.nixos.org/wiki/NixOS_on_ARM) — AArch64 support status
|
||||
- [Orange Pi 5 Plus wiki](https://wiki.nixos.org/wiki/NixOS_on_ARM/Orange_Pi_5_Plus) — reference for Rockchip NixOS approach
|
||||
- [nix-community/nixos-images](https://github.com/nix-community/nixos-images) — aarch64 kexec installer images
|
||||
- eMMC multi-boot setup: `docs/emmc-multiboot.md`
|
||||
- Armbian install pattern: `docs/armbian-install.md`
|
||||
- Bring-up reference: `docs/initial-bringup.md`
|
||||
Loading…
Add table
Add a link
Reference in a new issue