---
name: brain-recovery
version: 1.1
updated: 2026-08-21
purpose: Operate, verify, restore, and troubleshoot Brain backups and recovery.
---

# Brain recovery skill

## Recovery model

Authoritative data:

- `/mnt/data/Library`
- `/mnt/data/memory`

Engineered environment:

- `/home/bart/brain`
- `/home/bart/brain-web`
- `/home/bart/brain-test`
- `/home/bart/signals-agent`
- other scripts, configuration, prompts, and operational files under `/home/bart`

Rebuildable and excluded:

- `venv` and `.venv`
- `/home/bart/brain/chroma`
- `__pycache__`, caches, Trash, and model directories

## Storage

- Root: `/dev/nvme0n1p2`
- EFI: `/dev/nvme0n1p1`
- Data disk: ext4 `/dev/sda1` mounted at `/mnt/data`
- NAS: `192.168.1.67`
- Data share: `//192.168.1.67/data_NAS` at `/mnt/data_NAS`
- Home share: `//192.168.1.67/bart_NAS` at `/mnt/bart_NAS`
- Credentials: `/etc/samba/nas-credentials`

## Installed backup files

```text
/usr/local/sbin/brain-backup
/usr/local/sbin/brain-backup-all
/usr/local/sbin/brain-backup-promote
/etc/brain-backup/data.conf
/etc/brain-backup/home.conf
/etc/systemd/system/brain-backup.service
/etc/systemd/system/brain-backup.timer
/var/lib/brain-backup/data.last_success_epoch
/var/lib/brain-backup/home.last_success_epoch
/mnt/data/backup/events.log
/mnt/data/backup/promotions.log
```

## Service entry point

`brain-backup.service` must contain one backup command:

```text
ExecStart=/usr/local/sbin/brain-backup-all
```

`brain-backup-all` always attempts both independent jobs:

1. `brain-backup data`
2. `brain-backup home`

It records both exit statuses. The overall service fails if either job fails, but a failed data job cannot suppress the home attempt or prevent its append-only event.

Do not restore two separate sequential `ExecStart=` lines.

Verify:

```bash
sudo systemctl cat brain-backup.service
sudo bash -n /usr/local/sbin/brain-backup-all
sudo systemd-analyze verify   /etc/systemd/system/brain-backup.service   /etc/systemd/system/brain-backup.timer
```

Warnings naming unrelated distribution units such as XFS scrub units are benign. Resolve any message naming a Brain backup unit.

## Scheduling

The timer attempts five minutes after boot and every 30 minutes after the previous attempt. Each job runs only if its own previous success is at least 24 hours old. NAS and Brain clocks are not compared. Wake-on-LAN is not used.

Enable and verify:

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now brain-backup.timer
systemctl is-enabled brain-backup.timer
systemctl is-active brain-backup.timer
systemctl list-timers brain-backup.timer --all
```

Expected: `enabled`, `active`, and a future trigger.

## Snapshot safety

Each job validates the exact CIFS mount and exact-content sentinel before writes or pruning. It creates a fresh `.partial-*` directory, uses `rsync --link-dest`, writes `.complete` only after success, then retains three completed rolling snapshots.

Never restore `.partial-*`.

Layout:

```text
_brain_backup/
├── rolling/<snapshot-id>/{.complete,tree/}
├── baseline/current/
├── baseline/previous/
├── meta/
└── probe/
```

## Frozen baselines

```bash
sudo brain-backup-promote data latest
sudo brain-backup-promote home latest
```

Promotion requires `PROMOTE <snapshot-id>`, records user, host, time, job, and snapshot, moves old `current` to `previous`, and never occurs automatically.

First approved baselines:

```text
data: 20260821T105003Z-76ffc6c2
home: 20260821T114153Z-79ad4192
```

## Health checks

```bash
systemctl status brain-backup.timer --no-pager
systemctl status brain-backup.service --no-pager -l
sudo tail -n 50 /mnt/data/backup/events.log
```

A completed oneshot service normally becomes `inactive (dead)`. Valid routine events are `SUCCESS`, `SKIPPED_RECENT`, and `SKIPPED_BUSY`. Mount, sentinel, rsync, or unexpected errors must be explicit and nonzero.

Manual nonblocking run:

```bash
sudo systemctl start --no-block brain-backup.service
sudo tail -f /mnt/data/backup/events.log
```

`Ctrl+C` exits `tail`, not the backup.

## CIFS troubleshooting

Before touching NAS paths:

```bash
findmnt /mnt/data_NAS
findmnt /mnt/bart_NAS
```

If no backup is running and a mount is stale:

```bash
sudo timeout 20 umount /mnt/data_NAS || sudo umount -l /mnt/data_NAS
```

Mountpoints are `bart:bart`, mode `755`. CIFS presents content as UID/GID 1000. Thunar browsing through GVFS does not prove these mountpoints are mounted.

## Restore data

Approved baseline:

```bash
sudo rsync -a --numeric-ids   /mnt/data_NAS/_brain_backup/baseline/current/tree/   /mnt/data/
```

Check Library count, recent memory logs, classifications sidecar, and zero-length Markdown files.

## Restore home

```bash
sudo rsync -a   /mnt/bart_NAS/_brain_backup/baseline/current/tree/   /home/bart/
sudo chown -R bart:bart /home/bart
```

Rebuild excluded venvs from lock files and rebuild Chroma from `/mnt/data/Library`.

## Total OS loss

1. Install Ubuntu in UEFI mode without formatting a surviving data disk.
2. Mount `/mnt/data`.
3. Install NVIDIA and complete physical-console MOK enrollment.
4. Install Tailscale, Ollama, BGE-M3, TabbyAPI, and system packages.
5. Mount both NAS shares.
6. Restore data and home from approved baselines.
7. Rebuild venvs and Chroma.
8. Restore services and secrets.
9. Restore all backup files, including `brain-backup-all`.
10. Verify the service entry point, enable the timer, and run end-to-end tests.

## Timeshift

Use RSYNC mode on the ext4 data disk, exclude `/home/**` and `/mnt/data/**`, and retain three daily snapshots. Before kernel or NVIDIA changes:

```bash
sudo timeshift --create --comments "before kernel or NVIDIA change" --tags O
sudo timeshift --list
```

For unbootable Ubuntu, boot an Ubuntu live USB in UEFI mode, install Timeshift, select `/dev/sda1` as snapshot storage, restore root to `/dev/nvme0n1p2` and EFI/GRUB to `/dev/nvme0n1p1`. Timeshift cannot recreate MOK state held in UEFI NVRAM.

## Residual risks

No protection against a shared physical event affecting tower and NAS, malicious root using stored NAS credentials, corruption deliberately promoted into a baseline, unread silent corruption, exact ACL/xattr loss through CIFS, failure of ASUSTOR internal replication, or loss of firmware MOK state. Perform periodic scratch restores.