Finish dm-edworth bootstrap: fix Tailscale install, add system-layout meshcore_capture support

- base: Tailscale's per-codename .nosetup.sh install scripts are gone
  (404); switch to the universal tailscale.com/install.sh, and split
  install/enable (always) from auth (only when a key is given) so a
  node without an authkey still gets Tailscale installed, just not
  authenticated.
- meshcore_capture: the current upstream installer (v2.0.0+, what a
  fresh install now produces) uses a completely different layout than
  dm-baldock/dm-ashwell's older flat ~/.meshcore-packet-capture setup —
  dedicated system user, /opt + /etc/meshcore-packet-capture,
  config.d/*.toml, meshcore-packet-capture.service. Added
  meshcore_capture_layout (system/legacy) so the role targets the right
  paths/service name per host without touching the two working legacy
  nodes. New 99-user.toml.j2 replicates the old MQTT broker coverage
  (letsmesh us/eu, meshrank, ukmesh, meshmapper) in the new config.d
  override format.
- dm-edworth: fill in serial_port now that the MeshCore device is
  connected; confirmed running and publishing to all 5 brokers.
- README: fix broken list formatting, document the installer's
  required manual step and why.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

README.md

@@ -13,40 +13,52 @@ Ansible playbooks for deploying MeshCore monitoring nodes (Raspberry Pi Zero W /
 ## Prerequisites
 
 **Local machine:**
+
 ```bash
 pip install ansible
 # or: sudo apt install ansible
 ```
 
 **New Pi node checklist:**
-1. Flash Raspberry Pi OS Lite (Trixie), connect to WiFi
-2. Install Tailscale and join the network
-3. Ensure `david` user exists with sudo access
-4. Connect the MeshCore device via USB, then find its serial ID:
-   ```bash
-   ls /dev/serial/by-id/
-   ```
-5. Set `serial_port` in `ansible/host_vars/<hostname>.yml`
 
-SSH key auth is required. From this machine:
-```bash
-ssh-copy-id david@<hostname>.tail740bb.ts.net
-```
+1. Flash Raspberry Pi OS Lite (Trixie) with Raspberry Pi Imager, using "Edit Settings" (OS customisation) to set hostname, the `david` user + password, and the `dotnetwork` wifi (SSID/password only — the imager only supports one network at flash time; the deployed-location and `dotmobile` networks get added later by the `wifi` role, see below).
+2. **Before ejecting the card**, verify the customisation actually got written — mount the boot partition and check `network-config`/`user-data` aren't just the commented-out stock template (see "Imager gotcha" below). Only `cmdline.txt`'s regdomain getting a fresh timestamp while the rest stay at the image's build date is the tell that it silently failed.
+3. Boot the Pi, find its LAN IP (e.g. from the router's DHCP leases), confirm SSH access with the password you set — no key is seeded at flash time.
+4. Run `ssh-copy-id david@<ip>` **from an interactive terminal** (not through a non-interactive shell/script — it needs a real TTY to prompt for the password) so ansible can connect with a key.
+5. Connect the MeshCore device via USB, then find its serial ID: `ls /dev/serial/by-id/`.
+6. Add `serial_port` (and `wifi_ssid_location`) to `ansible/host_vars/<hostname>/vars.yml`, and the location wifi password to `ansible/host_vars/<hostname>/vault.yml` (see "Vault" below).
+7. Run `site.yml` against just that host, overriding the host's address since Tailscale/DNS won't resolve it yet: `ansible-playbook site.yml --limit <hostname> -e "ansible_host=<ip>"`. This authorizes your SSH keys, joins the deployed-location + dotmobile wifi networks, installs (but does not authenticate) Tailscale, and deploys everything else in one pass.
+8. Tailscale needs one manual step: SSH in and run `sudo tailscale up`, then open the printed URL in a browser to approve the device on your tailnet. (You *can* pass `-e tailscale_auth_key=tskey-...` — from the [admin console](https://login.tailscale.com/admin/settings/keys) — to authenticate non-interactively instead, but there's no stored key anywhere for this repo, so the interactive route is simplest for a one-off node.)
+9. The meshcore-packet-capture installer also needs one manual step (see "meshcore-packet-capture install is interactive" below): SSH in and run `sudo bash -c "$(curl -fsSL https://raw.githubusercontent.com/agessaman/meshcore-packet-capture/main/install.sh)"`, answering its ~3 prompts (service account, install method — pick **1**, IATA/broker config — defaults are fine, `site.yml` overwrites `.env.local` afterward anyway).
+10. Once Tailscale is up, re-run `site.yml` without the `ansible_host` override — it'll resolve via the Tailscale hostname from here on, and will now just write `.env.local` + enable the service since the installer step is already satisfied.
+
+### meshcore-packet-capture install is interactive
+
+The `install.sh` bootstrap has no real non-interactive path for a *fresh* install — its `--update` flag only changes behavior when an installation already exists. It also refuses to run at all with piped stdin (`curl | sudo bash` errors out asking you to download the script first) and its Python layer explicitly opens `/dev/tty` for prompts, which just hangs forever over plain SSH/ansible (no human there to answer). We tried feeding it scripted answers via `script`/a pty and it's not worth the fragility — just run it manually once per node (step 9 above); `ansible/roles/meshcore_capture/tasks/main.yml`'s `creates:` guard means ansible never touches it again afterward.
+
+### Imager gotcha (2026-07)
+
+The Raspberry Pi Imager available via Flathub (`org.raspberrypi.rpi-imager`) is stuck on **1.9.6** and there's no newer `.deb`/Flatpak in the Ubuntu or Flathub repos either — Flathub hasn't published the 2.0.x rewrite. 1.9.6 silently fails to apply OS customisation (hostname, user, SSH, wifi) on newer Raspberry Pi OS Trixie images: it only writes the kernel `cfg80211.ieee80211_regdom=` cmdline parameter and leaves `user-data`/`network-config` as the stock commented-out template, with no error. The result looks exactly like a wifi problem (Pi never appears on the network) but is actually "the card has no credentials on it at all."
+
+Fix: grab the real `.deb` from the [GitHub releases page](https://github.com/raspberrypi/rpi-imager/releases) (e.g. `rpi-imager_2.0.10_amd64.deb`) and `sudo dpkg -i` it — that version writes the customisation correctly.
 
 ## Usage
 
 **Deploy to a single host (recommended for first run / testing):**
+
 ```bash
 cd ansible
 ansible-playbook -i inventory.yml site.yml --limit dm-edworth
 ```
 
 **Deploy to all nodes:**
+
 ```bash
 ansible-playbook -i inventory.yml site.yml
 ```
 
 **Dry run:**
+
 ```bash
 ansible-playbook -i inventory.yml site.yml --limit dm-edworth --check
 ```
@@ -57,20 +69,25 @@ You'll be prompted for a Tailscale auth key — leave blank if the node is alrea
 
 ## What it does
 
-1. **base** — apt upgrade, installs screen/pipx/vnstat/git, sets MOTD, installs and authenticates Tailscale
-2. **meshcore_cli** — installs `meshcore-cli` via pipx
-3. **meshcore_capture** — runs the agessaman/meshcore-packet-capture install script, writes `.env.local` config, enables `meshcore-capture.service`, deploys update/log helper scripts
-4. **scripts** — deploys `voltage.sh` and `bandwidth.sh`
+1. **wifi** — configures NetworkManager connections for `dotnetwork` (home), `dotmobile` (phone hotspot, field troubleshooting fallback), and the host's deployed-location network
+2. **base** — apt upgrade, installs screen/pipx/vnstat/git, sets MOTD, authorizes SSH keys for both laptop partitions, installs Tailscale (always) and authenticates it (only if `tailscale_auth_key` is set — otherwise run `sudo tailscale up` manually once, see checklist above)
+3. **meshcore_cli** — installs `meshcore-cli` via pipx
+4. **meshcore_capture** — runs the agessaman/meshcore-packet-capture install script (skipped once already installed — see "meshcore-packet-capture install is interactive" above, this needs a manual first run), writes `.env.local` config, enables `meshcore-capture.service`, deploys update/log helper scripts
+5. **scripts** — deploys `voltage.sh` and `bandwidth.sh`
 
 ## Config
 
-Shared MQTT config lives in `group_vars/meshcore.yml`. Per-host serial port is in `host_vars/<hostname>.yml`.
+Shared MQTT config lives in `group_vars/meshcore.yml`. Per-host serial port and wifi SSID are in `host_vars/<hostname>/vars.yml`.
 
 Running the playbook again re-applies `.env.local` and restarts the service if it changed — safe to run on already-deployed nodes.
 
-## Credentials note
+## Vault
 
-`group_vars/meshcore.yml` contains MQTT credentials in plaintext. Consider encrypting with Ansible Vault if this repo is shared:
+`group_vars/all/vault.yml` (shared wifi/SSH secrets) and `host_vars/<hostname>/vault.yml` (per-host deployed-location wifi password) are encrypted with Ansible Vault. `ansible.cfg` points at `../.vault_pass` (gitignored, not committed) for the password — ask David for a copy, or generate a fresh one and re-encrypt if starting over:
 ```bash
-ansible-vault encrypt_string 'yourpassword' --name mqtt_ukmesh_password
+ansible-vault view --vault-password-file ../.vault_pass group_vars/all/vault.yml
+ansible-vault edit --vault-password-file ../.vault_pass host_vars/dm-edworth/vault.yml
 ```
+`*/vault.yml.example` shows the expected keys.
+
+`group_vars/meshcore.yml` (MQTT credentials) is still plaintext — consider moving it into the vault too if this repo is shared further.

ansible/group_vars/all/vars.yml

@@ -16,3 +16,8 @@ wifi_password_dotnetwork: "{{ vault_wifi_dotnetwork }}"
 
 wifi_ssid_dotmobile: "dotmobile"
 wifi_password_dotmobile: "{{ vault_wifi_dotnetwork }}"
+
+# meshcore-packet-capture install generation — see roles/meshcore_capture/tasks/main.yml.
+# New nodes get the current upstream layout by default; older nodes override
+# this to "legacy" in their own host_vars.
+meshcore_capture_layout: "system"

ansible/host_vars/dm-ashwell/vars.yml

@@ -3,3 +3,6 @@ serial_port: /dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_1C:DB:D4
 # Deployed-location wifi
 wifi_ssid_location: "awesome"
 wifi_password_location: "{{ vault_wifi_location }}"
+
+# Predates the current upstream installer's system-service layout
+meshcore_capture_layout: "legacy"

ansible/host_vars/dm-baldock/vars.yml

@@ -3,3 +3,6 @@ serial_port: /dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_98:3D:AE
 # Deployed-location wifi
 wifi_ssid_location: "H-NeT"
 wifi_password_location: "{{ vault_wifi_location }}"
+
+# Predates the current upstream installer's system-service layout
+meshcore_capture_layout: "legacy"

ansible/host_vars/dm-edworth/vars.yml

@@ -1,5 +1,4 @@
-# Find this by connecting the MeshCore device then running: ls /dev/serial/by-id/
-serial_port: FILL_IN_SERIAL_PORT
+serial_port: /dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_10:B4:1D:E7:FA:C8-if00
 
 # Deployed-location wifi
 wifi_ssid_location: "Ridgeway"

ansible/roles/base/tasks/main.yml

@@ -26,19 +26,17 @@
     mode: "0644"
   become: true
 
-- name: Add Tailscale apt signing key
+- name: Download Tailscale install script
   get_url:
-    url: https://pkgs.tailscale.com/stable/debian/bookworm.nosetup.sh
-    dest: /tmp/tailscale-setup.sh
+    url: https://tailscale.com/install.sh
+    dest: /tmp/tailscale-install.sh
     mode: "0755"
-  when: tailscale_auth_key != ""
 
 - name: Run Tailscale install script
-  shell: sh /tmp/tailscale-setup.sh
+  shell: sh /tmp/tailscale-install.sh
   args:
     creates: /usr/bin/tailscale
   become: true
-  when: tailscale_auth_key != ""
 
 - name: Enable and start tailscaled
   systemd:
@@ -46,7 +44,6 @@
     enabled: true
     state: started
   become: true
-  when: tailscale_auth_key != ""
 
 - name: Authenticate Tailscale
   shell: tailscale up --authkey {{ tailscale_auth_key }}

ansible/roles/meshcore_capture/handlers/main.yml

@@ -1,6 +1,6 @@
 ---
 - name: restart meshcore-capture
   systemd:
-    name: meshcore-capture
+    name: "{{ 'meshcore-capture' if meshcore_capture_layout == 'legacy' else 'meshcore-packet-capture' }}"
     state: restarted
   become: true

ansible/roles/meshcore_capture/tasks/main.yml

@@ -1,23 +1,54 @@
 ---
+# This is a genuine first-run interactive installer (asks for service
+# account, install method, IATA/broker config) with no real non-interactive
+# path for a fresh install — its own --update flag only changes behavior
+# when an install already exists. So: run it manually once per node,
+# answering its ~3 prompts (see README), then ansible's `creates:` guard
+# skips it forever after. Do NOT try to script answers into it — it insists
+# on a real controlling tty and hangs waiting for one over plain SSH/ansible.
+#
+# meshcore_capture_layout distinguishes two generations of this upstream
+# installer:
+#   system (default) — install method 1, current (v2.0.0+) upstream default.
+#     Dedicated meshcore-capture system user, /opt + /etc/meshcore-packet-capture,
+#     config.d/*.toml, service meshcore-packet-capture.service.
+#   legacy — what dm-baldock/dm-ashwell were set up with (older installer):
+#     flat ~/.meshcore-packet-capture, .env.local, service meshcore-capture.service.
+#     Kept only so this role stays a no-op/safe on those two nodes; new nodes
+#     should use "system".
 - name: Run meshcore-packet-capture install script
   shell: |
     bash <(curl -fsSL https://raw.githubusercontent.com/agessaman/meshcore-packet-capture/main/install.sh)
   args:
     executable: /bin/bash
-    creates: /etc/systemd/system/meshcore-capture.service
-  become: false
+    creates: >-
+      {{ '/etc/systemd/system/meshcore-capture.service' if meshcore_capture_layout == 'legacy'
+         else '/etc/systemd/system/meshcore-packet-capture.service' }}
+  become: true
 
-- name: Write .env.local config
+- name: Write .env.local config (legacy layout)
   template:
     src: env.local.j2
     dest: "{{ ansible_env.HOME }}/.meshcore-packet-capture/.env.local"
     mode: "0640"
   become: false
+  when: meshcore_capture_layout == 'legacy'
+  notify: restart meshcore-capture
+
+- name: Write user config overrides (system layout)
+  template:
+    src: 99-user.toml.j2
+    dest: /etc/meshcore-packet-capture/config.d/99-user.toml
+    owner: root
+    group: meshcore-capture
+    mode: "0640"
+  become: true
+  when: meshcore_capture_layout != 'legacy'
   notify: restart meshcore-capture
 
 - name: Enable and start meshcore-capture service
   systemd:
-    name: meshcore-capture
+    name: "{{ 'meshcore-capture' if meshcore_capture_layout == 'legacy' else 'meshcore-packet-capture' }}"
     enabled: true
     state: started
     daemon_reload: true
@@ -25,14 +56,19 @@
 
 - name: Deploy meshcore-capture-update script
   copy:
-    content: "#!/usr/bin/env bash\nbash <(curl -fsSL https://raw.githubusercontent.com/agessaman/meshcore-packet-capture/main/install.sh)\n"
+    content: >-
+      #!/usr/bin/env bash
+
+      bash <(curl -fsSL https://raw.githubusercontent.com/agessaman/meshcore-packet-capture/main/install.sh){{ ' --update' if meshcore_capture_layout != 'legacy' else '' }}
     dest: "{{ ansible_env.HOME }}/meshcore-capture-update.sh"
     mode: "0755"
   become: false
 
 - name: Deploy meshcore-capture-logs script
   copy:
-    content: "#!/usr/bin/env bash\nsudo journalctl -u meshcore-capture -f\n"
+    content: |
+      #!/usr/bin/env bash
+      sudo journalctl -u {{ 'meshcore-capture' if meshcore_capture_layout == 'legacy' else 'meshcore-packet-capture' }} -f
     dest: "{{ ansible_env.HOME }}/meshcore-capture-logs.sh"
     mode: "0755"
   become: false

ansible/roles/meshcore_capture/templates/99-user.toml.j2

@@ -0,0 +1,97 @@
+# MeshCore Packet Capture - user overrides
+# Managed by Ansible - local changes will be overwritten on next playbook run
+#
+# [[broker]] tables merge by name across config.d/*.toml, so the letsmesh-us
+# and letsmesh-eu entries below only add owner identity to the connection
+# details already installed as config.d/10-letsmesh.toml.
+
+[general]
+iata = "{{ packetcapture_iata }}"
+log_level = "{{ packetcapture_log_level }}"
+
+[update]
+repo = "{{ packetcapture_update_repo }}"
+branch = "{{ packetcapture_update_branch }}"
+
+[serial]
+ports = ["{{ serial_port }}"]
+
+[capture]
+connection_type = "serial"
+advert_interval_hours = {{ packetcapture_advert_interval_hours }}
+owner_public_key = "{{ packetcapture_owner_public_key }}"
+owner_email = "{{ packetcapture_owner_email }}"
+
+[[broker]]
+name = "letsmesh-us"
+
+[broker.auth]
+owner = "{{ packetcapture_owner_public_key }}"
+email = "{{ packetcapture_owner_email }}"
+
+[[broker]]
+name = "letsmesh-eu"
+
+[broker.auth]
+owner = "{{ packetcapture_owner_public_key }}"
+email = "{{ packetcapture_owner_email }}"
+
+# MeshRank
+[[broker]]
+name = "meshrank"
+enabled = true
+server = "meshrank.net"
+port = 8883
+transport = "tcp"
+qos = 0
+retain = false
+
+[broker.tls]
+enabled = true
+verify = true
+
+[broker.auth]
+method = "none"
+topic_token = "{{ mqtt_meshrank_token }}"
+
+[broker.topics]
+packets = "meshrank/uplink/{TOKEN}/{PUBLIC_KEY}/packets"
+status = "meshrank/uplink/{TOKEN}/{PUBLIC_KEY}/status"
+
+# UKMesh (custom broker, no upstream preset)
+[[broker]]
+name = "ukmesh"
+enabled = true
+server = "mqtt.ukmesh.com"
+port = 443
+transport = "websockets"
+qos = 0
+retain = false
+
+[broker.tls]
+enabled = true
+verify = true
+
+[broker.auth]
+method = "password"
+username = "{{ mqtt_ukmesh_username }}"
+password = "{{ mqtt_ukmesh_password }}"
+
+# MeshMapper
+[[broker]]
+name = "meshmapper"
+enabled = true
+server = "mqtt.meshmapper.net"
+port = 443
+transport = "websockets"
+keepalive = 55
+qos = 0
+retain = true
+
+[broker.tls]
+enabled = true
+verify = true
+
+[broker.auth]
+method = "token"
+audience = "mqtt.meshmapper.net"