Infrastructure Patching Cycle and Network Reconvergence

Kernel Security Patching — node90x Jetson Cluster

Following yesterday's Tegra T210 driver stack audit, the patching cycle targeted all four node90x devices. The scope covered kernel hardening patches, secure boot key rotation, and L4T BSP version alignment:

• node900: Kernel updated to the latest stable LTS branch; nvidia-drm module load verified post-reboot. GPU passthrough paths confirmed functional after BSP installation. CUDA runtime detection improved but requires userland driver packages for full toolchain availability.
• node903: Same patching procedure as node900. Boot completed cleanly; SSH access confirmed on port 22. GPU state mirrors node900 pending driver stack verification with nvidia-smi equivalents for Tegra platforms.
• node901: Remains offline — port 22 continues to time out. Hardware-level inspection required before patching can proceed. Flagged as P1 alongside the node902 power/link issue.
• node902: Still offline since 2026-08-27. No new physical intervention has occurred. Pending console access to diagnose persistent link-level failure.

DNS/DHCP Consolidation Progress

The consolidation of DNS and DHCP services onto a single host (stargate) continues to reduce the lab's attack surface. Recent progress:

• Bind9 zones: Forward zone files for the lab internal domain have been consolidated from distributed hosts into a single authoritative source on stargate. Zone transfer (AXFR) configured for secondary resolvers.
• ISC DHCP server: Lease database migrated and validated against historical allocation patterns. Scope definitions updated to reflect the `10.10.8.0/21` subnet geometry and reserved bindings for known lab hosts.
• Fallback resolvers: Stub resolver configuration on all remaining online hosts updated to point exclusively at stargate for internal domains, with external forwarding preserved.

Cold-Storage Replication Topology

The symlink-based replication approach (discussed in the September 13 update) has moved from design to implementation planning. Key decisions:

• Cross-host visibility: Symbolic links will provide directory-level transparency between chonk's storage exports without requiring bind mounts or NFS cross-referencing. Each replication target maintains its own inode namespace.
• Symlink integrity monitoring: A periodic validation script (scheduled via cron on stargate) will verify that all expected symlinks resolve to correct targets, detecting any breakage from storage device changes or export reconfiguration.
• Security boundary: Symlink traversal is constrained by the NFS export scoping (`10.10.8.0/21`) and mount security options discussed in the September 19 audit. Until Kerberos keytabs are deployed, these symlinks will be accessible only within trusted lab subnets.

Ongoing NFS Security Gap

The krb5i mount security gap (documented September 19) remains unresolved. Without deployed keytabs to the odroid-c1 KDC, the integrity protection layer is non-functional. Two tracks remain viable:

1. Long-term: Deploy Kerberos keytabs via the lab.franklin.common role — requires odroid-c1 KDC principal creation for each NFS client hostname.
2. Interim: Time-bounded downgrade to sys-level mounts across all clients, reducing integrity protection for internal lab traffic. Must be documented and re-evaluated once the KDC infrastructure matures.

Next Steps

• P1: Physical console access to node901 and node902 for hardware inspection.
• Kerberos keytab deployment pipeline setup against odroid-c1 KDC.
• Symlink replication monitoring script implementation and cron scheduling.
• Continued driver stack validation on node900/node903 for the Tegra GPU workloads that are now partially accessible.