Roadmap & Status¶
Where the code is today¶
Nodewright has shipped its read-only observation bridge and the full write-side of the bond-flip machine — the forward path, the automated recovery ladder (rollback / bounded retry / escalate-when-unprovable), and the I6 data-consistency gate — built and exercised end-to-end against a fake DRBD/KubeVirt harness, and gated default-off. The remaining work is real host actuation (hardware-lab validation) and the replication-upgrade workflow. The plan front-loaded the low-risk, high-value work; the dangerous paths are built but not yet armed.
| Area | Status | Notes |
|---|---|---|
| Design, PRDs, CI, governance | Solid | Architecture brief, security/test PRDs, multi-lane CI on self-hosted runners. |
VMVolumeMigration — RWX migration |
Built, tested | Typed CRD, phase FSM, cold + hot paths, snapshot bookends, checksum verify. |
Observation bridge (NodewrightCluster / AgentStatus) |
Built | Read-only rollup of the DaemonSet's live 21-state machine + per-node heartbeats. |
| Storage abstraction + coordination webhook | Built | Tier-1 capabilities + the ReplicationCoordination axis. Piraeus real, local-path degraded, Portworx declaration-only. |
Node agent + HostOperation protocol |
Built (dormant) | Go DaemonSet, per-node liveness Lease, four host-op primitives — fake-tested, no real host mutation. |
BondModeFlipPlan forward path + 6 invariants |
Built (default-off) | 12-state forward path, safe-halt, instance-fenced lock — all against the fake harness. |
| Recovery ladder + I6 data-consistency gate | Built (default-off) | Rollback / bounded retry / escalate-when-unprovable; I6 distrusts "UpToDate" (oos + generation-UUID + no split-brain, two-sided, fail-closed) and gates every reconnect. |
Real host actuation (bond · networkctl · drbdadm · production DRBD) |
Not built | Gated to hardware-lab validation. |
ReplicationUpgrade, EdgeProfile |
Not built | Interfaces exist; the workflows are future. |
Docs now match code — with the built/gated line drawn
The Architecture pages describe what's built. Where something is designed but not yet armed — real host mutation, rollback, replication upgrade — it's called out inline. This page is the consolidated map.
How it was built: bridge first, then the gated write-side¶
The productization ran C → B, and both have landed:
flowchart LR
C["C — Observation bridge<br/>read-only health surface<br/>✓ shipped"]:::done
B["B0–B3 — Write-side<br/>storage tier · agent · HostOperation<br/>bond-flip FSM · recovery ladder<br/>6 invariants incl. I6<br/>✓ built, default-off"]:::done
N["B4–B6 — Next<br/>replication upgrade<br/>hardware lab · cut-over<br/>blast radius: HIGH"]:::next
A["A — VMVolumeMigration<br/>the RWX migration workflow<br/>✓ shipped, parallel track"]:::done
C --> B --> N
A -.parallel.-> B
classDef done fill:#1F7A78,color:#fff,stroke:#005B5B
classDef next fill:#B94B01,color:#fff,stroke:#7a3200
- C — the observation bridge (shipped). Taught the operator to read and surface the DaemonSet's live state. Shipped value in days, cannot harm the running cluster, and produced the accurate typed state model the rest of the work needed.
- B0–B3 — the write-side (built, default-off). The storage abstraction + coordination webhook (B0),
the node agent +
HostOperationprotocol (B1), the 12-state forward-path bond-flip FSM with its safety invariants (B2), and the automated recovery ladder + I6 data-consistency gate (B3) — all built and tested against a fake DRBD/KubeVirt harness. Over-specified and human-gated because the blast radius is high; no real host is touched yet. - A —
VMVolumeMigration(shipped, parallel). The RWX migration workflow. It automates a runbook the DaemonSet never did, so it advanced on its own track.
What's next, in dependency order:
- B4 — replication upgrade.
ReplicationUpgradeas a first-class, staged workflow on top of the flip. - B5 — hardware lab + real actuation. Wire the real command/filesystem seams and production DRBD coordinator. The first tranche where a flip touches a real host — behind lab validation.
- B6 — cut-over. The operator consumes the agent's liveness leases and takes over from the bash DaemonSet, per cluster.
Release phases¶
- The read-only health surface (
NodewrightCluster.status,NodewrightAgentStatus). VMVolumeMigrationcontroller (cold + hot paths, snapshot bookends, checksum verify).- The DaemonSet stays behaviorally untouched — the operator wraps and observes it.
-
BondModeFlipPlanis a real controller — forward path and recovery ladder, built default-off. - The I6 data-consistency gate distrusts "UpToDate" and fences every replication reconnect.
- The DaemonSet's node role becomes a thin Go agent that renews a per-node liveness Lease.
- The pluggable storage interface +
HostOperationprotocol go live (dormant actuation). - Real host actuation and
ReplicationUpgradecomplete the takeover. -
EdgeProfileships as a signed, versioned artifact.
- Multi-cluster fleet aggregation (read-only view; every action stays a per-cluster CRD apply).
- Boot-recovery, snapshot, and per-node version policies.
- Customer self-service for the safe operations.
Design principles that don't change¶
- Orchestrate, don't re-implement — Palette, LINSTOR, KubeVirt, and Kairos stay; Nodewright coordinates them.
- Operator owns intent, agent owns mutation.
- Fail loud; honor gates; never partially proceed.
- Product-neutral by construction — no customer or hardware name is compiled into the operator; every site is a profile.