ADR-016 Network Topology for 2026 OpenShift Engagement

Status: superseded by adr-019-hybrid-network-rollout-aci-a-ai-only-evpn-b-target.

Context and Problem Statement

The initial OpenShift clusters require a network topology that meets OpenShift requirements while remaining compatible with the DC 3.0 intent-based operating model and long-term SDN direction.

Earlier drafts mentioned re-using Cisco ACI for initial clusters, but the target topology for 1H2026 is the NX-OS-based EVPN/VXLAN spine-leaf fabric as described in the network work packages.

Decision Drivers

• Must support OpenShift platform requirements (API/ingress VIP reachability, predictable MTU, and routed multi-tenancy).
• Must align with DC 3.0 delivery: NetBox-as-intent, deterministic config generation, and full replace deployment.
• Prefer open standards to reduce lock-in and simplify long-term interoperability (BGP EVPN + VXLAN).
• Avoid introducing an additional controller/policy abstraction layer that conflicts with the desired intent pipeline.
• Avoid known scalability/operability risks with ACI floating L3 behavior for our expected tenant and routing scale.

Considered Options

1. Cisco ACI-based fabric (rejected)
2. Cisco Nexus 9000 (NX-OS) EVPN/VXLAN spine-leaf (chosen)
3. Hybrid approach: ACI for baseline, EVPN/VXLAN for future SDN transition (rejected)

Decision Outcome

Historical chosen option (at the time): use a NX-OS-based EVPN/VXLAN spine-leaf fabric for initial and production clusters.

This provides a standards-aligned underlay/overlay model (BGP + EVPN + VXLAN) that is compatible with the DC 3.0 operating model and scales as the tenant count and routing policy grow.

Explicitly not chosen: Cisco ACI for cluster networking.

Rationale summary:

• ACI’s floating L3 and policy model introduces an additional abstraction that does not map cleanly to NetBox-rendered intent and replace-style delivery.
• We prefer a fabric design where control-plane and operational evidence can be validated directly from generated configuration artifacts and AAP-orchestrated vendor CLI evidence profiles, with optional deeper analysis tooling when required.

Implementation details for the active rollout are now split by location and captured in adr-019-hybrid-network-rollout-aci-a-ai-only-evpn-b-target.

Pros and Cons of the Options

Option 1 – Cisco ACI-based fabric (rejected)

Pros

• Leverages an existing vendor stack and familiar operational tooling.

Cons

• Adds a controller/policy layer (ACI model) that does not align with DC 3.0 intent delivery.
• Floating L3 behavior and related scalability constraints are a poor fit for the expected tenant and routing scale.
• Harder to validate “intent → rendered config → runtime evidence” using the DC 3.0 toolchain.

Option 2 – Cisco Nexus 9000 (NX-OS) EVPN/VXLAN spine-leaf (chosen)

Pros

• Uses open standards (BGP EVPN + VXLAN) as the core control plane and encapsulation.
• Aligns with DC 3.0 deterministic config generation and the evidence-based validation workflow.
• Clear separation between abstract fabric intent (NetBox model) and concrete implementation profile (NX-OS).

Cons

• Requires investment in a clear, repeatable implementation blueprint and device profile standards.

Option 3 – Hybrid approach (rejected)

Pros

• Could reduce short-term delivery risk if the EVPN/VXLAN blueprint is not ready.

Cons

• Dual-operating models increase complexity and make validation/auditing harder.
• Creates avoidable migration work while not improving long-term alignment.