Service Mesh Architecture (OSSM 3, ambient + multi-cluster)

On this page

Topic – Service Mesh Architecture (OSSM 3, ambient + multi-cluster)
Overview
Context
Applicable Principles
Decisions
Current State
Data Plane Modes
Multi-Mesh and Multi-Control-Plane
Tenant isolation definitions
Terminology guardrails
Mesh topology options (baseline boundaries + overlay)
Central / External Control Plane

Topic – Service Mesh Architecture (OSSM 3, ambient + multi-cluster)

Overview

Define how the platform provides service mesh capabilities using OpenShift Service Mesh 3 and how tenants consume it across clusters.

This topic focuses on the architecture choices and the operational model (central vs per-cluster control plane, ambient vs sidecar, and multi-mesh boundaries). Work-package planning lives in WP-18 Service Mesh.

The platform must preserve the Cluster Management posture via Red Hat Advanced Cluster Management and external storage (ODF-managed Ceph) by minimizing per-cluster control plane overhead. This drives a governed service-mesh architecture variant: centralize mesh control planes on dedicated platform-owned clusters instead of workload clusters, keeping workload clusters data-plane-only by default.

Context

Service mesh is used when workloads need one or more of:

mTLS-by-default between services
traffic governance and routing policy
standardized service telemetry and topology
controlled east-west connectivity across clusters

OSSM 3 is evolving rapidly, especially around ambient mode and multi-mesh topologies. The vault therefore treats this as a platform-managed capability that is versioned, scoped, and rolled out deliberately.

Applicable Principles

Security by Design — mesh policy is enforced by default.
Observability by Default — mesh telemetry provides evidence.
Reliability and Resilience — mesh supports availability targets.

Key platform constraints shaping this topic:

Workload clusters are built on Red Hat Advanced Cluster Management and should avoid heavy in-cluster control planes that inflate baseline sizing.
The external Ceph + ODF model assumes storage services run outside workload clusters; the mesh should follow a similar “shared platform capability” pattern.
Multi-mesh support in ambient mode is still limited; default topology should follow the documented OSSM/Istio support matrix.

Decisions

Default posture (ambient-first where supported): prefer ambient mode (ztunnel + waypoints) when the chosen topology and required features are GA/supported per the OSSM 3.2 feature support tables. OSSM 3.2 release notes announce GA for core ambient features, while the installing guide still labels ambient mode as Technology Preview (see OSSM 3.2 ambient installing docs). Production rollout is gated via adr-017-service-mesh-ambient-mode-production-exception; sidecar remains the fallback for unsupported features/topologies.
Coexistence guardrail: Red Hat documents coexistence limitations: the installing guide warns that an ambient mesh alongside a sidecar mesh “has not been thoroughly validated” and recommends installing ambient only on clusters without an existing OSSM installation; it is also not compatible with OSSM 2.6 or earlier (see OSSM 3.2 ambient installing docs). The OSSM 3.2 feature support tables further mark mixing sidecar and ambient namespaces as DP and deploying ambient on a cluster with an existing sidecar mesh as NA. Treat ambient+sidecar coexistence as unsupported unless Red Hat explicitly documents GA/support for the exact topology; internal validation cannot change vendor support posture.
Control plane model: provide a platform-managed control plane on dedicated service-mesh control plane clusters, with clear boundaries for who owns upgrades and how workload clusters attach.
Multi-mesh: use separate meshes + trust domains to create explicit administrative boundaries and reduce blast radius between tenants/domains. Use revisions only for canary upgrades/rollback within a mesh where supported (sidecar mode in OSSM 3.2); revisions are not an isolation boundary. Use ambient multi-mesh only where OSSM/Istio support is documented; otherwise use sidecar-based meshes for those boundaries.
Multi-cluster: choose the topology per environment and data plane mode. Sidecar-mode multicluster deployment models are GA (see OSSM 3.2 Istio deployment and lifecycle table); ambient mode support is limited (multi-primary DP; other topologies NA) per the OSSM 3.2 ambient mode table.
Service-mesh architecture variant: treat the centralized-control-plane pattern as a governed platform default (owned by the platform team, not a git branch). Deviations (in-cluster control planes or non-standard topologies) require an explicit ADR and CODEOWNERS review.

Current State

Data Plane Modes

Ambient mode (OSSM 3.2 core features GA; feature-dependent): node-level ztunnel provides L4 mTLS and L4 policy enforcement; L7 features (routing/authz) require waypoint proxies (see adr-017-service-mesh-ambient-mode-production-exception and the OSSM 3.2 feature support tables).
Sidecar mode (current production default): per-pod proxy provides the broadest L7 feature set and most predictable support posture; remains the default until ambient mode is approved/validated for production (see adr-017-service-mesh-ambient-mode-production-exception).

Multi-Mesh and Multi-Control-Plane

Sidecar mode: use revision-based control planes to run multiple versions side-by-side for canary upgrades/rollback (GA; see OSSM 3.2 Istio deployment and lifecycle table). Revisions are for upgrades/rollback, not isolation.
Ambient mode: RevisionBased upgrades are NA and InPlace upgrades are GA (see OSSM 3.2 ambient mode table); plan upgrades and rollback without revision-based canaries.
Use selectors/labels (including Istio discovery selectors where applicable) to scope which namespaces are onboarded to each control plane; this is an operational guardrail and not a tenant-isolation/security boundary.
Treat mesh membership as an opt-in platform contract (not a default for all workloads).
Document ambient-mode limitations and any sidecar-only mesh cases for required L7 features.

Tenant isolation definitions

This topic uses “tenant isolation” in three distinct senses:

Administrative boundary: who owns/operates the control plane, upgrades, and mesh-wide policy (platform team vs tenant team).
Trust domain: the identity and certificate boundary for mTLS; a shared trust domain means workloads share trust roots and identity semantics.
Blast radius: the scope of impact of outages, upgrades, and policy mistakes (shared vs per-tenant/per-env meshes).

Scoping a control plane with labels/selectors (including discovery selectors) reduces accidental onboarding and operational blast radius, but it is not an access-control boundary between tenants.

Terminology guardrails

Mesh boundary: the configuration scope for a set of workloads managed by a control plane (often aligned with a trust domain).
Trust domain: the mTLS identity boundary; separate trust domains mean separate trust roots and identity semantics.
Control plane: the OSSM/Istio control plane (e.g., istiod) that distributes config and establishes trust for a mesh; can be hosted centrally (external control plane topology).
Revision: a versioned control plane instance used for canary upgrades/rollback where supported (sidecar mode in OSSM 3.2); not a tenant-isolation boundary.
ztunnel: ambient node-level component providing L4 mTLS and L4 policy enforcement.
Waypoint: L7 proxy used in ambient mode for selected namespaces/services to enable L7 routing and authorization.
Not a security boundary: onboarding selectors/labels and “explicit attachment” reduce operational mistakes but do not isolate tenants; isolation requires explicit mesh/trust-domain boundaries plus the underlying Kubernetes and network controls.

Mesh topology options (baseline boundaries + overlay)

The platform can technically run a shared mesh for all tenants, but the architecture must weigh the definitions above. Think of the decision along these axes:

Mesh boundary (control plane + trust domain): shared vs per-tenant/per-domain.
Environment scoping: one mesh shared across environments vs a mesh per environment.
L7 enforcement mechanism: ambient baseline with waypoints for selective L7 vs sidecar-based meshes where required.

Options A–C describe baseline boundary choices. Option D is an overlay strategy that can be applied on top of A or C (and sometimes B) to selectively add L7 enforcement without changing the underlying mesh boundary.

Option A — Single shared mesh across tenants

Boundary: shared platform-operated control plane + shared trust domain.
Blast radius: largest (shared upgrades/policy changes affect all participants).
Notes: segmentation relies on mesh policy + underlying cluster tenancy; not a hard tenant-isolation boundary.

Option B — Per-tenant (or per-domain) meshes

Boundary: separate (platform-operated) control plane per tenant/domain + separate trust domain.
Blast radius: smallest (upgrade/policy failures contained per mesh).
Notes: higher control plane count/overhead; ambient multi-mesh may be limited (may require sidecar-only meshes); preferred when compliance/change independence outweigh overhead.

Option C — Shared mesh per environment (e.g., prod vs non-prod)

Boundary: separate (platform-operated) control plane per environment + per-env trust domain.
Blast radius: bounded per environment; still shared between tenants inside that environment.
Notes: tenants share trust roots and upgrade cadence per environment; good default when this is acceptable.

Option D — Overlay: shared L4 ambient + selective L7 enforcement

How it works: one ambient mesh provides shared L4 mTLS/L4 policy via ztunnel; selected namespaces/services enable L7 enforcement via waypoint proxies.
Caution: ambient+sidecar coexistence is vendor-unvalidated in OSSM 3.2; do not treat sidecars as a casual “exception path” inside an ambient mesh. If a workload requires sidecars, prefer a separate mesh boundary (Option B) or a dedicated sidecar-only mesh, and confirm vendor-documented support/constraints for the intended topology via Red Hat OSSM feature support levels, the OSSM ambient coexistence guidance, and the OSSM 3.2 ambient docs.
Isolation impact: improves L7 policy separation and reduces per-pod overhead, but does not create a separate trust domain by itself.
Trade-offs: operational complexity (waypoint lifecycle, policy targeting, troubleshooting) and potential need for separate meshes/gateways for strict boundaries.
Fit: complements A/C only when the required L7 features can be met with waypoints and the OSSM support matrix confirms the topology.

Central / External Control Plane

OSSM 3.2 documents an “external control plane topology” where one cluster hosts and manages the control plane and applications run on one or more data plane clusters (see OSSM 3.2 docs), while keeping data plane components in workload clusters. This requires installing the OpenShift Service Mesh Operator on both the control plane cluster and each data plane cluster, and following the OSSM 3.2 prerequisites and version alignment guidance. This enables:

a consistent security baseline (policies and trust roots)
centralized upgrades with explicit blast radius control
fewer duplicated control plane components across clusters

The preferred pattern is dedicated service-mesh control plane clusters with:

clear sizing baselines that avoid inflating Red Hat Advanced Cluster Management workload clusters
explicit onboarding workflow for workload clusters (trust domain, east-west gateway, and mesh membership)
defined failure-domain boundaries for each mesh (per tenant, per environment, or per policy domain)

Observability and Policy

Mesh telemetry integrates with Security Operations Center.
For platform-level admission policies, use policy-as-code admission control; mesh policy focuses on in-mesh identity and service-to-service controls.

Network and MTU Constraints

Ambient mode and overlay tunnels increase MTU sensitivity. The mesh baseline assumes the EVPN/VXLAN fabric and MTU posture defined in topic-network-nxos-evpn-implementation and WP-01 DC3.0 Network and Supporting Services.

Future Work

Define the supported “platform service mesh profiles” (ambient-only, ambient+waypoints, sidecar-required).
Define multi-mesh naming conventions (trust domains, mesh IDs, revision tags).
Publish upgrade runbooks (sidecar: revision-based canary + rollback; ambient: InPlace upgrade + rollback strategy) and acceptance gates.
Decide the default multi-cluster topology for Location A/B and document east-west gateway patterns.
Document ambient-mode multi-mesh and multi-cluster limitations from current OSSM/Istio docs and record any sidecar-only requirements.