Red Hat Quay Solution Architecture

Introduction and Goals

This HLD describes the selected target Red Hat Quay registry pattern for DC 3.0. It turns the product-neutral registry ABB into an implementable solution architecture for the red-hat-quay SBB, while leaving service-onboarding evidence to the SBB activation gates.

Primary goals:

• Provide a controlled enterprise OCI registry for OpenShift platform, workload, and automation flows.
• Support restricted-network operations by mirroring approved upstream content into a governed internal endpoint.
• Define the target tiering model: one main ACM-governed seed Quay tier for approved platform content, one DCS-internal platform Quay tier in each AM1 and AM4 Datacenter Services cluster, and one shared tenant-facing Quay service tier per datacenter for tenant push/pull.
• Make the tenant access model explicit: tenants use tenant-facing Quay endpoints and do not directly use DCS-internal registry services; the main seed tier is not the normal tenant endpoint.
• Preserve a path toward stronger disconnected or air-gapped models without imposing that operating cost on Phase 1.
• Make registry security, evidence, retention, backup, and operations explicit enough for service onboarding.

Stakeholders:

Constraints

• adr-005-datacenter-services-baseline selects Red Hat Quay as the Registry SBB for Datacenter Services.
• adr-011-connected-vs-disconnected defines Phase 1 as restricted-network, not fully disconnected or air-gapped.
• adr-004-single-active-phased-rollout defines rollout sequencing; the Quay target topology itself is location-neutral and applies per available Datacenter Services locality.
• Quay production durability requires highly available object storage, database, and cache dependencies; local storage and raw block/PVC storage are not acceptable production blob backends.
• Quay scan visibility does not by itself enforce release policy; deployment gates require separate policy tooling.
• ACM governs fleet lifecycle and policy. Quay provides trusted content distribution and registry evidence, not ACM governance replacement.
• Red Hat generally prefers a single shared Quay registry with organizations, repositories, and RBAC. The RWS target uses distinct platform-facing and tenant-facing registries only because the DCS security boundary, endpoint exposure, and configuration differ by audience.

Context and Scope

Business Context

The registry service is a shared DC 3.0 platform capability. It reduces uncontrolled external image consumption, gives platform teams a governed mirror and artifact endpoint, and supports fleet lifecycle operations such as operator catalogs and update content.

Technical Context

In Scope

• Red Hat Quay Operator deployment on OpenShift.
• Quay application, Clair, mirror workers, and registry endpoint design.
• Integration with object storage, PostgreSQL, Redis, OIDC, TLS, monitoring, and backup.
• Tenant model, access model, automation credentials, and evidence controls.
• Main seed, DCS-internal platform, and tenant-facing registry tier responsibilities.

Out of Scope

• Formal product selection for signing, SBOM, and admission-control tooling.
• Full air-gapped transfer logistics and offline vulnerability-feed handling.
• Detailed implementation manifests, runbook commands, bucket names, and account names.
• Dedicated tenant-owned Quay instance design, except for stating the exception boundary.

Solution Strategy

Use Operator-managed Quay instances as a tiered registry fabric. The main ACM-governed seed tier curates upstream content for the fleet. The AM1 and AM4 Datacenter Services clusters each host a DCS-internal platform Quay instance for platform and cluster lifecycle pull paths. Tenant publishing and workload image pulls use one shared tenant-facing Quay service per datacenter so tenant traffic is not direct access to DCS-internal services.

This is a workable and recommended RWS target because the platform-facing and tenant-facing audiences need different exposure and security posture. It is not a recommendation to deploy one Quay per tenant or one Quay per cluster: tenant isolation remains Quay organizations, teams, repository permissions, and robot accounts inside the tenant-facing service.

Strategy highlights:

• Deploy Quay through the Red Hat Quay Operator for repeatability, OpenShift integration, and lifecycle management.
• Keep state outside Quay pods: PostgreSQL for metadata, Redis for runtime cache/state, and S3-compatible object storage for blobs.
• Mirror curated upstream content into the main seed tier, then synchronize the approved platform subset to each AM1 and AM4 DCS-internal platform Quay instance.
• Make clusters pull platform content from DCS-internal platform Quay and workload content from tenant-facing Quay through configured mirror policies and digest-based references wherever possible.
• Synchronize tenant images between AM1 and AM4 tenant-facing endpoints only for selected repositories, tags, or promoted digests; do not duplicate whole registries by default and do not copy the platform mirror catalog into tenant-facing Quay.
• Use OIDC-backed human access, scoped teams, and robot accounts for automation.
• Enable Clair for vulnerability visibility and feed results into security operations and release governance.
• Treat signing, SBOM, provenance, and admission blocking as adjacent controls that reference registry evidence.

Building Block View

Level 1

Level 2

Runtime View

Mirrored Upstream Content

1. Platform Operations approves an upstream source, repository, organization, and tag pattern.
2. Main seed Quay mirror workers authenticate to the upstream registry where required.
3. Mirrored content lands in a controlled main seed organization or repository.
4. AM1 and AM4 DCS-internal platform Quay instances synchronize the approved subset needed by their local platform and cluster lifecycle flows.
5. OpenShift mirror configuration redirects platform pulls to DCS-internal platform Quay.
6. Security and platform operations monitor mirror backlog, sync errors, scan state, and content freshness across both tiers.

Workload Image Promotion

1. CI builds or imports an image and pushes it to a non-production repository.
2. Release automation signs, scans, labels, and promotes the image according to the selected release policy.
3. The promoted digest is published in the target tenant-facing Quay organization.
4. GitOps references the digest or governed tag.
5. Admission policy verifies the required trust evidence before deployment.
6. If the same tenant image must be available in another datacenter, controlled mirroring copies the selected repository, tag, or digest between AM1 and AM4 tenant-facing Quay endpoints.

Tenant and Cluster Consumption

1. Tenant onboarding creates a tenant-facing Quay organization, teams, repository defaults, robot accounts, retention class, and naming contract.
2. CI/CD pushes workload images and OCI artifacts to the tenant organization in the tenant-facing Quay service endpoint for the target datacenter or service domain.
3. Local OpenShift clusters pull platform lifecycle content from DCS-internal platform Quay and workload content from tenant-facing Quay.
4. The main seed tier is used for approved platform content distribution, bootstrap, recovery, or governed fallback; it is not the normal tenant push/pull endpoint.
5. A dedicated tenant Quay instance requires an exception decision or time-bound exception record with owner, expiry, IAM, mirroring, backup, scanning, logging, and admission-control evidence.

Vulnerability Visibility

1. Quay stores image metadata and blobs after push or mirror sync.
2. Clair indexes the image and reports vulnerability findings.
3. Security tooling and platform dashboards consume scan results and freshness indicators.
4. Release governance decides whether the finding blocks promotion or deployment.

Deployment View

The baseline uses explicit mirroring rather than Quay geo-replication: main-to-DCS mirroring for platform content and tenant-facing cross-datacenter mirroring for selected workload repositories or digests. Multi-site geo-replication is a later design because it needs storage-health, database/cache, and global load-balancing decisions and does not provide selective repository synchronization.

Cross-cutting Concepts

Identity and Access

• OIDC is the target human authentication path through Keycloak.
• Quay organizations map to platform tenants, service domains, or curated mirror domains.
• Teams and default permissions define repository access; superuser rights stay restricted to platform operations.
• Robot accounts are scoped per automation use case and rotated through the secrets-management process.
• Main seed organizations are controlled by Platform Operations and release governance.
• Tenant-facing organizations are the default tenant onboarding unit inside the shared AM1 or AM4 tenant-facing Quay service.
• DCS-internal platform repositories are not tenant push targets.
• Dedicated tenant Quay instances are not self-service; they require an exception decision or time-bound exception record.

Network and TLS

• Registry routes are exposed only through approved Datacenter Services external-profile ingress and load-balancing paths for their audience.
• TLS is mandatory for user-facing and machine-facing endpoints.
• Tenant-facing endpoints and DCS-internal platform endpoints use separate audience-specific routing and firewall policy.
• Backend TLS for database, Redis, and object storage is required where the selected dependency supports it.

Data Protection

• PostgreSQL backup protects registry metadata.
• Object-storage backup or replication protects image and artifact blobs.
• Config bundle backup protects Quay application configuration.
• Restore tests must include push, pull by digest, OIDC login, main-to-DCS mirror sync, tenant-facing cross-datacenter mirror sync, and Clair scan validation.

Observability

• Quay metrics are scraped through the Operator-managed metrics service.
• Alerts cover registry availability, dependency health, object-storage errors, upstream mirror lag, main-to-DCS mirror lag, tenant-facing cross-datacenter mirror lag, garbage collection, scan freshness, certificate expiry, and capacity thresholds.
• Action logs and repository logs feed audit and security operations workflows.

Supply-chain Governance

• Quay stores and distributes images, OCI artifacts, signatures, SBOMs, and related metadata where supported.
• Quay does not replace release governance. Admission policy and promotion automation remain responsible for trust decisions.
• Mirroring policies must account for current vendor limits, including OCI referrers such as Cosign signatures and SBOMs that are not mirrored automatically by repository or organization mirroring.

Architecture Decisions

Accepted Decisions

• Accepted: Red Hat Quay realizes registry in the Datacenter Services baseline through adr-005-datacenter-services-baseline.
• Accepted: Phase 1 is restricted-network and keeps stronger disconnected models out of baseline through adr-011-connected-vs-disconnected.
• Accepted: adr-004-single-active-phased-rollout constrains implementation sequencing; this HLD defines the final Quay target topology without using rollout location labels as architecture tiers.

Implementation Stance

• Accepted in this HLD: Quay Operator is the target deployment model; standalone Quay requires a future exception decision.
• Accepted in this HLD: Quay image blobs use the S3-compatible ODF object path backed by the dedicated Ceph platform baseline; raw block/PVC storage is not the production blob backend, and any other S3 endpoint requires an exception ADR.
• Accepted in this HLD: PostgreSQL and Redis are highly available Datacenter Services dependencies with service-onboarding evidence for ownership, monitoring, lifecycle, and recovery.
• Accepted in this HLD: tenant-facing Quay is separate from DCS-internal platform Quay; tenants do not push to, or directly consume from, DCS-internal registry services.
• Accepted in this HLD: AM1 and AM4 each host one shared tenant-facing Quay service for HCP tenants; tenant isolation is handled by organizations, teams, repository permissions, and robot accounts, while dedicated tenant-owned Quay stays exception-only.
• Accepted in this HLD: dedicated tenant Quay instances are exception patterns with explicit owner, expiry, IAM, mirroring, backup, scanning, logging, and admission-control evidence.
• Adjacent control plane: signing, SBOM, provenance, and admission-control products remain release-governance implementation choices that consume Quay evidence.

Quality Requirements

Risks and Technical Debt

Glossary

Source Notes

• source-red-hat-quay-architecture
• source-red-hat-quay-management
• source-red-hat-quay-usage