Chapter 7 — Storage & Data Layer

7.1 Data Layer Overview

Three engines serve the Platform, each with a distinct role; nothing else is introduced without a Change Order:

As of 2026-07-18 none of the three is installed — this chapter is fully normative, with the §6.5 interim placement (containers on sbsdash-server-prod) as the build path and managed services as the preferred end state.

7.2 Managed vs. Self-Hosted Decision

• Preferred: DigitalOcean Managed Databases for PostgreSQL, MongoDB, and Redis (Valkey) — provider-handled failover, automated backups, patch management, and TLS-enforced connections reduce the operational load on a small team.
• Blocking check (Build Register, carried from §2.3): Managed Database availability in NYC2 is unconfirmed — newer DO capacity often lands in NYC1/NYC3 first. The check is: can all three engines be provisioned in NYC2 attached to the target Prod VPC (prod-vpc-nyc2, §3.1)? Cross-region managed databases are disqualified — data would transit public networking, violating INF-01, the same class of defect as the wiki scrape (§3.3).
• Fallback (interim, buildable now): self-hosted containers per §6.5. The remainder of this chapter is written engine-by-engine to be valid under either deployment, with deltas flagged.
• Migration posture: if the build starts self-hosted and Managed becomes available, migration is dump/restore per engine during a maintenance window, amended by Change Order. The isolation model (§7.4) is deployment-invariant, so no application change is required — only connection endpoints (Vault-managed, Ch. 8).

7.3 Object Storage

DigitalOcean Spaces (S3-compatible), region NYC3 (Spaces is not offered in NYC2; object storage over TLS to a neighboring region is acceptable — unlike database traffic, it is not latency- or VPC-bound):

Rules: all buckets private (no public-read ever — FE family applies to storage too); access exclusively via scoped Spaces keys stored in Vault, one keypair per purpose, never a shared account-wide key (INF-02); TLS enforced on every operation; bucket inventory and lifecycle policies defined in IaC (Ch. 10). CDN feature on Spaces stays off — these are private buckets, and enabling the CDN endpoint would create an unauthenticated public URL surface.

7.4 Data Isolation Boundaries (per-Compartment)

The data and storage slices of the eight-layer model (§1.2), stated per engine. The principle throughout: isolation is enforced by the engine's own privilege system, not by application discipline — a compromised Compartment 03 credential must be structurally unable to read Compartment 08 data.

• PostgreSQL — one database per Compartment (c01_hr … c10_pr) in the cluster; one role per Compartment with CONNECT on its own database only; PUBLIC connect revoked on every database; no cross-database query paths (no postgres_fdw, no dblink extensions installed); platform-shared configuration lives in a separate platform database with its own role. Superuser credentials exist only in Vault, used only by migration tooling.
• MongoDB — one database per Compartment; per-Compartment user with readWrite scoped to its own database; authentication mandatory (--auth), authorization enabled from first boot — never an open dev instance that later gets locked down.
• Redis — Redis ACLs (v6+) with one ACL user per Compartment, key-pattern-restricted (~c01:*), dangerous commands (FLUSHALL, CONFIG, KEYS, SCRIPT) denied to all Compartment users; default user disabled. If workload interference ever matters more than memory efficiency, escalation path is numbered logical databases or separate instances per Compose network — Change Order.
• Connection path (self-hosted interim): Compartment containers join only the data networks they are entitled to (§6.5), so isolation is double-walled — network non-membership and engine ACL. Under Managed Databases the network wall is the VPC + per-database credentials; the engine ACL layer is identical.
• Object storage: per-Compartment prefix (c01/…) with per-Compartment Spaces keys whose policy is prefix-scoped. AI-03 (credential isolation) extends here: an agent's runtime credentials reach exactly its Compartment's database, its Redis keyspace, and its Spaces prefix — nothing else.

7.5 Encryption

• At rest, block storage: DigitalOcean encrypts droplet disks and Volumes at rest (AES-256) at the platform layer. This is accepted as the baseline for the interim self-hosted deployment; no LUKS layer is added on top — key management for host-level LUKS on cloud VMs adds operational fragility (unattended reboot problem) for marginal gain against the relevant threat model (DO platform compromise is out of scope; stolen-disk scenarios are covered by DO's layer).
• At rest, engine level: PostgreSQL — no TDE in community builds; compensating controls are the DO layer plus strict dump handling (§7.6). MongoDB Community — same posture. Redis — ephemeral data, AOF file inherits disk encryption.
• At rest, object storage: Spaces server-side encryption; audit archives additionally carry their own integrity chain (hash-chained, Ch. 13) so tampering is detectable independent of encryption.
• In transit: TLS on every database connection including VPC-internal (zero-trust, NIST SP 800-207 — the private network is not a trust boundary): PostgreSQL ssl=on with sslmode=verify-full in clients; MongoDB tls=true; Redis TLS listener. Internal certs from a platform-internal CA generated and rotated by IaC (not Let's Encrypt — these hosts have no public names). Managed Databases enforce TLS natively.
• Backups/dumps: encrypted client-side (age or GPG, key in Vault) before upload to sbs-dash-backups — a leaked bucket key must not equal readable data. Detail in Ch. 14.

7.6 Operational Rules

• No production data leaves the Prod boundary (§2.2.3): Dev databases are seeded synthetically or from sanitized dumps produced by a reviewed script (PII/tenant content stripped), never raw prod dumps.
• Ad-hoc human access to prod data engines is break-glass only, via bastion (§3.3), with individually attributed credentials (post INF-04 remediation, §2.5), logged as a MON-01 event. Routine inspection happens through dashboards and read-only replicas if/when provisioned.
• Retention: Compartment operational data per SBS function-level policy captured at Discovery (LH-SBS-DISC-001/002 outputs); audit log 7-year minimum (Ch. 13); Redis nothing beyond declared AOF keyspaces.
• Volume/disk capacity is a Prometheus-alerted metric (Ch. 12); databases run on dedicated DO Volumes (not the droplet root disk) so storage scales and migrates independently of the droplet — Volume creation is part of the data-services Build Register item.

7.7 Verification

Gates (LH-SBS-INST-001): cross-Compartment access probe per engine fails (c03 role → c08 database denied in PostgreSQL, MongoDB, Redis ACL, and Spaces prefix); no public listener on any engine port (ss -tlnp + external scan); TLS handshake required on every engine (plaintext connection refused); PUBLIC/default access revoked (engine ACL dump review); dumps encrypted before leaving host (backup pipeline inspection); Dev contains no production data (sampled content check); Spaces buckets private (unauthenticated GET fails); audit bucket writer key cannot DELETE (negative test).