Skip to main content

Chapter 3 — Network Architecture

 


3.1 VPC Design

TwoTarget: two dedicated VPC networks, one per environment, both in NYC2:NYC2, no peering:

VPC Environment Members
Peering prod-vpc-nyc2 Prod sbsdash-server-prod, monitor-servers, sbs-wiki (post NYC2 rebuild) None dev-vpc-nyc2 Dev Dev-SBS-server

As built: neither dedicated VPC exists. Current membership:

VPC (actual) CIDR Members Nonedefault-nyc2 10.100.0.0/20 Dev-SBS-server (.2), sbsdash-server-prod (.3), monitor-servers (.4) — Dev and Prod share one flat private network ⚠ default-nyc1 10.116.0.0/20 sbs-wiki (.2) — unreachable from NYC2 private networking 10.120.x Dev-Gitlab-server (§2.2.2)

DesignConstraint constraints,driving statedthe fordecision: accuracya againstDigitalOcean DigitalOcean'droplet's VPC model:assignment is fixed at creation — moving a droplet to a new VPC requires snapshot-and-rebuild. Two paths (Build Register item 18):

  • A(a) Rebuild into dedicated VPCs — recommended. At the current fleet size (4–5 droplets, pre-data-services, pre-tenant-data) rebuilds are cheap: snapshot, re-create in target VPC, re-run configuration. Cost grows sharply once databases and Compartments are live. This path restores the "no shared network" rule of §2.2.3 as a structural guarantee. The wiki NYC2 rebuild is already mandatory, so path (a) folds it into one rebuild wave.
(b) Accept shared default-nyc2 and compensate with strict host firewalls on every droplet. Dev↔Prod isolation then rests entirely on firewall rulesets rather than network non-adjacency — a weaker claim, and a permanent audit caveat against INF-01.

This specification proceeds on path (a); §3.2 onward describes the target VPCs. If path (b) is elected, §3.1–3.2 are amended by Change Order.

DigitalOcean VPC model notes (unchanged): a VPC is a single flat private network per region there is no user-defined sub-subnetting within a VPC, nosubnets, route tables, and noor security groupsgroups; in the AWS sense. Segmentation inside the intra-VPC segmentation is therefore enforced by Cloud Firewalls (tag-scoped) plus host firewalls (§3.5, §3.6),. not by subnet boundaries.

VPCs are region-scoped.scoped, Thiswhich is the technical basis for the sbs-wiki NYC2 rebuild requirement (§2.3): a NYC1 droplet cannot be a member of prod-vpc-nyc2. No VPC peering between Dev and Prod — the environments have no private network path. Promotion of artifacts occurs through the IaC/CI workflow (Chapter 10) via the GitHub repository, never via direct network transfer. requirement.
3.2 Address Plan
Network CIDR NotesState
prod-vpc-nyc2 10.10.0.0/20 AssignedTarget at VPCcreated creation;during DigitalOceanrebuild allocates member addresses from this range automaticallywave
dev-vpc-nyc2 10.20.0.0/20 DeliberatelyTarget — non-overlapping with Prod toand keepwith operatorthe tooling,legacy monitoring10.100/10.116/10.120 configs,ranges, so logs and logstooling remain unambiguous during migration
default-nyc2 (10.100.0.0/20), default-nyc1 (10.116.0.0/20), 10.120.x Legacy — vacated at end of rebuild wave

Reserved documentation convention: Prod privatePrivate IPs are recorded in the Build Register per droplet at VPC enrollment; the authoritative address inventory lives in the IaC state (Chapter 10), not in this document, to avoid drift.

3.3 Internal Routing and Service Communication

All service-to-service traffic between Prod droplets transits the VPC private interface (eth1), never public IPs. Enforced expectations:

  • Application → data services (target) — the application stack on sbsdash-server-prod reaches PostgreSQL, MongoDB, and Redis exclusively on VPC-private addresses.addresses; Datadata services bind to the private interface only;only, theywith have no listener on the public interfacelistener (verified at gate V-seriesLH-SBS-INST-001 network checks, LH-SBS-INST-001)gates).
  • Prometheus scrape paths (deployed, verified 2026-07-17)monitor-servers scrapes exportersnode_exporter on sbsdash-server-prod (10.100.0.3:9100) and sbs-wikidev (10.100.0.2:9100) over the VPC. Exporter ports (9100 node_exporter, application exporters) are admitted only from the monitor-servers private IPnetworking; viaexporters tag-scopedbind firewallprivate-only rule — theyand are unreachable frompublicly. On the publicmonitor interfaceitself, Prometheus (9090), Grafana (3000), and fromlocal othernode_exporter VPC(9100) bind 127.0.0.1 only, fronted by nginx TLS — confirmed by ss -tlnp verification. This matches the target design and is marked verified for the NYC2 members.
  • Wiki isolationscrape (interim nonconformance) BookStack on sbs-wiki originates(NYC1) nois connectionsscraped on its public IP (167.172.135.88:9100), unencrypted, admitted only from the monitor's public IP by ufw rule. Accepted temporarily (metrics only, low sensitivity); eliminated by the NYC2 rebuild, after which the wiki exporter rebinds to its private address like the applicationrest orof datathe tier.fleet.
ItsJump-host / bastion pattern (as built, retained in design) — sbsdash-server-prod accepts SSH only VPC-internalvia trafficProxyJump is inbound scrape fromthrough monitor-servers (ssh -J). This reflectsmakes the monitor a de facto bastion. Consequences now normative: the monitor carries the tightest SSH admission (admin allowlist, §3.5); its role:compromise operationalradius surface,includes noprod tenantSSH datareachability, so it receives priority hardening, patching, and log scrutiny (§1.2).MON-01/MON-02); bastion SSH sessions are logged. Whether the bastion role remains on the monitor or moves to a dedicated minimal droplet is revisited at the rebuild wave. DNS resolution — droplets use systemd-resolved against DigitalOcean resolvers; internal service addressing uses IaC-managed /etc/hosts entries (small fixed fleet) rather than a private DNS zone.. Revisit if the fleet grows pastexceeds ~10 hosts.

Inter-Compartment note:note (unchanged): Compartment isolation at the network layer (one of the eight isolation layers) is realized within sbsdash-server-prod at the Docker Compose network level — each Compartment's containers join aone dedicated compose network;network per Compartment, no twoshared Compartmentbridges, networks share a bridge. Cross-cross-Compartment traffic is denied by non-membership. Docker network29.1.3 non-membership,/ consistentCompose with2.40.3 §2.2.1are ofnow thepresent platformon architecture.prod, dev, and monitor (2026-07-18), so this layer is buildable. Container-level detail is specified in Chapter 6.

3.4 Egress Control

DigitalOcean provides no managed NAT/egress gateway; egress control is enforced at the droplethost firewall layer:

  • Default-deny outbound is not practical at the cloud firewall (Cloud Firewalls support outbound rules but a strict allowlist breaks OS packaging and ACME). Policy instead: restricted-Restricted-egress allowlist at the host firewall (nftables) on each Prod droplet, permitting:
    • TCP 443 to Anthropic API endpointsendpoints; (platform AI traffic)
    TCP 443 to GitHub (deploy pulls;pulls, pinned to GitHub's published IP ranges, refreshed by IaC); TCP 443 to OS package mirrors and container registries enumerated in Chapter 1111; (supply-chain allowlist) TCP/UDP 123 (NTP); to configured pool TCP 443 to Let's Encrypt ACME endpointsendpoints; (TLS issuance, Chapter 5) DNS (53)53 to DigitalOcean resolvers only only. All other outbound denied and logged — egress-deny log lines ship to the observability pipeline (Chapter 12) andas are an anomaly-detection input undera MON-02.02 anomaly input. Unexpected egress attempts from a Compartment container areis a primary indicator for AI-02 (tool allowlisting) and LLM01 (prompt-injection attempting exfiltration) violations. Compartment containers inherit a narrower egress set than the host: only the Anthropic API andplus explicitly allowlisted per-Compartment integration endpoints.endpoints, Enforcedenforced via per-compose-network nftablesfirewall chains; the allowlist ischains, IaC-managed and change-controlled.

    As built: no egress control exists on any droplet (all outbound open). Egress enforcement lands with the host-firewall baseline (§3.6, Build Register item 20).

    3.5 Cloud Firewall Rules (DigitalOcean Cloud Firewalls, tag-scoped)

    ThreeAs built: no Cloud Firewalls exist. Host-level ufw is active on monitor and wiki only; prod and dev have no active firewall at any layer — their public interfaces are protected solely by service bind addresses. This is the most exposed element of the current state and is remediated first in the firewall workstream (Build Register item 19).

    Target — four firewalls, attached by droplet tag:

    FW prod-app → tag prod (sbsdash-server-prod)

    Dir Proto/Port Source / Destination Purpose
    In TCP 443 0.0.0.0/0, ::/0 Tenant surfaces (sbsdash.com, admin., clients.) via reverse proxy
    In TCP 80 0.0.0.0/0, ::/0 ACME HTTP-01 + 301 redirect to 301→443 only
    In TCP 22 Adminmonitor-servers allowlistprivate IPsIP (Build Register)only SSH,SSH key-onlyvia bastion (§3.3) — no direct public SSH
    In TCP 9100,9100 9x+ xxapp (exporters)exporters monitor-servers private IP Prometheus scrape
    Out Per §3.4 Restricted egress

    FW prod-monitor → tag grafana (monitor-servers)

    Dir Proto/Port Source Purpose
    In TCP 443 0.0.0.0/0 monitor.sbsdash.com (Grafana behind IdP auth,IdP, Chapter 9)
    In TCP 80 0.0.0.0/0 ACME + redirect
    In TCP 22 Admin allowlist IPs (Build Register) Bastion SSH — sole public SSH entry to Prod
    Prometheus (9090) and Alertmanager (9093)3000/9090/9100 bind VPC-private only127.0.0.1 — no public rule exists (verified as-built)

    FW prod-wiki → tag wiki (sbs-wiki), post-rebuild)

    Dir Proto/Port Source Purpose
    In TCP 443 0.0.0.0/0 wiki.sbsdash.com (BookStack behind SSO)
    In TCP 80 0.0.0.0/0 ACME + redirect
    In TCP 22 Adminmonitor-servers allowlistprivate IP SSH via bastion
    In TCP 9100 monitor-servers private IP node_exporter scrape (rebinds private post-rebuild)

    FW dev-app → tag develop mirrors prod-app with SSH open to the samefrom admin allowlist and no public 443 requirement (Dev has no production DNS); Devallowlist; web surfaces admit only admin allowlist IPs.only (no production DNS points at Dev — enforcement of this depends on resolving §2.2.2).

    Deviation from prior draft, deliberate: direct public SSH to prod-app is removed in favor of bastion-only SSH, codifying the as-built jump-host pattern — it is the stronger posture. Rule hygiene:hygiene unchanged: firewalls are IaC-defined (Chapter 10);defined; console edits to firewall rules are treated as drift and reverted.reverted; Thethe admin SSH allowlist is a named IaC variable in IaC, reviewed at each IAM-06 access review.

    3.6 Host Firewall (nftables)

    Cloud Firewalls filter at the platform edge; nftablesa host firewall on each droplet provides defense-in-depth and the per-container egress allowlist (§3.4), which Cloud Firewalls alone cannot expressexpress.

    As built: ufw on monitor (22/80/443 in) and wiki (22/80/443 + 9100-from-monitor in); ufw inactive on prod and dev; no egress rules anywhere; no nftables rulesets deployed.

    Target: nftables on all droplets — required because per-container:

    Compartment
      egress chains (one chain per compose network) and logged default-deny outbound are beyond ufw's expressible policy. Migration order: (1) activate baseline host firewall on prod and dev immediately (ufw acceptable as a stopgap — inbound mirror of the Cloud Firewall admit set); (2) replace fleet-wide with IaC-rendered nftables rulesets at the rebuild wave; (3) add per-Compartment chains when Compartment compose networks are created (Chapter 6). Base policy: inbound default-deny (mirror ofmirroring the cloud firewall admit set),firewall, outbound default-deny with the §3.4 allowlist, established/related admitted. Per-Compartment chains: one nftables chain per Compartment compose network implementing that Compartment's egress set. Rulesets are rendered from IaC templates and versioned; the live ruleset hash is a monitored value (MON-01 input)01) so tampering surfaces as drift.
      3.7 Verification

      Network controls in this chapter map to gates in (LH-SBS-INST-001:001): public-listener scan (no data-service or Prometheus ports on public interfaces),interfaces (passes today on monitor per ss -tlnp evidence; wiki exporter public listener is a tracked exception until rebuild); Dev↔Prod isolation probe (no private path),path between environments (fails today: shared default-nyc2; passes after rebuild wave); egress-allowlist test (denied denied-domain attempt logged),logged and(not testable today: no egress control); per-Compartment cross-network denial test.test (pending Chapter 6 build); bastion path test — prod SSH unreachable except via monitor. INF-01 is the controlling SEC-001 requirement throughout.