TCP edge

This page defines the implemented edge contract. Manifest additions, managed TLS delivery, Supervisor relays, protocol policy, private APIs, Envoy/xDS, and the MySQL gateway exist in source with component and local integration tests. The port map and client examples do not establish public deployment or verified connectivity. Check the returned endpoint and protocol readiness before attempting a connection.

Four external ports#

Ports identify protocols, never tenants. Every tenant shares the applicable protocol port; SNI hostnames or platform-minted credentials select the tenant. No additional external TCP port is permitted. Runtime ports and internal Unix sockets are independent of this map.

External application traffic uses TLS only. Plaintext RESP, AMQP, and database sessions are never externally offered. A schema protocol value alone does not prove TLS capability: for example, a MongoDB service without server TLS remains internal-only. There is no external plaintext Redis listener or per-tenant port.

Routing mechanisms#

Client-first TLS through Envoy#

Envoy inspects the ClientHello on 443 and 5671, chooses an SNI filter chain, checks the source allowlist, and forwards the unmodified TLS stream to the selected Supervisor endpoint relay. xDS supplies endpoint and grant state. Missing or unknown SNI and unauthorized sources fail closed before wake. Use the assigned service hostname rather than an IP address; verify its certificate on the client.

PostgreSQL SSLRequest-aware negotiation#

PostgreSQL normally sends an eight-byte SSLRequest before a TLS ClientHello. The Envoy C module consumes that negotiation before SNI selection, then sends an independent SSLRequest on the backend leg; a generic TLS inspector alone is insufficient. The SSLRequest is negotiation, not permission to skip authentication. Database credentials are always required; TLS exists only when the database itself has ssl enabled, because the edge relays the stream byte for byte and never terminates it. A client without SNI cannot select a tenant on the shared listener.

MySQL gateway#

MySQL sends its server greeting before the client sends a TLS ClientHello. The gateway on 3306 therefore terminates client TLS, verifies platform-minted credentials, and selects the endpoint by the unique routing username. It is a peer of Envoy, not an Envoy listener. The private backend is plaintext by design; protocol: mysql rejects the platform backend tls: {} marker.

MySQL/MariaDB/WordPress template defaults render u_{{ site.token }} using the persisted site ID. Connect with that resolved username from the owner connection panel, not a shared app username or the literal placeholder. Credential generation, gateway snapshot consumption, and revocation enforcement are implemented and tested. Public listener activation still requires deployment verification. Custom usernames require a valid unique gateway binding before external access can be enabled.

Grants and allowlists#

The current owner API accepts POST /sites/{slug}/tcp-access with a non-public manifest service name and a duration of 1–1440 minutes. It derives the source IP from the request and returns hostname, port, connection, cidr, expires_at, operation_id, and grant_id, plus edge_state, protocol_state, protocol, and nullable routing_username. Connect only from the reported CIDR. A grant is network authorization; it does not replace database authentication or TLS verification.

Source responses use mysql://<routing_username>@<hostname>:3306 for MySQL and tcp://<hostname>:<port> for other protocols. They report configured_unverified / unverified; allocation or configuration is not a successful protocol probe. The source policy selects services.<name>.protocol, uses the fixed port map above, and retains the legacy private database@5432 inference for compatibility. tcp, redis, mongodb, and amqp require the tls: {} marker for an external grant. Runtime TLS capability must still be verified. Service names must also form a safe endpoint identity for provisioning.

Omitting protocol is intended to leave a new private service internal-only. Current validation normalizes omission to tcp; the policy can admit that normalized service if it has tls: {}. Omission alone therefore does not prove isolation. Declare the protocol explicitly and withhold grants for internal services. Strict omitted-protocol isolation remains a policy gap.

Envoy/gateway admission and Supervisor relay checks enforce grant CIDRs and expiry before wake. Revocation and credential rotation invalidate cached authorization. The relay bounds established connections by the grant deadline. HTTP public / authenticated access is a separate policy.

Private control-plane interfaces — implemented in source#

These routes are outside /v/api/v1, use dedicated internal bearer credentials, and are not customer or agent PAT interfaces:

Edge state/authz/drain require SITE_EDGE_API_TOKEN; credential sync requires SITE_MYSQL_GATEWAY_API_TOKEN. Empty or incorrect tokens fail closed. Snapshots include replace: true, a content revision, cache_ttl_seconds, and valid_until; consumers must evict absent entries and expired snapshots. The configured TTL is capped at five seconds and bounded by grant expiry. Rotation, expiry, drain, and purge affect the next snapshot. The gateway evicts stale/failed snapshots and revokes sessions. Envoy removes draining resources; controller ACK and connection grace-period checks must precede Supervisor quiescence in the lifecycle coordinator.

Connect with protocol clients#

These examples apply only after the relevant edge stage and endpoint have been verified. Set TCP_HOST to the returned service hostname and CA_FILE to a trusted CA bundle. Use the owner-revealed credentials; never send a control-plane PAT to an application endpoint. Keep passwords out of URLs and shell history.

psql "host=$TCP_HOST port=5432 dbname=app user=app"
# With ssl enabled on the database:
psql "host=$TCP_HOST port=5432 dbname=app user=app sslmode=verify-full sslrootcert=$CA_FILE"
mysql --host="$TCP_HOST" --port=3306 --user="$ROUTING_USERNAME" \
  --ssl-mode=VERIFY_IDENTITY --ssl-ca="$CA_FILE" --password app
redis-cli --tls --sni "$TCP_HOST" --cacert "$CA_FILE" \
  -h "$TCP_HOST" -p 443 --user app --askpass PING

MariaDB clients must use their equivalent CA and server-certificate verification options. A Redis/Valkey server must enable its TLS port, disable plaintext (--port 0), load its service certificate/key, and authenticate the app ACL user. The seeded ACL disables default and removes @dangerous commands.

Reconnect and awake time#

Client connections do not survive instance recycle. Checkpoint/restore, snapshots, restart, node moves, and idle scale-to-zero require clients to reconnect. Configured durable state persists, but a socket and an in-flight transaction are not durable state. Use connection-pool retries with bounded backoff and resolve ambiguous write outcomes before retrying.

Client keepalives hold an instance awake; that awake time is billable. Tune pool lifetime, idle connections, and keepalives to the workload's cost and latency needs. Supervisor relays own connection idle timeouts. Edge active health checks against wake-gated runtimes are forbidden because they would continually wake the workload.

Implementation and metering status#

The Supervisor is the authoritative billing meter. Edge counters are for reconciliation only and must not independently charge usage. Relay source emits network_bytes with received/sent byte counts. Django ingestion records their sum as bytes with directional counters and duration in ledger metadata, once per node/stream/sequence. Its metered event set includes wake_minutes, requests, storage_gb_hours, and network_bytes. Ingestion is a recorded fact, not proof of deployment, complete coverage, or a new pricing rule. The public site usage response includes network_bytes as an integer.

This page supersedes the historical dedicated-port and TCP pool designs.