PharosVPN — Platform Design

Status: draft v2 · 2026-05-17

This is the single source of truth for the PharosVPN platform. Every subproject README and BUILD.md defers to this document. When code and this document disagree, the document is wrong — fix it in the same PR.

0. What PharosVPN is

A self-hostable, open-source, dual-protocol VPN fleet platform. One codebase serves two postures from the same binaries:

• Personal — “I want my own VPN,” one operator, a handful of nodes.
• Enterprise — a team managing many users across many regions.

Defaults differ; the engine is identical.

The data plane is AmneziaWG (obfuscated WireGuard) and XRay (VLESS + REALITY), both terminating end-user tunnels on UDP/TCP 443. The platform is the control plane, account system, and clients around that data plane.

1. Goals

• Self-hostable in under 30 minutes. Clone, follow the README, get a working fleet. No lock-in beyond the chosen cloud provider.
• Defense-in-depth control plane. The controller issues credentials and rotates server config — the highest-value target. It assumes it will be attacked: no inbound ports, no public DNS, no public IP.
• Dumb nodes. A compromised VPN node must not yield control of the fleet. Nodes act only on cryptographically validated instructions.
• Survive a controller outage. The controller is always-on and continuously reconciles the fleet, but it is a control-plane, not a data-plane, dependency: if it is briefly unavailable, every node keeps serving existing tunnels indefinitely. The same applies to clients: a client connects from cached profiles when the account service is unreachable.
• The controller never holds usable user secrets. User profiles are end-to-end encrypted; a controller compromise yields ciphertext, not profiles.

2. The three node roles + clients

                  ┌──────────────────────────────────┐
                  │  coxswain  — CONTROLLER            │
                  │  (private network, behind NAT)    │
                  │  vpn-mgr daemon + admin Web UI     │
                  │  + SQLite state + CA + embedded    │
                  │    relay (toggleable)              │
                  └───────┬──────────────────┬─────────┘
        mTLS, coxswain-initiated outbound    │ reverse tunnel
        gRPC/HTTP2 to each node              │ (coxswain dials OUT to a
                  │                          │  remote relay)
        ┌─────────┼─────────┐                ▼
        ▼         ▼         ▼          ┌──────────────┐
   ┌────────┐┌────────┐┌────────┐      │    RELAY     │  (public)
   │  node  ││  node  ││  node  │ ...  │ mTLS ingress │
   │ NODE A ││ NODE B ││ NODE N │      │ for clients  │
   │ public ││ public ││ public │      └──────┬───────┘
   │ AWG udp ││  ...   ││  ...   │             │ mTLS
   │ XRay tcp││        ││        │             ▼
   └────┬───┘└────┬───┘└────┬───┘      ┌──────────────┐
        │ udp/tcp 443       │          │   caravel    │
        ▼        end users  ▼          │ mobile client│
     end-user tunnels                  └──────────────┘

Key inversion: the controller dials out to everything. Nodes are already public (they must be, to terminate tunnels), so coxswain initiates outbound mTLS to each node. coxswain also dials out to a remote relay (reverse tunnel), so the controller needs zero inbound ports anywhere.

relay embedded vs remote. A relay always runs in-process inside coxswain (toggleable off in the admin UI). When the controller sits behind NAT and must serve clients, deploy a remote relay on a public host (its own VM, or co-located on a node); coxswain dials out to it over a persistent reverse tunnel. Embedded and remote are transport differences only — identical trust.

3. Component responsibilities

coxswain (controller)

• Source of truth. SQLite holding fleet inventory, profiles, users, devices, peers, admins, sessions, the CA, audit log, metrics samples. See §10.
• Admin Web UI — SvelteKit SPA embedded in the binary, served on localhost.
• Outbound control loop — holds a long-lived mTLS/gRPC connection to each node; pushes config, pushes/revokes peers, and receives a live event stream (see §7).
• Node onboarding over SSH — installs and updates the node agent on operator-provided VMs over SSH; all node control is gRPC. See §5. coxswain does not call cloud-provider APIs — the operator creates the VM.
• Issues node certs, the controller’s own client cert, relay certs, and per-user/device certs. Holds the CA. See §4.
• Embedded relay and the reverse-tunnel dialer for remote relays.
• Account & sync service — authenticates users/admins, serves E2E-encrypted profile bundles (see §8). This surface is reached only via a relay.

node (VPN node agent)

• Stateless except for what coxswain gave it. All config is written to disk only after coxswain pushes it over mTLS.
• Data plane: awg-quick@awg0 on UDP 443, xray.service on TCP 443.
• Control port (mTLS-only, gRPC). Operations: status, metrics, push config, add/remove peer (live, no restart), handshake stats, restart service, and a server-stream of live events back to coxswain.
• SSH is install-only. coxswain reaches a node over SSH solely to install and update the node agent; every operational instruction is gRPC.
• Cold-start resilient. Comes up from disk every boot using the last config coxswain pushed. Controller offline ⇒ existing peers keep working.

Node network policy

Each node carries a network policy the operator sets per node from the admin UI — three independent toggles coxswain pushes over the control channel:

Masquerade and isolation require forwarding. The toggles translate to a canonical PostUp / PostDown rule set — coxswain generates it, shows it read-only in an advanced UI panel, and pushes the policy; node applies the same set (egress interface autodetected):

forwarding   iptables -A FORWARD -i %i -j ACCEPT;  -A FORWARD -o %i -j ACCEPT
             sysctl net.ipv4/ipv6 ... forwarding = 1
masquerade   iptables -t nat -A POSTROUTING -o <egress> -j MASQUERADE
isolation    iptables -I FORWARD 1 -i %i -o %i -j DROP

PostDown removes each with the matching -D. The rule set is the contract: coxswain’s preview and node’s application must not drift.

Endpoint diversity & rotation

A node does not expose a single address. It accepts AmneziaWG on a set of public IPs and a UDP port range, yielding a large pool of reachable (ip, port) endpoints from a small config (node binds the IPs and DNATs the port range onto the WireGuard listener). A profile carries that pool plus a rotation policy — { enabled, interval, jitter }. The client picks a random endpoint and, when rotation is enabled, re-picks every interval ± jitter, updating the WireGuard peer endpoint live (WireGuard peers roam by design, so this costs nothing).

The purpose is anti-correlation. When every member of an organisation always reaches the same ip:port, a network observer clusters them by endpoint — same endpoint, same affiliation. A random, rotating endpoint per client removes that stable signal without touching the tunnel crypto. Rotation defaults off for --personal (a lone operator gains nothing from it) and on for --enterprise; operator-overridable either way.

Endpoints are always represented as an array, even when a node has a single ip:port — there is no singular-endpoint form.

relay

• Stateless public proxy. Terminates client mTLS, forwards gRPC streams to coxswain. Holds no database; all lookups delegated to coxswain.
• Strips spoofable client metadata; injects exactly one trusted value: the verified device fingerprint.
• Two transports to coxswain: embedded (in-process, in-memory pipe) or remote reverse tunnel (coxswain dials out, multiplexed substreams).
• Carries only ciphertext profile bundles — see §8 — so a compromised remote relay host cannot read user profiles.

caravel (mobile client)

• Two decoupled layers: a VPN engine (establishes tunnels, multi-node / multi-protocol) and a set of pluggable profile sources (see §8).
• Posture-aware: personal (account login, QR, file import; admin section if the logged-in account is an admin) vs managed (MDM config present — account login and admin hidden, profiles locked). One app, one store listing.

4. Trust model & PKI

A single in-repo root CA, generated on coxswain’s first run, stored in coxswain’s SQLite, never copied off the controller. Two intermediates under it:

• Fleet CA — issues node node certs, the controller’s client cert, and relay relay certs.
• Device CA — issues per-user/per-device leaf certs for caravel and the admin browser.

Compromise containment:

• Compromised node → attacker gets that node’s key + the CA cert (not key). Cannot impersonate coxswain or other nodes. Operator revokes the node cert.
• Compromised remote relay → attacker can see traffic metadata but profile bundles are E2E-encrypted ciphertext (§8). Cannot mint certs.
• Compromised coxswain → attacker gets the CA key. Fleet fully compromised — but user profiles remain encrypted (the controller never holds users’ private keys in usable form, §8). Hence coxswain’s “no inbound ports, behind NAT” posture.

Post-quantum hardening. Every AmneziaWG peer is issued a unique 256-bit preshared key, mixed into the WireGuard handshake. WireGuard’s ECDH (Curve25519) is quantum-vulnerable; the symmetric PSK is not — so recorded tunnel traffic stays confidential against a future harvest-now-decrypt-later attacker. coxswain generates the PSK per peer, ships it inside the E2E profile bundle (§8), and pushes it to node. This is a pragmatic interim measure, not a full post-quantum handshake.

5. Bootstrap & enrollment

Node enrollment — cox nodes add <ssh-host>:

1. The operator creates a VM on any provider and adds coxswain’s SSH public key (printed by cox ssh-key) to its authorized_keys. coxswain has its own SSH keypair, generated on first run and stored in SQLite.
2. coxswain connects out over SSH, pins the host key on first use (TOFU), and installs the node agent — either by uploading a bundled binary or running a one-line download.
3. node generates its own keypair on the node and emits a CSR. coxswain pulls the CSR back over SSH, signs it with the Fleet CA, and pushes the certificate plus the CA back. The node’s private key never leaves the node and coxswain never holds it.
4. coxswain starts the node service. From here every instruction is gRPC.

SSH is a deployment channel only — install and update of the agent. There is no enrollment-mode listener and no one-time bootstrap token; the trusted SSH channel replaces both.

Relay enrollment — same SSH pattern for a remote relay.

User / device enrollment — a user is given an enrollment ticket (QR or deep link, see §9). caravel scans it, contacts relay→coxswain, the device generates a keypair, coxswain issues a Device-CA leaf, and the device is bound to the user account. The enrollment ticket is the only moment of weakness: short TTL, one-use, scoped.

6. Wire protocol

gRPC over mTLS (HTTP/2, TLS 1.3). Decided — not plain JSON. The live event streaming in §7 needs server-streaming, which is native to gRPC and awkward otherwise. Both Go ends make codegen cheap; caravel consumes generated clients too.

• Protobuf schemas are the contract. They live in docs/proto/ (may graduate to a dedicated proto repo). No subproject hand-rolls message types.
• Schemas are versioned; unknown fields are ignored, never rejected.

7. Real-time & multi-admin

The admin UI must feel live — a client connecting to a node appears immediately, not on a 30-second poll.

• node → coxswain: coxswain holds its outbound mTLS connection open and the node streams events (handshake up/down, peer connect/disconnect, errors) over a gRPC server-stream. Polling remains only as a fallback heartbeat.
• coxswain → browser: every open admin page holds a WebSocket. coxswain pushes state changes to all of them — open the dashboard on three machines, all three update together.

Optimistic concurrency. Every mutable record carries a version integer. A mutation must send the version the admin loaded. If coxswain’s current version is higher, it rejects with HTTP 409 Conflict — “changed by someone else, reload.” Admin A editing a stale copy of a user is refused because Admin B already bumped it. Live WebSocket replication makes conflicts rare (A’s screen usually updates before A saves); the version check is the hard safety net.

8. Accounts, profiles & sync

Accounts and roles

users are authentication principals (unlike the old codebase, where only admins logged in). On login the role decides the surface: user → own profiles only; admin → also the admin console (web = full; caravel = a small glance-and-quick-actions subset).

Profile sources (the unified-client model)

caravel has a VPN engine that only ever reads a local profile store. Profiles enter that store from interchangeable sources — “synced vs unsynced” is just which sources are enabled, not two apps:

The account/sync service + relay are an optional platform component: an enterprise doing only MDM/QR runs no relay and no account service.

End-to-end profile encryption

Each user has a long-lived keypair. coxswain holds only the public key. coxswain generates a profile, encrypts it to the user’s public key, stores ciphertext, discards plaintext. Only the user’s devices decrypt. Hybrid envelope:

• Random data key → profile encrypted with XChaCha20-Poly1305 (AEAD).
• Data key wrapped to the user’s public key.
• Bundle signed.

Private-key storage — DECIDED: passphrase-wrapped blob held by coxswain. The user’s private key is encrypted with a key derived (Argon2id) from the user’s passphrase; coxswain stores only that opaque blob and never the passphrase or a usable private key. Any new device unwraps it with the passphrase. This gives seamless multi-device + recovery; a coxswain compromise yields only a passphrase-encrypted blob. (Device-to-device-only transfer was the alternative — stronger, but no recovery and clunkier enrollment; rejected for v1.)

9. The .pharos file format

One extension: .pharos. Encryption is a property inside the file, not a separate extension (no .spharos). The app reads the header and auto-routes.

A .pharos file is a JSON container with an always-readable header:

{
  "fmt": "pharos-profile",
  "v": 1,
  "enc": "none" | "password" | "account",
  // password mode: "kdf": {argon2id params + salt}, "cipher", "nonce"
  // account mode:  "recipient", "wrapped_key", "cipher", "nonce"
  "payload": <profile object>  |  "<base64 AEAD ciphertext>"
}

The header is fed as AAD so enc/v/KDF params are authenticated (no downgrade). Content-sniff on fmt so renamed files still import. Register MIME application/vnd.pharosvpn.profile, an iOS UTI, an Android intent filter.

Future-proof protocol model. The profile carries nodes, each with a versioned, tagged list of protocols:

{ "nodes": [ { "id": "...", "endpoints": ["..."], "protocols": [
    { "type": "amneziawg",    "v": 2, "params": { } },
    { "type": "xray-reality", "v": 1, "params": { } }
] } ] }

Clients keep a registry of handlers keyed by type. Ignore-unknown, never reject: an old client skips a protocol/node it can’t handle. Adding a protocol = a new type string + a handler, zero format break. Plus metadata: user, device, fleet_id, issued_at, expires_at, revision.

QR codes

Compression does not solve QR size — profiles are mostly incompressible key material. A reliably-scannable QR holds ~150–300 bytes; a multi-node profile is kilobytes. Two QR kinds:

• Enrollment ticket (default): relay endpoint + one-time claim token + CA fingerprint to pin. ~150 bytes, fixed size regardless of profile size. The app scans it and downloads the full .pharos over the network.
• Self-contained .pharos QR (offline/air-gapped): single-node fits one QR via CBOR/binary → deflate → base45 → alphanumeric QR. Multi-node offline ⇒ multi-part / structured-append / animated QR (“scan 1 of N”).

A separate Amnezia-compatible .vpn export remains for users on the Amnezia client. .pharos is our format and is not asked to do that job.

10. Persistence

Single SQLite database on coxswain (state/app.db), Goose migrations.

Tables: ca (the root + intermediate CAs, §4), nodes, profiles, users, devices, peers, admins, sessions, node_certs, device_certs, bootstrap_tokens, audit_log, metrics_samples, relays. Every mutable row carries version INTEGER and updated_at for §7 optimistic concurrency. YAML projections under state/snapshots/ continue for git-friendly diffs. node and relay have no database.

11. Failure modes

12. Defaults: personal vs enterprise

Same binaries, two presets at cox init:

13. License & contribution

• Apache-2.0 for the entire platform, every repo. Rationale: the goal is the widest possible adoption and contribution. AGPL’s network copyleft was considered — it would force service operators to publish their changes — but in practice it repels the commercial users whose contributions we want (many organisations ban AGPL outright), shrinking both adoption and the contributor pool. A permissive licence maximises reach, and Apache-2.0’s explicit patent grant suits a cryptography/networking project, giving adopters and contributors clear patent terms.
• DCO (Developer Certificate of Origin, Signed-off-by) for contributions. No CLA — a permissive licence needs neither relicensing nor dual-licensing.
• Every source file carries a short SPDX-License-Identifier: Apache-2.0 header. Each repo ships LICENSE (full Apache-2.0 text) and CONTRIBUTING.md (DCO instructions).

14. Repo map

15. Decisions log

Still open

• caravel: native per-platform vs Kotlin Multiplatform — decide in caravel/BUILD.md.
• Whether protobuf contracts graduate from docs/proto/ to a dedicated repo.
• Metrics export: built-in dashboard only, or also Prometheus push.