Compare commits

..
10 Commits
Author SHA1 Message Date
dckcandClaude Opus 5.5 e5ed407e55 capsite/toybox: a mount outlives its handler: the vat restarts on demand
A handler is a live reference, held by the VatAdmin DO; workerd evicts
an idle DO within seconds, and with it the handler, so a vat's page went
503 almost at once.

createVat now keeps a vat's params when they hold only data and
ingresses (an ingress by its key: the VatAdmin can make it again), and
which vat was given which ingress. A request for a mount with no handler
restarts that vat with its params; it rebuilds its root, serves again,
and the request is answered. A vat whose params hold another peer's
objects isn't restartable yet.

The test crashes its own toybox (a crash ends every session to it), and
the creator waits for the root's handoff before it disconnects, as
scripts/hello.js now does too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:44:29 -05:00
dckcandClaude Opus 5.5 a984461cee capsite/toybox: ingress is a capability; a hello-world web vat
E(vatAdmin).makeIngress(labels) reserves a mount and returns an ingress;
whoever holds it serves requests carrying those labels by calling
E(ingress).serve(handler). A vat gets one in its params, like any other
remote object; nothing in a vat's module is an entrypoint. A request
reaches the handler as passable data (method, path, labels, body), and
the response comes back as data (status, headers, body). The handler is
a live reference for now: after a restart its mount answers 503.

vats/hello.js serves hello, world; pnpm hello creates it on a running
toybox (client.js is the Node OCapN client, shared with the tests).

pnpm start and pnpm switchboard never ran: systemd-socket-activate wants
absolute paths for unix sockets.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:44:29 -05:00
dckcandClaude Opus 5.5 8025ec4b20 capsite/toybox: TODO: the counter vat as a durable Exo
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:44:29 -05:00
dckcandClaude Opus 5.5 f64dac4434 capsite/toybox: an OCapN counter vat; no import GC in workerd peers
vats/counter.js keeps its count in the vat's storage; the test checks
that it counts and that two counter vats count apart.

Running it after the greeter test hung: a FinalizationRegistry callback
in the vatAdmin, collecting an import, sent op:gc over the client's
WebSocket, and workerd forbids I/O outside a request. The vatAdmin's and
the vats' OCapN clients now run with enableImportCollection: false, so
imports go when their session does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:44:29 -05:00
dckcandClaude Opus 5.5 193a126d1c capsite/toybox: routing carries no secrets
The ingress no longer reads labels from a Toybox-Labels header. A header
was no place for the webkey pattern: browsers can't set one on a
WebSocket, and the pattern puts the secret in-band anyway. A page keeps
it in its fragment, reaches the vat by public labels, and sends it in
the body of a call, which the vat checks like a swissnum.

A request still reaches a vat only by naming every label of its mount.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:44:29 -05:00
dckcandClaude Opus 5.5 28b1c3c9d3 capsite/toybox: no encoder object in module scope
A module-level TextEncoder is a mutable object every importer shares.
Keep only its encode method, bound and hardened; the encoder itself is
unreachable. The code hash uses it too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:44:29 -05:00
dckcandClaude Opus 5.5 7fcd251261 capsite/toybox: mount lookups in SQL, not a scan in JS
Each request parsed every mount's labels and searched them in JS. Labels
now sit one per row in mount_label, indexed on (key, value): a lookup
joins the request's labels (json_each) on the index and keeps the vat
whose every label matched. The overlap check at registration is SQL too.

mount.labels keeps the whole set as JSON; mounts stored before
mount_label get their rows filled in when the schema runs. The SQL lives
in mounts-sql.js, tested on node:sqlite, old-shape state included.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:44:29 -05:00
dckcandClaude Opus 5.5 9356f63c64 capsite/toybox: OCapN vats; createVat takes remote objects
The admin socket now also speaks OCapN over Cap'n Web: a WebSocket there
reaches the VatAdmin DO, an OCapN peer whose bootstrap (swissnum
vatAdmin) offers createVat(src, params). params may hold remote objects;
the vat's root comes back as one.

Each OCapN vat is its own peer: SES and @endo/ocapn run in its facet
(vat-runtime.js, bundled with esbuild), around the vat's own module,
which exports buildRootObject(vatPowers, params). Passing a client's
object to a vat, or a vat's root to the client, is an OCapN handoff, so
the receiver dials the exporter. Nothing here can dial out, so the
VatAdmin is also the router: a client registers its gateway under a
designator, a vat's designator is its id, and every peer reaches every
other through it. The netlayer is ertp-clerk's, made to work over Workers
RPC stubs and plain promises as well as Cap'n Web.

Not yet: sturdyrefs that survive a restart, an adminNode, and grants
(egress, mount) for OCapN vats. App vats (export class App) still work
beside them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:44:29 -05:00
dckcandClaude Opus 5.5 8d8900d84d capsite/toybox: vats have ids, not names; ingress by mount
createVat (POST / on the admin socket) makes a vat with newUniqueId and
returns its id, which addresses it only there. A vat is reachable only
where it is mounted: ?mount=<labels> on each start, as ?egress= is
granted; absent means none.

The ingress reads a request's labels from its query string, plus a
Toybox-Labels header for secret ones (a URL ends up in logs and
Referers). It routes to the vat whose mount labels the request carries,
every one: stricter than Flower's no-conflict match, since an ingress
client holds no constrained handle and a label it omits must not match.
Registration keeps Flower's rule, so a mount must conflict with every
other and no request can carry two. The app keeps the whole path and
sees its connection labels in Toybox-Connection-Labels.

A vat from before ids lives at idFromName(name). PUT /by-name/<name>
reaches it on the admin socket; it stays unreachable from outside until
mounted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:44:29 -05:00
dckcandClaude Opus 5.5 f0557d29d6 capsite/toybox: egress through Flower-style switchboard handles
The switchboard now serves a Cap'n Web Handle on each of its sockets:
public (no loopback or private addresses) and local. A socket's labels
are fixed in config, so holding the socket's fd is holding that handle;
constrain(labels) returns a narrower one, as Flower's Constrain returns
a narrower fd. Labels only accumulate: a missing key matches anything,
and a key both sides set must agree (labels.js, shared by both).

The switchboard enforces the labels; toybox's Egress no longer checks
URLs itself. It opens a session to its one switchboard socket, constrains
the handle to the vat's grant, and forwards through it; a WebSocket
crosses the session as a pair of message streams. A vat narrows its own
grant with env.egress.constrain().

Grants are now labels as JSON. One stored before, a bare host[:port],
reads as {host, port}.

pnpm start and the systemd units give toybox the public handle; the
switchboard unit binds the local one in its own 0700 runtime directory.
The tests give their toybox the local handle to reach an echo server,
and check that a toybox holding only the public one never reaches it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:44:29 -05:00
28 changed files with 1972 additions and 196 deletions

No files matched your search

+2 -1
View File
@@ -1,3 +1,4 @@
state/
admin.sock
switchboard.sock
switchboard-*.sock
dist/
+39 -17
View File
@@ -1,45 +1,67 @@
# toybox
Vats whose code is installed at runtime: each vat is a Durable Object whose app runs as a [facet](https://developers.cloudflare.com/dynamic-workers/usage/durable-object-facets/) on code loaded through workerd's Worker Loader. Think `vatAdmin.createVat(bundle)`, with a supervisor per vat.
Vats whose code is installed at runtime: each vat is a Durable Object whose app runs as a [facet](https://developers.cloudflare.com/dynamic-workers/usage/durable-object-facets/) on code loaded through workerd's Worker Loader. Think `vatAdmin.createVat(bundle)`, with a supervisor per vat. Vats have no names: creating one returns its id, which addresses it only on the admin socket. Everything else a vat can do or be reached by is granted explicitly: egress by labels it may fetch within, ingress by labels it is mounted at.
- [supervisor.js](supervisor.js): `Vat` stores code by SHA-256 and runs the app on it; `Public` and `Admin` are the two entrypoints; `Egress` is a grant of one host.
- [switchboard.js](switchboard.js): a separate workerd process, the vats' only way out: a forward proxy for HTTP and WebSockets.
- [supervisor.js](supervisor.js): `Vat` stores code by SHA-256 and runs the app on it; `Directory` holds the mounts; `Public` (the ingress) and `Admin` are the two entrypoints; `Egress` is an egress grant.
- [switchboard.js](switchboard.js): a separate workerd process, the vats' only way out, after [Flower](https://github.com/NuxiNL/flower)'s switchboard: each of its sockets serves a Cap'n Web `Handle` with fixed labels, and `constrain(labels)` returns a narrower one.
- [labels.js](labels.js): Flower's label rule, shared by both: labels only accumulate, a missing key matches anything, and a key both sides set must agree. Ingress lookup is stricter (below).
- [config.capnp](config.capnp): the sockets have no addresses; whoever starts workerd hands them in, so holding a socket is the authority it grants.
| socket | reaches | what its holder can do |
|---|---|---|
| `http` | `Public` | `GET /<vat>/…`: talk to a vat's app |
| `admin` | `Admin` | `PUT /<vat>` with an ES module body: install it and (re)start the app; its storage carries over. `?egress=<host>` grants the app that one host. |
| `http` | `Public` | reach the vat mounted at labels the request carries |
| `admin` | `Admin` | `POST /` with an ES module body: create a vat and start it; the reply is `{"vat": <id>, "hash": <code hash>}`. `PUT /<id>` with a body: upgrade it; its storage carries over. `PUT /by-name/<name>`: the same, for a vat from before ids. |
An app must export `class App extends DurableObject` (see [apps/](apps/)). Its global `fetch` always throws: nothing in a vat is ambient. A granted host arrives as `env.egress`, an `Egress` with the host in `ctx.props`: its `fetch` forwards matching requests, WebSocket upgrades included, to the switchboard and refuses the rest with a 403. It is an object, so the app can pass it to whichever of its parts should have it.
Every start states the vat's grants, as JSON labels in query parameters; absent means none:
toybox reaches the switchboard over a unix socket (`-e switchboard=unix:…`), in HTTP proxy style so the full URL, scheme included, survives the hop.
- `?egress=<labels>`: what the app may fetch (below).
- `?mount=<labels>`: where the ingress finds it. A mount must conflict with every other one (some shared key, different values), so no request can carry two mounts' labels; an overlapping mount is a 409.
## Ingress
A request's labels are its query parameters; a key given twice is a 400. Routing carries no secrets: a URL ends up in logs and `Referer`s. A secret travels in-band, as on a webkey page: the page keeps it in its fragment, reaches the vat by public labels, and sends it in the body of a call, which the vat checks like a swissnum.
The request reaches the vat whose mount labels it carries, every one with the same value. That is stricter than Flower's match (no conflicting key): an ingress client holds no constrained handle that supplies labels for it, so it reaches a vat only by naming every label of its mount. The path is the app's own. The app sees the mount's and the request's labels together in `Toybox-Connection-Labels` (JSON), as a Flower server gets its connection labels. Responses carry `Referrer-Policy: no-referrer`.
## Egress
An app must export `class App extends DurableObject` (see [apps/](apps/)). Its global `fetch` always throws: nothing in a vat is ambient. A grant arrives as `env.egress`, an `Egress` with the labels in `ctx.props`. Its `fetch`, WebSocket upgrades included, goes through a switchboard handle constrained to those labels; the switchboard refuses the rest, which comes back as a 403. `env.egress.constrain(labels)` returns a narrower `Egress`. It is an object, so the app can pass it to whichever of its parts should have it.
| label | constrains |
|---|---|
| `scheme` | `http` or `https` |
| `host` | host name or address, no port |
| `port` | explicit, or the scheme's default |
| `network` | set by the switchboard socket: `public` or `local` |
The switchboard has two sockets, so two root handles: `public` (no loopback or private addresses) and `local`. toybox holds one, as an external service (`-e switchboard=unix:…`): `pnpm start` and the systemd units give it `public`; the tests give theirs `local` to reach an echo server.
## Run it
```sh
pnpm install
pnpm test # real workerds: install, upgrade, isolation, egress grants, fetch and WebSocket out, crash and restart
pnpm switchboard # systemd-socket-activate on ./switchboard.sock
pnpm test # real workerds: createVat, upgrade, mounts, egress grants and constrain, fetch and WebSocket out, crash and restart
pnpm switchboard # systemd-socket-activate on ./switchboard-public.sock and ./switchboard-local.sock
pnpm start # likewise on 127.0.0.1:8787 and ./admin.sock; state in ./state
```
Then, in a third terminal:
```sh
curl -X PUT --unix-socket admin.sock --data-binary @apps/counter-v1.js http://toybox/alice
curl localhost:8787/alice # v1 count 1
curl -X PUT --unix-socket admin.sock --data-binary @apps/counter-v2.js http://toybox/alice
curl localhost:8787/alice # v2 count 2
admin() { curl -sg --unix-socket admin.sock "$@"; }
alice=$(admin -X POST --data-binary @apps/counter-v1.js 'http://toybox/?mount={"vat":"alice"}' | jq -r .vat)
curl 'localhost:8787/?vat=alice' # v1 count 1
admin -X PUT --data-binary @apps/counter-v2.js "http://toybox/$alice?mount={\"vat\":\"alice\"}"
curl 'localhost:8787/?vat=alice' # v2 count 2
curl -X PUT --unix-socket admin.sock --data-binary @apps/egress-probe.js 'http://toybox/erin?egress=example.com'
curl 'localhost:8787/erin?url=https://example.com/' # got: <!doctype html>…
curl 'localhost:8787/erin?url=https://example.org/' # got: egress to example.org not granted
admin -X POST --data-binary @apps/egress-probe.js 'http://toybox/?mount={"vat":"erin"}&egress={"host":"example.com"}'
curl 'localhost:8787/?vat=erin&url=https://example.com/' # got: <!doctype html>…
curl 'localhost:8787/?vat=erin&url=https://example.org/' # got: egress refused: egress to example.org not granted
```
## Under systemd
[systemd/](systemd/) has socket and service pairs for toybox and its switchboard, assuming this directory is installed at `/opt/toybox`. toybox gets no network of its own (`PrivateNetwork=yes`): only the two sockets it is handed and the switchboard's socket, which only members of the `toybox` group can open. The switchboard has the network, fenced by `IPAddressDeny=`; `switchboard.capnp` allows `local` addresses only so the tests can reach a local echo server.
[systemd/](systemd/) has socket and service pairs for toybox and its switchboard, assuming this directory is installed at `/opt/toybox`. toybox gets no network of its own (`PrivateNetwork=yes`): only the two sockets it is handed and the switchboard's `public` socket, which only members of the `toybox` group can open. The switchboard binds its `local` socket in its own 0700 runtime directory, so no other unit holds that handle.
Neither unit has been run under the system manager yet.
+5 -3
View File
@@ -1,13 +1,15 @@
import { DurableObject } from "cloudflare:workers";
/** `?url=<target>` fetches it through `env.egress`; with `&ws=1`, opens a WebSocket there and reports the echo of "hello". */
/** `?url=<target>` fetches it through `env.egress` (narrowed by `&constrain=<labels as JSON>`, if given); with `&ws=1`, opens a WebSocket there and reports the echo of "hello". */
export class App extends DurableObject {
async fetch(request) {
const params = new URL(request.url).searchParams;
const target = params.get("url");
const { egress } = this.env;
if (!egress) return new Response("no egress granted\n");
if (!this.env.egress) return new Response("no egress granted\n");
try {
const egress = params.has("constrain")
? await this.env.egress.constrain(JSON.parse(params.get("constrain")))
: this.env.egress;
if (!params.has("ws")) return new Response(`got: ${await (await egress.fetch(target)).text()}\n`);
const resp = await egress.fetch(target, { headers: { Upgrade: "websocket" } });
const ws = resp.webSocket;
+9
View File
@@ -0,0 +1,9 @@
import { DurableObject } from "cloudflare:workers";
/** Reports its path and its connection labels. */
export class App extends DurableObject {
fetch(request) {
const { pathname } = new URL(request.url);
return new Response(`${pathname} ${request.headers.get("Toybox-Connection-Labels")}\n`);
}
}
+16
View File
@@ -0,0 +1,16 @@
/** @file Bundle the supervisor and the OCapN vat runtime for workerd (see config.capnp). */
import { build } from "esbuild";
const common = {
bundle: true,
format: "esm",
platform: "neutral",
mainFields: ["module", "main"],
conditions: ["workerd", "import"],
external: ["cloudflare:workers", "node:*"],
logLevel: "warning",
};
await build({ ...common, entryPoints: ["supervisor.js"], outfile: "dist/supervisor.js", external: [...common.external, "vat-runtime.txt"] });
// loaded as text, beside the vat's own module (vat.js)
await build({ ...common, entryPoints: ["vat-runtime.js"], outfile: "dist/vat-runtime.js", external: [...common.external, "./vat.js"] });
+33
View File
@@ -0,0 +1,33 @@
/**
* @file An OCapN client for the admin socket, from Node: it registers its
* own gateway with the router there and reaches every other peer through
* it. Run under SES (see test/lockdown.js).
*/
import { makeClient, swissnumFromBytes } from "@endo/ocapn";
import { newWebSocketRpcSession } from "capnweb";
import WebSocket from "ws";
import { locationOf, makeRpcNetLayer } from "./netlayer.js";
/**
* An OCapN client on the admin socket: registers its own gateway with the
* router there, and reaches every other peer through it.
*
* @param {string} adminSock
* @param {string} designator
* @param {{ verbose?: boolean }} [opts] OCapN logging
*/
export const connectClient = async (adminSock, designator, { verbose = false } = {}) => {
const ws = new WebSocket(`ws+unix:${adminSock}:/`);
await new Promise((resolve, reject) => (ws.once("open", resolve), ws.once("error", reject)));
const session = newWebSocketRpcSession(ws);
const ocapn = makeClient({ debugLabel: designator, verbose });
const netlayer = await ocapn.registerNetlayer(
makeRpcNetLayer({ designator, openGateway: (d) => session.gateway(d) }),
);
await session.register(designator, netlayer.gateway);
const vatAdmin = await ocapn.enlivenSturdyRef(
ocapn.makeSturdyRef(locationOf("vatAdmin"), swissnumFromBytes(new TextEncoder().encode("vatAdmin"))),
);
return { vatAdmin, close: () => (ocapn.shutdown(), ws.close()) };
};
+16 -6
View File
@@ -10,25 +10,35 @@ const config :Workerd.Config = (
services = [
(name = "main", worker = .supervisor),
(name = "nowhere", network = (allow = [])),
# the way out: see switchboard.capnp
(name = "switchboard", external = (address = "unix:switchboard.sock", http = (style = proxy))),
# the way out: one switchboard socket, i.e. one handle (see switchboard.capnp)
(name = "switchboard", external = (address = "unix:switchboard.sock", http = ())),
(name = "state", disk = (path = "state", writable = true)),
],
sockets = [
# reach a running vat by name
# the ingress: reach the vat mounted at a request's labels
(name = "http", http = (), service = (name = "main", entrypoint = "Public")),
# install code into a vat and (re)start it
# create vats, install code, grant egress and ingress
(name = "admin", http = (), service = (name = "main", entrypoint = "Admin")),
],
);
const supervisor :Workerd.Worker = (
compatibilityDate = "2026-10-01",
modules = [(name = "supervisor.js", esModule = embed "supervisor.js")],
durableObjectNamespaces = [(className = "Vat", uniqueKey = "toybox-vat", enableSql = true)],
# bundled by build.js
modules = [
(name = "supervisor.js", esModule = embed "dist/supervisor.js"),
(name = "vat-runtime.txt", text = embed "dist/vat-runtime.js"),
],
durableObjectNamespaces = [
(className = "Vat", uniqueKey = "toybox-vat", enableSql = true),
(className = "Directory", uniqueKey = "toybox-directory", enableSql = true),
(className = "VatAdmin", uniqueKey = "toybox-vat-admin", enableSql = true),
],
durableObjectStorage = (localDisk = "state"),
bindings = [
(name = "VATS", durableObjectNamespace = "Vat"),
(name = "DIRECTORY", durableObjectNamespace = "Directory"),
(name = "VAT_ADMIN", durableObjectNamespace = "VatAdmin"),
(name = "LOADER", workerLoader = ()),
(name = "SWITCHBOARD", service = "switchboard"),
],
+53
View File
@@ -0,0 +1,53 @@
/**
* @file Egress labels, after [Flower](https://github.com/NuxiNL/flower)'s
* switchboard: string key-value pairs that only accumulate. A missing key
* matches anything; a key present on both sides must agree.
*
* Keys that constrain a URL: `scheme` (`http`, `https`), `host` (name or
* address, no port), `port` (explicit or the scheme's default). The
* switchboard sets `network` per socket; it constrains no URL.
*/
const DEFAULT_PORT = { http: "80", https: "443" };
/**
* Union of two label sets; throws on a key they give different values,
* naming the key but not the values (they may end up in logs).
*
* @param {Record<string, string>} held
* @param {Record<string, string>} more
*/
export const mergeLabels = (held, more) => {
const merged = { ...held };
for (const [k, v] of Object.entries(more)) {
if (typeof v !== "string") throw new TypeError(`label ${k} is not a string`);
if (k in merged && merged[k] !== v) throw new Error(`label ${k} is already set to a different value`);
merged[k] = v;
}
return Object.freeze(merged);
};
/**
* The labels a URL carries.
*
* @param {string} url
*/
export const urlLabels = (url) => {
const u = new URL(url);
const scheme = u.protocol.replace(/:$/, "");
return { scheme, host: u.hostname, port: u.port || DEFAULT_PORT[scheme] || "" };
};
/**
* Throws unless `url` is within `labels`.
*
* @param {Record<string, string>} labels
* @param {string} url
*/
export const checkUrl = (labels, url) => {
const actual = urlLabels(url);
for (const [k, v] of Object.entries(actual)) {
if (k in labels && labels[k] !== v) throw new Error(`egress to ${actual.host} not granted`);
}
};
+53
View File
@@ -0,0 +1,53 @@
/**
* @file The ingress's mounts, in SQL: which vat serves which label set.
*
* Labels sit one per row, so a lookup can use an index; `mount.labels`
* keeps the whole set as JSON. Request and mount labels arrive as JSON
* objects, split into rows by `json_each`.
*/
export const MOUNT_SCHEMA = [
"CREATE TABLE IF NOT EXISTS mount (vat TEXT PRIMARY KEY, labels TEXT)",
"CREATE TABLE IF NOT EXISTS mount_label (vat TEXT, key TEXT, value TEXT, PRIMARY KEY (vat, key))",
"CREATE INDEX IF NOT EXISTS mount_label_kv ON mount_label (key, value)",
// mounts made before mount_label existed
`INSERT OR IGNORE INTO mount_label
SELECT m.vat, j.key, j.value FROM mount m, json_each(m.labels) j
WHERE NOT EXISTS (SELECT 1 FROM mount_label l WHERE l.vat = m.vat)`,
];
/**
* Another mount (than ?2's) whose labels conflict with none of ?1's.
* Flower's registration rule: every mount must conflict with every other.
*/
export const OVERLAP = `
SELECT 1 FROM mount m WHERE m.vat != ?2 AND NOT EXISTS (
SELECT 1 FROM mount_label l JOIN json_each(?1) r ON r.key = l.key
WHERE l.vat = m.vat AND l.value != r.value)
LIMIT 1`;
export const MOUNT = "INSERT OR REPLACE INTO mount VALUES (?1, ?2)";
export const MOUNT_LABELS = "INSERT INTO mount_label SELECT ?1, key, value FROM json_each(?2)";
export const UNMOUNT = "DELETE FROM mount WHERE vat = ?1";
export const UNMOUNT_LABELS = "DELETE FROM mount_label WHERE vat = ?1";
/**
* The mount whose every label ?1 (the request's labels) carries, with the
* same value: the labels that match, counted per vat, against how many
* the vat has. A mount with no labels matches any request.
*
* Stricter than Flower's match (no conflict): an ingress client holds no
* constrained handle that supplies labels for it, so a request reaches a
* vat only by naming every label of its mount.
*/
export const LOOKUP = `
SELECT m.vat, m.labels FROM mount m JOIN (
SELECT l.vat, count(*) AS hits
FROM json_each(?1) r JOIN mount_label l ON l.key = r.key AND l.value = r.value
GROUP BY l.vat
) h ON h.vat = m.vat
WHERE h.hits = (SELECT count(*) FROM mount_label WHERE vat = m.vat)
UNION ALL
SELECT m.vat, m.labels FROM mount m
WHERE NOT EXISTS (SELECT 1 FROM mount_label l WHERE l.vat = m.vat)
LIMIT 1`;
+133
View File
@@ -0,0 +1,133 @@
/**
* @file An OCapN netlayer whose transport is an RPC session: Cap'n Web, or
* Workers RPC, or one proxied through the other.
*
* Each OCapN connection is a pair of {@link ByteInbox}es, one per
* direction, exchanged through a gateway; each `deliver` call carries
* exactly one OCapN message, so no framing is needed. The RPC session
* provides ordering and only-the-peer-can-inject; OCapN does the rest.
*
* After vattr97's ertp-clerk (`ocapn-capnweb-netlayer.ts`), with the stub
* methods only Cap'n Web has (`onRpcBroken`) made optional.
*
* @see https://github.com/ocapn/ocapn/blob/main/draft-specifications/Netlayers.md
*/
import { RpcTarget } from "capnweb";
export const TRANSPORT = "capnweb";
/**
* A peer's location, by designator alone: a router finds the peer.
*
* @param {string} designator
*/
export const locationOf = (designator) => ({ type: "ocapn-peer", transport: TRANSPORT, designator, hints: false });
/** Same as `locationToLocationId` in `@endo/ocapn`, which its exports map doesn't expose. */
const locationId = ({ designator, transport }) => `ocapn://${designator}.${transport}`;
/** Receives one direction of an OCapN connection, one message per call. */
export class ByteInbox extends RpcTarget {
#onMessage;
constructor(onMessage) {
super();
this.#onMessage = onMessage;
}
/** @param {Uint8Array} bytes */
deliver(bytes) {
// OCapN's syrup reader needs offset 0; a pooled Buffer view may not have it.
this.#onMessage(bytes.byteOffset === 0 ? bytes : new Uint8Array(bytes));
}
}
const dispose = (stub) => stub?.[Symbol.dispose]?.();
/** Write to a remote inbox, discarding each result at once so the session retains none. */
const makeSocket = (inbox, close) => ({
write: (bytes) => dispose(inbox.deliver(bytes)),
end: close,
});
/**
* A remote inbox reached through a gateway that is only a promise (a
* local async call, not a pipelining stub): writes queue in order behind it.
*/
const deferredInbox = (gatewayP, myInbox) => {
const inboxP = Promise.resolve(gatewayP).then((gateway) => gateway.connect(myInbox));
return {
deliver: (bytes) => void inboxP.then((inbox) => dispose(inbox.deliver(bytes))),
onRpcBroken: (cb) => void inboxP.then((inbox) => inbox.onRpcBroken?.(cb), cb),
};
};
/**
* Make an OCapN netlayer for `client.registerNetlayer`.
*
* @param {object} opts
* @param {string} opts.designator this peer's designator
* @param {(designator: string) => any} opts.openGateway the gateway of
* the peer with this designator: a stub or pipelining promise
* (`openGateway(d).connect(...)`), or a plain promise for one
* @returns a netlayer maker; the netlayer has a `gateway` that accepts
* incoming connections
*/
export const makeRpcNetLayer = ({ designator, openGateway }) => {
const location = locationOf(designator);
const connections = new Set();
return (handlers, logger) => {
const track = (connection, inbox) => {
connections.add(connection);
inbox.onRpcBroken?.((reason) => {
if (connections.delete(connection)) logger.log("rpc netlayer: connection broken", reason);
handlers.handleConnectionClose(connection, reason);
});
};
const netlayer = {
location,
locationId: locationId(location),
connect: (peer) => {
const gateway = openGateway(peer.designator);
let connection;
const myInbox = new ByteInbox((bytes) => handlers.handleMessageData(connection, bytes));
// Pipelined: writes before the peer answers queue in order.
const peerInbox =
typeof gateway.connect === "function" ? gateway.connect(myInbox) : deferredInbox(gateway, myInbox);
connection = handlers.makeConnection(
netlayer,
true,
makeSocket(peerInbox, () => {
connections.delete(connection);
dispose(gateway);
}),
);
track(connection, peerInbox);
return connection;
},
shutdown: () => {
for (const connection of [...connections]) connection.end();
},
};
class Gateway extends RpcTarget {
connect(peerInbox) {
const peer = peerInbox.dup?.() ?? peerInbox;
const connection = handlers.makeConnection(
netlayer,
false,
makeSocket(peer, () => {
connections.delete(connection);
dispose(peer);
}),
);
track(connection, peer);
return new ByteInbox((bytes) => handlers.handleMessageData(connection, bytes));
}
}
return Object.assign(netlayer, { gateway: new Gateway() });
};
};
+12 -3
View File
@@ -4,12 +4,21 @@
"private": true,
"type": "module",
"scripts": {
"start": "mkdir -p state && rm -f admin.sock && systemd-socket-activate -l 127.0.0.1:8787 -l ./admin.sock -- node_modules/workerd/bin/workerd serve --experimental config.capnp -S http=3 -S admin=4 -d state=state -e switchboard=unix:./switchboard.sock",
"switchboard": "rm -f switchboard.sock && systemd-socket-activate -l ./switchboard.sock -- node_modules/workerd/bin/workerd serve --experimental switchboard.capnp -S http=3",
"test": "node --test"
"start": "node build.js && mkdir -p state && rm -f admin.sock && systemd-socket-activate -l 127.0.0.1:8787 -l \"$PWD/admin.sock\" -- node_modules/workerd/bin/workerd serve --experimental config.capnp -S http=3 -S admin=4 -d state=state -e \"switchboard=unix:$PWD/switchboard-public.sock\"",
"switchboard": "rm -f switchboard-public.sock switchboard-local.sock && systemd-socket-activate -l \"$PWD/switchboard-public.sock\" -l \"$PWD/switchboard-local.sock\" -- node_modules/workerd/bin/workerd serve --experimental switchboard.capnp -S public=3 -S local=4",
"test": "node build.js && node --import ./test/lockdown.js --test test/*.test.js",
"build": "node build.js",
"hello": "node --import ./test/lockdown.js scripts/hello.js"
},
"devDependencies": {
"esbuild": "^0.28.2",
"workerd": "1.20261006.1",
"ws": "8.22.0"
},
"dependencies": {
"@endo/far": "^1.1.14",
"@endo/init": "^1.1.13",
"@endo/ocapn": "1.1.1",
"capnweb": "0.12.0"
}
}
+486
View File
@@ -7,7 +7,23 @@ settings:
importers:
.:
dependencies:
'@endo/far':
specifier: ^1.1.14
version: 1.1.14
'@endo/init':
specifier: ^1.1.13
version: 1.1.13
'@endo/ocapn':
specifier: 1.1.1
version: 1.1.1
capnweb:
specifier: 0.12.0
version: 0.12.0
devDependencies:
esbuild:
specifier: ^0.28.2
version: 0.28.2
workerd:
specifier: 1.20261006.1
version: 1.20261006.1
@@ -47,6 +63,242 @@ packages:
cpu: [x64]
os: [win32]
'@endo/base64@1.1.1':
resolution: {integrity: sha512-6Aiy5oU/5/LYrHLbEPz0OWQ+CQsnAOC3qQYQ3nlwWi2JsU8fIZWjwfa+/+MSx8LTzITC250ZnMvtR0y/FIOQKw==}
'@endo/bytes@1.0.2':
resolution: {integrity: sha512-gKG8v4c7SD/QzULlFl1OKBuUCCy+9H78xsCL40hZn1q2/hVCNAHrxPJFO6VbA8Nk8UJP6P+NjvZY+xdrfwytMQ==}
'@endo/cache-map@1.1.0':
resolution: {integrity: sha512-owFGshs/97PDw9oguZqU/px8Lv1d0KjAUtDUiPwKHNXRVUE/jyettEbRoTbNJR1OaI8biMn6bHr9kVJsOh6dXw==}
'@endo/common@1.4.0':
resolution: {integrity: sha512-+d0CCzEYUP4PZDVGAUHMNfCEBn/GOyq/39y33N4FFUb6VPNJosfKFK5bI38mTaqwr6gxLGbzm8aeiElz+NaSDQ==}
'@endo/env-options@1.1.11':
resolution: {integrity: sha512-p9OnAPsdqoX4YJsE98e3NBVhIr2iW9gNZxHhAI2/Ul5TdRfoOViItzHzTqrgUVopw6XxA1u1uS6CykLMDUxarA==}
'@endo/errors@1.3.1':
resolution: {integrity: sha512-AmR2DSiahQR66SHlEczNWe0eUotMYE2uDwxPj9fAY2l2lovwHWGzZKXNXdQP/u62c8vetU8mG98Mh5vRZEvFTg==}
'@endo/eventual-send@1.5.0':
resolution: {integrity: sha512-mIRewS15mk+oV8y7kszyE0KWHZAlVoe98bEo/WJFJvBv1Rd83IjnofbuZcyWToWd70vgCxfkutWRJBK2Xtwd5A==}
'@endo/far@1.1.14':
resolution: {integrity: sha512-OlM+yCNWVqhwSZa6MyjkVB0oQSTAQ15KoiDGL1qowtnTnhuGUbOXuaUAsDf4eAJZ4L+h2KsJOUdyYEuhNsDPLg==}
'@endo/harden@1.1.0':
resolution: {integrity: sha512-d0CF7EVqymx9YUbkRI2sj10NrMLhrpmsaq5aBbEEbWDFxNe7F/mM8Gv8zT+iHErKEfeMCId90n5mk665SnxrRA==}
'@endo/hex@1.1.1':
resolution: {integrity: sha512-I2Q7roaHk0fE4JTj2GXfxHPR14qTFK9bOvp3nKf2Fk0r/CjgL4zFcHYolvqY15SRNmZDcywnQ+MBTW82b7BUpQ==}
'@endo/immutable-arraybuffer@2.0.0':
resolution: {integrity: sha512-xvfJvo4CgETh6UjSKwhNsFLZrqrTf2k4ZFpcprEN2UtCQRSOk+CcLP70kFbrOesB2Z/c1z9MaUZDS1JY6IlnUg==}
'@endo/init@1.1.13':
resolution: {integrity: sha512-7I94pHD0ax+WDj5dP1MKgzgSY6/GsdwCc/Ap2zFY4lzhCO944zQ4MChAW2yX28QKJK80VyJzqoW5BQ56ngEHHA==}
'@endo/lockdown@1.0.19':
resolution: {integrity: sha512-pcc6TqTBlx43422gHFzS+FrUfyJlbDoWUrynp78Mv+FrIZv4947FPcfjlXolgwvO00alMlSYxfXrrXUV93ywaA==}
'@endo/marshal@1.10.0':
resolution: {integrity: sha512-1ny6r0+XFpkpJNJEOApZQlFNB8FsDxzU2oCv1hpN1HjpszsKcTiyfXU/n/cJIs3KOOS06Tkt72pgPlasHpsVGA==}
'@endo/nat@5.2.0':
resolution: {integrity: sha512-yTNaysErNbrAWV8H2oC4e83nytG8Vzcu6VrxOkSIJQt+IVS9BUlSa1KkiKrVwu2IcIC67gJIkolPq3SgMGDjfQ==}
'@endo/ocapn@1.1.1':
resolution: {integrity: sha512-3VtHmyMsHwDPwcVusCTgtb8VXr539iZk+r8vzzOU6/cWyMhJuxRGOKR3r5EzDHVG4tY/R6tihoGt7QLnNDuceQ==}
'@endo/pass-style@1.8.2':
resolution: {integrity: sha512-63kbFNT+mropjQ0wzCkjSR5uEOT8FAIk16qHqrE+ciqSjMVd2hsW6vx10dbyo1s2xtmvXduUiTl+LqtnPa2oSw==}
'@endo/promise-kit@1.2.1':
resolution: {integrity: sha512-1nyXBOuUnVo/i47On8RCXPaT/1RIsSpJ1sGBClfZrcTtnsJfj8fElEVlJIE8NFsQiZPLNANYI+i/NYi837+s0Q==}
engines: {node: '>=11.0'}
'@endo/stream@1.3.1':
resolution: {integrity: sha512-v21VRzo5wjzw7XMHCWAmDS50hkUQi/m9C0MQTIAAqLm6w9+/Er9AEXl8FGIbz+VjtJWk5SNVbuqA51N7oBaLDQ==}
'@endo/syrup-frame@0.1.1':
resolution: {integrity: sha512-+uha+zI05RHPlSNVI7uaB2KD1EQ9gKAMcOzj1eT3y7Ng/vcW1bo7clCVVo3t+dOFOc6vYSsXgKufi40qxXRgYQ==}
'@esbuild/aix-ppc64@0.28.2':
resolution: {integrity: sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ==}
engines: {node: '>=18'}
cpu: [ppc64]
os: [aix]
'@esbuild/android-arm64@0.28.2':
resolution: {integrity: sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A==}
engines: {node: '>=18'}
cpu: [arm64]
os: [android]
'@esbuild/android-arm@0.28.2':
resolution: {integrity: sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg==}
engines: {node: '>=18'}
cpu: [arm]
os: [android]
'@esbuild/android-x64@0.28.2':
resolution: {integrity: sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q==}
engines: {node: '>=18'}
cpu: [x64]
os: [android]
'@esbuild/darwin-arm64@0.28.2':
resolution: {integrity: sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw==}
engines: {node: '>=18'}
cpu: [arm64]
os: [darwin]
'@esbuild/darwin-x64@0.28.2':
resolution: {integrity: sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw==}
engines: {node: '>=18'}
cpu: [x64]
os: [darwin]
'@esbuild/freebsd-arm64@0.28.2':
resolution: {integrity: sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw==}
engines: {node: '>=18'}
cpu: [arm64]
os: [freebsd]
'@esbuild/freebsd-x64@0.28.2':
resolution: {integrity: sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg==}
engines: {node: '>=18'}
cpu: [x64]
os: [freebsd]
'@esbuild/linux-arm64@0.28.2':
resolution: {integrity: sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug==}
engines: {node: '>=18'}
cpu: [arm64]
os: [linux]
'@esbuild/linux-arm@0.28.2':
resolution: {integrity: sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w==}
engines: {node: '>=18'}
cpu: [arm]
os: [linux]
'@esbuild/linux-ia32@0.28.2':
resolution: {integrity: sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ==}
engines: {node: '>=18'}
cpu: [ia32]
os: [linux]
'@esbuild/linux-loong64@0.28.2':
resolution: {integrity: sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ==}
engines: {node: '>=18'}
cpu: [loong64]
os: [linux]
'@esbuild/linux-mips64el@0.28.2':
resolution: {integrity: sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA==}
engines: {node: '>=18'}
cpu: [mips64el]
os: [linux]
'@esbuild/linux-ppc64@0.28.2':
resolution: {integrity: sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ==}
engines: {node: '>=18'}
cpu: [ppc64]
os: [linux]
'@esbuild/linux-riscv64@0.28.2':
resolution: {integrity: sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA==}
engines: {node: '>=18'}
cpu: [riscv64]
os: [linux]
'@esbuild/linux-s390x@0.28.2':
resolution: {integrity: sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg==}
engines: {node: '>=18'}
cpu: [s390x]
os: [linux]
'@esbuild/linux-x64@0.28.2':
resolution: {integrity: sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==}
engines: {node: '>=18'}
cpu: [x64]
os: [linux]
'@esbuild/netbsd-arm64@0.28.2':
resolution: {integrity: sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw==}
engines: {node: '>=18'}
cpu: [arm64]
os: [netbsd]
'@esbuild/netbsd-x64@0.28.2':
resolution: {integrity: sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw==}
engines: {node: '>=18'}
cpu: [x64]
os: [netbsd]
'@esbuild/openbsd-arm64@0.28.2':
resolution: {integrity: sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ==}
engines: {node: '>=18'}
cpu: [arm64]
os: [openbsd]
'@esbuild/openbsd-x64@0.28.2':
resolution: {integrity: sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw==}
engines: {node: '>=18'}
cpu: [x64]
os: [openbsd]
'@esbuild/openharmony-arm64@0.28.2':
resolution: {integrity: sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q==}
engines: {node: '>=18'}
cpu: [arm64]
os: [openharmony]
'@esbuild/sunos-x64@0.28.2':
resolution: {integrity: sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g==}
engines: {node: '>=18'}
cpu: [x64]
os: [sunos]
'@esbuild/win32-arm64@0.28.2':
resolution: {integrity: sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ==}
engines: {node: '>=18'}
cpu: [arm64]
os: [win32]
'@esbuild/win32-ia32@0.28.2':
resolution: {integrity: sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA==}
engines: {node: '>=18'}
cpu: [ia32]
os: [win32]
'@esbuild/win32-x64@0.28.2':
resolution: {integrity: sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g==}
engines: {node: '>=18'}
cpu: [x64]
os: [win32]
'@noble/curves@1.9.7':
resolution: {integrity: sha512-gbKGcRUYIjA3/zCCNaWDciTMFI0dCkvou3TL8Zmy5Nc7sJ47a0jtOeZoTaMxkuqRo9cRhjOdZJXegxYE5FN/xw==}
engines: {node: ^14.21.3 || >=16}
'@noble/hashes@1.8.0':
resolution: {integrity: sha512-jCs9ldd7NwzpgXDIf6P3+NrHh9/sD6CQdxHyjQI+h/6rDNo88ypBxxz45UDuZHz9r3tNz7N/VInSVoVdtXEI4A==}
engines: {node: ^14.21.3 || >=16}
capnweb@0.12.0:
resolution: {integrity: sha512-jgZ/LMtMVTi+RVlooIslH78Gg23XbDTdvcLghWx3oz17D6u5GuJARt+uhZk/wcTo+f81eeabfnow4d3zLBj+JQ==}
esbuild@0.28.2:
resolution: {integrity: sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA==}
engines: {node: '>=18'}
hasBin: true
ses@2.3.0:
resolution: {integrity: sha512-qd3iWzDqGKllI2FmExKsWLrJwt+6COWb8jUdzaCn8Cq5OTeoXdQ62Gn/xLy0GWLORC3du6yBAVe/V0B0dLVKew==}
workerd@1.20261006.1:
resolution: {integrity: sha512-8MadBuffZ5NdpLwZjUmsqT5QlaKXq4gjgT6wzcKeynZTICNXyePhc+5QVoKiK7+MMhFTLNrRO2KXLlpk8Q6BkA==}
engines: {node: '>=16'}
@@ -81,6 +333,240 @@ snapshots:
'@cloudflare/workerd-windows-64@1.20261006.1':
optional: true
'@endo/base64@1.1.1': {}
'@endo/bytes@1.0.2':
dependencies:
'@endo/harden': 1.1.0
'@endo/immutable-arraybuffer': 2.0.0
'@endo/cache-map@1.1.0': {}
'@endo/common@1.4.0':
dependencies:
'@endo/errors': 1.3.1
'@endo/eventual-send': 1.5.0
'@endo/harden': 1.1.0
'@endo/promise-kit': 1.2.1
'@endo/env-options@1.1.11': {}
'@endo/errors@1.3.1':
dependencies:
'@endo/harden': 1.1.0
ses: 2.3.0
'@endo/eventual-send@1.5.0':
dependencies:
'@endo/env-options': 1.1.11
'@endo/harden': 1.1.0
'@endo/far@1.1.14':
dependencies:
'@endo/errors': 1.3.1
'@endo/eventual-send': 1.5.0
'@endo/pass-style': 1.8.2
'@endo/harden@1.1.0': {}
'@endo/hex@1.1.1':
dependencies:
'@endo/harden': 1.1.0
'@endo/immutable-arraybuffer@2.0.0': {}
'@endo/init@1.1.13':
dependencies:
'@endo/base64': 1.1.1
'@endo/eventual-send': 1.5.0
'@endo/harden': 1.1.0
'@endo/lockdown': 1.0.19
'@endo/promise-kit': 1.2.1
'@endo/lockdown@1.0.19':
dependencies:
ses: 2.3.0
'@endo/marshal@1.10.0':
dependencies:
'@endo/common': 1.4.0
'@endo/env-options': 1.1.11
'@endo/errors': 1.3.1
'@endo/eventual-send': 1.5.0
'@endo/harden': 1.1.0
'@endo/nat': 5.2.0
'@endo/pass-style': 1.8.2
'@endo/nat@5.2.0': {}
'@endo/ocapn@1.1.1':
dependencies:
'@endo/bytes': 1.0.2
'@endo/eventual-send': 1.5.0
'@endo/harden': 1.1.0
'@endo/hex': 1.1.1
'@endo/init': 1.1.13
'@endo/marshal': 1.10.0
'@endo/nat': 5.2.0
'@endo/pass-style': 1.8.2
'@endo/promise-kit': 1.2.1
'@endo/stream': 1.3.1
'@endo/syrup-frame': 0.1.1
'@noble/curves': 1.9.7
'@noble/hashes': 1.8.0
ws: 8.22.0
transitivePeerDependencies:
- bufferutil
- utf-8-validate
'@endo/pass-style@1.8.2':
dependencies:
'@endo/common': 1.4.0
'@endo/env-options': 1.1.11
'@endo/errors': 1.3.1
'@endo/eventual-send': 1.5.0
'@endo/harden': 1.1.0
'@endo/promise-kit': 1.2.1
'@endo/promise-kit@1.2.1':
dependencies:
'@endo/harden': 1.1.0
'@endo/stream@1.3.1':
dependencies:
'@endo/eventual-send': 1.5.0
'@endo/harden': 1.1.0
'@endo/promise-kit': 1.2.1
ses: 2.3.0
'@endo/syrup-frame@0.1.1':
dependencies:
'@endo/harden': 1.1.0
'@endo/init': 1.1.13
'@endo/promise-kit': 1.2.1
'@endo/stream': 1.3.1
'@esbuild/aix-ppc64@0.28.2':
optional: true
'@esbuild/android-arm64@0.28.2':
optional: true
'@esbuild/android-arm@0.28.2':
optional: true
'@esbuild/android-x64@0.28.2':
optional: true
'@esbuild/darwin-arm64@0.28.2':
optional: true
'@esbuild/darwin-x64@0.28.2':
optional: true
'@esbuild/freebsd-arm64@0.28.2':
optional: true
'@esbuild/freebsd-x64@0.28.2':
optional: true
'@esbuild/linux-arm64@0.28.2':
optional: true
'@esbuild/linux-arm@0.28.2':
optional: true
'@esbuild/linux-ia32@0.28.2':
optional: true
'@esbuild/linux-loong64@0.28.2':
optional: true
'@esbuild/linux-mips64el@0.28.2':
optional: true
'@esbuild/linux-ppc64@0.28.2':
optional: true
'@esbuild/linux-riscv64@0.28.2':
optional: true
'@esbuild/linux-s390x@0.28.2':
optional: true
'@esbuild/linux-x64@0.28.2':
optional: true
'@esbuild/netbsd-arm64@0.28.2':
optional: true
'@esbuild/netbsd-x64@0.28.2':
optional: true
'@esbuild/openbsd-arm64@0.28.2':
optional: true
'@esbuild/openbsd-x64@0.28.2':
optional: true
'@esbuild/openharmony-arm64@0.28.2':
optional: true
'@esbuild/sunos-x64@0.28.2':
optional: true
'@esbuild/win32-arm64@0.28.2':
optional: true
'@esbuild/win32-ia32@0.28.2':
optional: true
'@esbuild/win32-x64@0.28.2':
optional: true
'@noble/curves@1.9.7':
dependencies:
'@noble/hashes': 1.8.0
'@noble/hashes@1.8.0': {}
capnweb@0.12.0: {}
esbuild@0.28.2:
optionalDependencies:
'@esbuild/aix-ppc64': 0.28.2
'@esbuild/android-arm': 0.28.2
'@esbuild/android-arm64': 0.28.2
'@esbuild/android-x64': 0.28.2
'@esbuild/darwin-arm64': 0.28.2
'@esbuild/darwin-x64': 0.28.2
'@esbuild/freebsd-arm64': 0.28.2
'@esbuild/freebsd-x64': 0.28.2
'@esbuild/linux-arm': 0.28.2
'@esbuild/linux-arm64': 0.28.2
'@esbuild/linux-ia32': 0.28.2
'@esbuild/linux-loong64': 0.28.2
'@esbuild/linux-mips64el': 0.28.2
'@esbuild/linux-ppc64': 0.28.2
'@esbuild/linux-riscv64': 0.28.2
'@esbuild/linux-s390x': 0.28.2
'@esbuild/linux-x64': 0.28.2
'@esbuild/netbsd-arm64': 0.28.2
'@esbuild/netbsd-x64': 0.28.2
'@esbuild/openbsd-arm64': 0.28.2
'@esbuild/openbsd-x64': 0.28.2
'@esbuild/openharmony-arm64': 0.28.2
'@esbuild/sunos-x64': 0.28.2
'@esbuild/win32-arm64': 0.28.2
'@esbuild/win32-ia32': 0.28.2
'@esbuild/win32-x64': 0.28.2
ses@2.3.0:
dependencies:
'@endo/cache-map': 1.1.0
'@endo/env-options': 1.1.11
'@endo/immutable-arraybuffer': 2.0.0
workerd@1.20261006.1:
optionalDependencies:
'@cloudflare/workerd-darwin-64': 1.20261006.1
+20
View File
@@ -0,0 +1,20 @@
/**
* @file Create the hello-world vat on a running toybox and mount it.
*
* Usage: node --import ./test/lockdown.js scripts/hello.js [name]
* then visit http://127.0.0.1:8787/?vat=<name> (default name: hello).
*/
import { readFile } from "node:fs/promises";
import { E } from "@endo/far";
import { connectClient } from "../client.js";
const name = process.argv[2] ?? "hello";
const client = await connectClient("admin.sock", `hello-script-${process.pid}`);
try {
const ingress = await E(client.vatAdmin).makeIngress({ vat: name });
const { id, root } = await E(client.vatAdmin).createVat(await readFile("vats/hello.js", "utf8"), { ingress });
await root; // its handoff, settled before the client goes
console.log(`vat ${id}\nhttp://127.0.0.1:8787/?vat=${name}`);
} finally {
client.close();
}
+500 -27
View File
@@ -1,10 +1,30 @@
/** @file toybox supervisor: one Vat per name; its app runs as a facet on installed code. */
/** @file toybox supervisor: vats with code installed at runtime, each app a facet; ingress by mount, egress by grant. */
import "@endo/init";
import { DurableObject, WorkerEntrypoint } from "cloudflare:workers";
import { RpcTarget, newWebSocketRpcSession, newWorkersRpcResponse } from "capnweb";
import { E, Far, passStyleOf } from "@endo/far";
import { makeClient, swissnumFromBytes } from "@endo/ocapn";
import { mergeLabels, urlLabels } from "./labels.js";
import { LOOKUP, MOUNT, MOUNT_LABELS, MOUNT_SCHEMA, OVERLAP, UNMOUNT, UNMOUNT_LABELS } from "./mounts-sql.js";
import { locationOf, makeRpcNetLayer } from "./netlayer.js";
import VAT_RUNTIME from "vat-runtime.txt";
/**
* A stored grant: labels as JSON, or (before labels) a bare `host[:port]`.
*
* @param {string} stored
*/
const grantLabels = (stored) => {
if (stored.startsWith("{")) return JSON.parse(stored);
const { host, port } = urlLabels(`http://${stored}`);
return stored.includes(":") ? { host, port } : { host };
};
const hex = (buf) => [...new Uint8Array(buf)].map((b) => b.toString(16).padStart(2, "0")).join("");
/**
* Holds the vat's installed code and egress grant; the app facet holds the app's state.
* One vat, by DO id: holds its installed code and egress grant; the app
* facet holds the app's state. Its mount, if any, is in the {@link Directory}.
*
* Code is stored by hash so the facet can be rebuilt after hibernation or a
* restart. The app's global `fetch` throws; a granted host arrives as
@@ -21,81 +41,534 @@ export class Vat extends DurableObject {
ctx.storage.sql.exec(
"CREATE TABLE IF NOT EXISTS incarnation (k INTEGER PRIMARY KEY CHECK (k = 0), n INTEGER, egress TEXT)",
);
// absent: an app vat (exports class App), from before OCapN vats
ctx.storage.sql.exec("CREATE TABLE IF NOT EXISTS kind (k INTEGER PRIMARY KEY CHECK (k = 0), kind TEXT)");
}
/**
* Install `src` and run the app on it; the app's storage carries over.
*
* @param {string} src an ES module exporting `class App extends DurableObject`
* @param {string | null} egress the one host (`name[:port]`) the app may fetch, or none
* @param {string | null} egress the app's grant: egress labels as JSON (see labels.js), or none
* @param {"app" | "ocapn"} kind `app`: `src` exports `class App`; `ocapn`:
* `src` exports `buildRootObject`, run by vat-runtime.js
* @returns {Promise<string>} the code's SHA-256, hex
*/
async start(src, egress) {
const hash = hex(await crypto.subtle.digest("SHA-256", new TextEncoder().encode(src)));
async start(src, egress, kind = "app") {
const hash = hex(await crypto.subtle.digest("SHA-256", encodeUtf8(src)));
const { sql } = this.ctx.storage;
sql.exec("INSERT OR IGNORE INTO bundle VALUES (?, ?)", hash, src);
sql.exec("INSERT OR REPLACE INTO current VALUES (0, ?)", hash);
const [prev] = sql.exec("SELECT n FROM incarnation").toArray();
sql.exec("INSERT OR REPLACE INTO incarnation VALUES (0, ?, ?)", (prev?.n ?? 0) + 1, egress);
sql.exec("INSERT OR REPLACE INTO kind VALUES (0, ?)", kind);
this.ctx.facets.abort("app", new Error("upgraded"));
return hash;
}
#app({ hash, src, n, egress }) {
#app({ src, n, egress, kind }) {
return this.ctx.facets.get("app", () => {
const env = egress ? { egress: this.ctx.exports.Egress({ props: { labels: grantLabels(egress) } }) } : {};
const code =
kind === "ocapn"
? {
mainModule: "runtime.js",
modules: { "runtime.js": VAT_RUNTIME, "vat.js": src },
env: { ...env, vatId: this.ctx.id.toString(), router: this.ctx.exports.Router({ props: {} }) },
}
: { mainModule: "app.js", modules: { "app.js": src }, env };
const worker = this.env.LOADER.get(`${this.ctx.id}:${n}`, () => ({
compatibilityDate: "2026-10-01",
mainModule: "app.js",
modules: { "app.js": src },
env: egress ? { egress: this.ctx.exports.Egress({ props: { host: egress } }) } : {},
...code,
globalOutbound: null,
}));
return { class: worker.getDurableObjectClass("App") };
});
}
fetch(request) {
#current() {
const [row] = this.ctx.storage.sql
.exec(
`SELECT b.hash, b.src, coalesce(i.n, 0) AS n, i.egress
FROM current c JOIN bundle b USING (hash) LEFT JOIN incarnation i ON i.k = 0`,
`SELECT b.hash, b.src, coalesce(i.n, 0) AS n, i.egress, coalesce(k.kind, 'app') AS kind
FROM current c JOIN bundle b USING (hash)
LEFT JOIN incarnation i ON i.k = 0 LEFT JOIN kind k ON k.k = 0`,
)
.toArray();
return row;
}
fetch(request) {
const row = this.#current();
if (!row) return new Response("no code installed\n", { status: 404 });
return this.#app(row).fetch(request);
}
/** An OCapN vat's gateway, for the router. */
gateway() {
const row = this.#current();
if (row?.kind !== "ocapn") throw new Error("not an OCapN vat");
return this.#app(row).gateway();
}
}
/**
* A grant of one host: `fetch`, WebSocket upgrades included, reaches the
* switchboard only for the host in `ctx.props`, which no holder can change.
* A vat's egress grant: a switchboard handle constrained to the labels in
* `ctx.props`, which no holder can change. The switchboard enforces them;
* a refusal comes back as a 403.
*/
export class Egress extends WorkerEntrypoint {
fetch(request) {
const { host } = new URL(request.url);
if (host !== this.ctx.props.host) return new Response(`egress to ${host} not granted\n`, { status: 403 });
return this.env.SWITCHBOARD.fetch(request);
/** A narrower grant: these labels too (conflicting ones throw). */
constrain(more) {
return this.ctx.exports.Egress({ props: { labels: mergeLabels(this.ctx.props.labels, more) } });
}
async #handle() {
const resp = await this.env.SWITCHBOARD.fetch("http://switchboard/", { headers: { Upgrade: "websocket" } });
const ws = resp.webSocket;
if (!ws) throw new Error(`switchboard refused a session: ${resp.status}`);
ws.accept();
return newWebSocketRpcSession(ws).constrain(this.ctx.props.labels);
}
async fetch(request) {
try {
const handle = await this.#handle();
if (request.headers.get("Upgrade")?.toLowerCase() !== "websocket") return await handle.fetch(request);
const protocols = (request.headers.get("Sec-WebSocket-Protocol") ?? "").split(",").map((p) => p.trim()).filter(Boolean);
const { readable, writable, protocol } = await handle.webSocket(request.url, protocols);
return this.#bridge(readable, writable, protocol);
} catch (e) {
return new Response(`egress refused: ${e.message}\n`, { status: 403 });
}
}
/** A WebSocket for the vat whose messages travel the switchboard's streams. */
#bridge(readable, writable, protocol) {
const [client, server] = Object.values(new WebSocketPair());
server.accept();
const writer = writable.getWriter();
server.addEventListener("message", (e) => writer.write(e.data).catch(() => server.close(1011)));
server.addEventListener("close", () => writer.close().catch(() => {}));
this.ctx.waitUntil(
(async () => {
for await (const m of readable) server.send(m);
server.close();
})().catch(() => server.close(1011)),
);
const headers = protocol ? { "Sec-WebSocket-Protocol": protocol } : {};
return new Response(null, { status: 101, webSocket: client, headers });
}
}
const vatOf = (env, url) => env.VATS.getByName(new URL(url).pathname.split("/")[1]);
/** The vatAdmin's designator, and the swissnum of its bootstrap object. */
const VAT_ADMIN = "vatAdmin";
/** The swissnum of an OCapN vat's bootstrap (see vat-runtime.js). */
const VAT_SWISSNUM = "vat";
/** UTF-8 bytes of a string: the method alone, so no encoder object is in module scope. */
const encodeUtf8 = harden(TextEncoder.prototype.encode.bind(new TextEncoder()));
/** The http socket: `/<vat>/...` goes to that vat's app. */
export class Public extends WorkerEntrypoint {
/**
* The vatAdmin: an OCapN peer, and the router through which every OCapN
* peer here reaches every other. One instance, by name.
*
* Peers find each other by designator: the vatAdmin's is `vatAdmin`, a
* vat's is its id, and a client registers its own when it connects to
* the admin socket. Nothing here can dial out, so a handoff's receiver
* reaches the exporter through this router.
*/
export class VatAdmin extends DurableObject {
#clients = new Map();
/** mount key → the handler serving it: live, so lost when this object is evicted */
#handlers = new Map();
/** mount key → resolvers waiting for a handler */
#waiting = new Map();
/** mount key ↔ its ingress object, made again as needed */
#ingresses = new Map();
#ingressKeys = new WeakMap();
#reviving = new Map();
#ready;
constructor(ctx, env) {
super(ctx, env);
const { sql } = ctx.storage;
// a vat's params, encoded (see #encode); a vat absent here can't be restarted
sql.exec("CREATE TABLE IF NOT EXISTS vat_params (vat TEXT PRIMARY KEY, params TEXT)");
// which vat was given which ingress, so a request can restart it
sql.exec("CREATE TABLE IF NOT EXISTS ingress_vat (key TEXT PRIMARY KEY, vat TEXT)");
this.#ready = this.#boot();
}
/** @param {string} key */
#ingressFor(key) {
let ingress = this.#ingresses.get(key);
if (!ingress) {
ingress = Far("ingress", {
/** @param {{ handle: (request: object) => object }} handler */
serve: (handler) => {
this.#handlers.set(key, handler);
for (const resolve of this.#waiting.get(key) ?? []) resolve();
this.#waiting.delete(key);
},
});
this.#ingresses.set(key, ingress);
this.#ingressKeys.set(ingress, key);
}
return ingress;
}
/**
* `params` as JSON, an ingress by its key; `null` if anything in it can't
* be made again here (a remote object of another peer, say).
*/
#encode(params) {
const keys = [];
const walk = (v) => {
const style = passStyleOf(v);
if (style === "remotable") {
const key = this.#ingressKeys.get(v);
if (!key) throw new Error("not restartable");
keys.push(key);
return { "@ingress": key };
}
if (style === "copyArray") return v.map(walk);
if (style === "copyRecord") return Object.fromEntries(Object.entries(v).map(([k, x]) => [k, walk(x)]));
if (["string", "number", "boolean", "null"].includes(style)) return v;
throw new Error("not restartable");
};
try {
return { json: JSON.stringify(walk(params)), keys };
} catch {
return null;
}
}
#decode(json) {
const walk = (v) => {
if (Array.isArray(v)) return v.map(walk);
if (v && typeof v === "object") {
if ("@ingress" in v) return this.#ingressFor(v["@ingress"]);
return Object.fromEntries(Object.entries(v).map(([k, x]) => [k, walk(x)]));
}
return v;
};
return harden(walk(JSON.parse(json)));
}
/** (Re)start a vat's code with `params`: it rebuilds its root. */
async #startVat(vat, params) {
const vatBootstrap = await this.#enliven(vat, VAT_SWISSNUM);
return E(vatBootstrap).start(params);
}
/**
* The handler for a mount whose handler went with an evicted instance:
* restart the vat given its ingress, and wait for it to serve again.
*
* @param {string} key
*/
#revive(key) {
let reviving = this.#reviving.get(key);
if (!reviving) {
reviving = (async () => {
const [row] = this.ctx.storage.sql
.exec("SELECT i.vat, p.params FROM ingress_vat i JOIN vat_params p USING (vat) WHERE i.key = ?", key)
.toArray();
if (!row) return null;
const served = new Promise((resolve) => this.#waiting.set(key, [...(this.#waiting.get(key) ?? []), resolve]));
await this.#startVat(row.vat, this.#decode(row.params));
await Promise.race([served, new Promise((resolve) => setTimeout(resolve, 5000))]);
return this.#handlers.get(key) ?? null;
})().finally(() => this.#reviving.delete(key));
this.#reviving.set(key, reviving);
}
return reviving;
}
async #boot() {
const vatAdmin = Far("vatAdmin", {
/**
* Create an OCapN vat running `src`; `params` (any passable,
* remote objects included) reach its `buildRootObject`.
*
* If `params` holds only data and ingresses, they are kept, and the
* vat is restarted with them when a request needs it after an
* eviction or a restart.
*
* @param {string} src an ES module exporting `buildRootObject(vatPowers, params)`
*/
createVat: async (src, params = {}) => {
const id = this.env.VATS.newUniqueId();
const vat = id.toString();
await this.env.VATS.get(id).start(src, null, "ocapn");
const encoded = this.#encode(params);
if (encoded) {
const { sql } = this.ctx.storage;
sql.exec("INSERT INTO vat_params VALUES (?, ?)", vat, encoded.json);
for (const key of encoded.keys) sql.exec("INSERT OR REPLACE INTO ingress_vat VALUES (?, ?)", key, vat);
}
const root = await this.#startVat(vat, params);
return harden({ id: vat, root });
},
/**
* Reserve a mount at `labels` and return the ingress for it: whoever
* holds the ingress (a vat it is passed to, say) serves requests
* that carry those labels.
*
* @param {Record<string, string>} labels
*/
makeIngress: async (labels) => {
const key = `ingress:${crypto.randomUUID()}`;
await this.env.DIRECTORY.getByName("directory").mount(key, harden({ ...labels }));
return this.#ingressFor(key);
},
});
// workerd runs FinalizationRegistry callbacks outside any request, where
// sending the resulting GC message is forbidden I/O. So imports are
// never collected here; they go when their session does.
this.ocapn = makeClient({
debugLabel: VAT_ADMIN,
swissnumTable: new Map([[VAT_ADMIN, vatAdmin]]),
enableImportCollection: false,
});
const netlayer = await this.ocapn.registerNetlayer(
makeRpcNetLayer({ designator: VAT_ADMIN, openGateway: (designator) => this.gateway(designator) }),
);
this.own = netlayer.gateway;
}
#enliven(designator, swissnum) {
return this.ocapn.enlivenSturdyRef(
this.ocapn.makeSturdyRef(locationOf(designator), swissnumFromBytes(encodeUtf8(swissnum))),
);
}
/**
* The gateway of the peer with this designator.
*
* @param {string} designator
*/
async gateway(designator) {
await this.#ready;
if (designator === VAT_ADMIN) return this.own;
const client = this.#clients.get(designator);
if (client) return client;
let id;
try {
id = this.env.VATS.idFromString(designator);
} catch {
throw new Error("no such peer");
}
return this.env.VATS.get(id).gateway();
}
/**
* One HTTP request, as passable data, for the handler serving `key`;
* `null` when nothing serves it.
*
* @param {string} key
* @param {object} request
*/
async serveHttp(key, request) {
await this.#ready;
const handler = this.#handlers.get(key) ?? (await this.#revive(key));
return handler ? E(handler).handle(harden(request)) : null;
}
/** A WebSocket here starts a Cap'n Web session whose main object is an {@link AdminSession}. */
fetch(request) {
return vatOf(this.env, request.url).fetch(request);
return newWorkersRpcResponse(request, new AdminSession(this, this.#clients));
}
}
/**
* The admin socket: `PUT /<vat>` with an ES module body installs and starts
* it; `?egress=<host>` grants it that one host.
* What an OCapN client on the admin socket gets: register its own
* gateway under a designator, and reach any peer's.
*/
class AdminSession extends RpcTarget {
#vatAdmin;
#clients;
constructor(vatAdmin, clients) {
super();
this.#vatAdmin = vatAdmin;
this.#clients = clients;
}
/** @param {string} designator @param {any} gateway */
register(designator, gateway) {
if (designator === VAT_ADMIN || /^[0-9a-f]{64}$/.test(designator) || this.#clients.has(designator)) {
throw new Error("designator taken");
}
this.#clients.set(designator, gateway.dup?.() ?? gateway);
}
/** @param {string} designator */
gateway(designator) {
return this.#vatAdmin.gateway(designator);
}
}
/** A vat's way to the router: `env.router` in an OCapN vat. */
export class Router extends WorkerEntrypoint {
gateway(designator) {
return this.env.VAT_ADMIN.getByName(VAT_ADMIN).gateway(designator);
}
}
/**
* Mounts: which vat serves which label set. One instance, by name.
*
* Registration follows Flower: a mount must conflict with every other, so
* no request can carry two mounts' labels. Lookup is stricter than
* Flower's: the request must carry all of a mount's labels.
*/
export class Directory extends DurableObject {
constructor(ctx, env) {
super(ctx, env);
for (const statement of MOUNT_SCHEMA) ctx.storage.sql.exec(statement);
}
/**
* Mount `vat` at `labels`, replacing its mount; `null` unmounts it.
*
* @param {string} vat a Vat's DO id, hex
* @param {Record<string, string> | null} labels
*/
mount(vat, labels) {
const { sql } = this.ctx.storage;
this.ctx.storage.transactionSync(() => {
sql.exec(UNMOUNT_LABELS, vat);
if (!labels) return void sql.exec(UNMOUNT, vat);
const json = JSON.stringify(labels);
if (sql.exec(OVERLAP, json, vat).toArray().length) {
throw new Error("mount overlaps another: add a label whose value tells them apart");
}
sql.exec(MOUNT, vat, json);
sql.exec(MOUNT_LABELS, vat, json);
});
}
/**
* The mount whose labels `request` carries, if any. Registration keeps
* mounts pairwise conflicting, so at most one can match.
*
* @param {Record<string, string>} request
*/
lookup(request) {
const [row] = this.ctx.storage.sql.exec(LOOKUP, JSON.stringify(request)).toArray();
return row ? { vat: row.vat, labels: JSON.parse(row.labels) } : null;
}
}
/**
* A request's labels: its query parameters.
*
* Routing carries no secrets: a URL ends up in logs and `Referer`s. A
* secret travels in-band instead, as on a webkey page: the page keeps it
* in its fragment, reaches the vat by public labels, and sends it in the
* body of a call, which the vat checks like a swissnum.
*
* @param {Request} request
*/
const requestLabels = (request) => {
const labels = {};
for (const [k, v] of new URL(request.url).searchParams) {
if (k in labels) throw new Error(`label ${k} given twice`);
labels[k] = v;
}
return Object.freeze(labels);
};
/**
* The http socket: the ingress. A request reaches the vat mounted at labels
* it carries, which sees the mount's and the request's labels together in
* a `Toybox-Connection-Labels` header (JSON), as a Flower server gets its
* connection labels.
*/
export class Public extends WorkerEntrypoint {
async fetch(request) {
let labels;
try {
labels = requestLabels(request);
} catch (e) {
return new Response(`bad labels: ${e.message}\n`, { status: 400 });
}
const mount = await this.env.DIRECTORY.getByName("directory").lookup(labels);
if (!mount) return new Response("nothing mounted here\n", { status: 404 });
if (mount.vat.startsWith("ingress:")) return this.#serveOcapn(mount, labels, request);
const forwarded = new Request(request);
forwarded.headers.set("Toybox-Connection-Labels", JSON.stringify(mergeLabels(mount.labels, labels)));
const resp = await this.env.VATS.get(this.env.VATS.idFromString(mount.vat)).fetch(forwarded);
if (resp.webSocket) return resp;
const sealed = new Response(resp.body, resp);
sealed.headers.set("Referrer-Policy", "no-referrer");
return sealed;
}
/** A mount made by `makeIngress`: the request goes, as data, to whoever serves it. */
async #serveOcapn(mount, labels, request) {
const { pathname } = new URL(request.url);
const body = ["GET", "HEAD"].includes(request.method) ? "" : await request.text();
const req = { method: request.method, path: pathname, labels: mergeLabels(mount.labels, labels), body };
const res = await this.env.VAT_ADMIN.getByName(VAT_ADMIN).serveHttp(mount.vat, req);
if (!res) return new Response("mounted, but nothing serves it now\n", { status: 503 });
const resp = new Response(res.body ?? "", { status: res.status ?? 200, headers: res.headers ?? {} });
resp.headers.set("Referrer-Policy", "no-referrer");
return resp;
}
}
/**
* Grants in a query parameter, as JSON labels; absent means none.
*
* @param {URL} url
* @param {string} name
*/
const grantParam = (url, name) => {
const v = url.searchParams.get(name);
return v ? JSON.parse(v) : null;
};
/**
* The admin socket: a WebSocket upgrade reaches the {@link VatAdmin} (OCapN
* over Cap'n Web); otherwise, app vats over HTTP:
*
* - `POST /` with an ES module body creates a vat and starts it; the reply
* is `{"vat": <id>, "hash": <code hash>}`, and the id addresses it here.
* - `PUT /<id>` with a body upgrades it; its storage carries over.
* - `PUT /by-name/<name>` does the same for a vat from before ids, when
* vats were named.
*
* Every start states the vat's grants, absent meaning none:
* `?egress=<labels>` (see {@link Egress}) and `?mount=<labels>` (ingress:
* see {@link Public}).
*/
export class Admin extends WorkerEntrypoint {
async fetch(request) {
if (request.method !== "PUT") return new Response("PUT code to /<vat>\n", { status: 405 });
const egress = new URL(request.url).searchParams.get("egress");
return new Response((await vatOf(this.env, request.url).start(await request.text(), egress)) + "\n");
if (request.headers.get("Upgrade")?.toLowerCase() === "websocket") {
return this.env.VAT_ADMIN.getByName(VAT_ADMIN).fetch(request);
}
const url = new URL(request.url);
const [, first, second] = url.pathname.split("/");
let id;
if (request.method === "POST" && !first) id = this.env.VATS.newUniqueId();
else if (request.method === "PUT" && first === "by-name" && second) id = this.env.VATS.idFromName(second);
else if (request.method === "PUT" && first && !second) {
try {
id = this.env.VATS.idFromString(first);
} catch {
return new Response("no such vat\n", { status: 404 });
}
} else return new Response("POST / or PUT /<id> or PUT /by-name/<name>, with code\n", { status: 405 });
const vat = id.toString();
try {
await this.env.DIRECTORY.getByName("directory").mount(vat, grantParam(url, "mount"));
} catch (e) {
return new Response(`${e.message}\n`, { status: 409 });
}
const egress = url.searchParams.get("egress");
const hash = await this.env.VATS.get(id).start(await request.text(), egress);
return Response.json({ vat, hash });
}
}
+20 -12
View File
@@ -1,26 +1,34 @@
# switchboard: toybox's only way out. A forward proxy (HTTP and WebSocket)
# that toybox's Egress gateway reaches over a socket handed in with -S/-s.
# switchboard: toybox's only way out, after Flower's switchboard.
#
# The network service is the switchboard's outer fence; under systemd,
# IPAddressAllow=/IPAddressDeny= narrow it further (see systemd/).
# Each socket is a handle with fixed labels: a WebSocket upgrade starts a
# Cap'n Web session whose main object is that handle (see switchboard.js).
# Whoever starts workerd hands the sockets in (-S/-s), so whoever holds a
# socket's fd holds that handle and no other.
using Workerd = import "/workerd/workerd.capnp";
const config :Workerd.Config = (
services = [
(name = "main", worker = .switchboard),
(name = "internet", network = (allow = ["public", "local"], tlsOptions = (trustBrowserCas = true))),
(name = "nowhere", network = (allow = [])),
(name = "public-net", network = (allow = ["public"], tlsOptions = (trustBrowserCas = true))),
(name = "local-net", network = (allow = ["local"], tlsOptions = (trustBrowserCas = true))),
],
sockets = [
# proxy style: the request line carries the full URL, scheme included
(name = "http", http = (style = proxy), service = "main"),
(name = "public", http = (), service = (name = "main", props = (json = "{\"network\":\"public\"}"))),
(name = "local", http = (), service = (name = "main", props = (json = "{\"network\":\"local\"}"))),
],
);
const switchboard :Workerd.Worker = (
compatibilityDate = "2026-10-01",
modules = [(name = "switchboard.js", esModule = embed "switchboard.js")],
durableObjectNamespaces = [(className = "Switchboard", uniqueKey = "toybox-switchboard")],
durableObjectStorage = (inMemory = void),
bindings = [(name = "BOARD", durableObjectNamespace = "Switchboard")],
globalOutbound = "internet",
modules = [
(name = "switchboard.js", esModule = embed "switchboard.js"),
(name = "labels.js", esModule = embed "labels.js"),
(name = "capnweb", esModule = embed "node_modules/capnweb/dist/index-workers.js"),
],
bindings = [
(name = "PUBLIC", service = "public-net"),
(name = "LOCAL", service = "local-net"),
],
globalOutbound = "nowhere",
);
+74 -15
View File
@@ -1,22 +1,81 @@
/** @file switchboard: forwards each proxied request, WebSocket upgrades included. */
import { DurableObject } from "cloudflare:workers";
/** @file switchboard: Flower-style egress handles, served over Cap'n Web. */
import { WorkerEntrypoint } from "cloudflare:workers";
import { RpcTarget, newWorkersRpcResponse } from "capnweb";
import { checkUrl, mergeLabels } from "./labels.js";
/**
* The one place egress happens. A 101 response from `fetch` carries its
* `webSocket`; returning it passes the socket through to the caller.
*
* Which vat may reach which host is decided before a request gets here
* (toybox's Egress gateway); this process's network service and its
* systemd unit are the outer fence.
* One egress handle: reaches only URLs within its labels, through the
* network its `network` label names. Like a Flower switchboard handle, it
* can only be narrowed; unlike one, it is a Cap'n Web stub, not an fd.
*/
export class Switchboard extends DurableObject {
export class Handle extends RpcTarget {
#labels;
#net;
/**
* @param {Record<string, string>} labels
* @param {Fetcher} net
*/
constructor(labels, net) {
super();
this.#labels = labels;
this.#net = net;
}
/** @param {Record<string, string>} more */
constrain(more) {
return new Handle(mergeLabels(this.#labels, more), this.#net);
}
labels() {
return this.#labels;
}
/** @param {Request} request */
fetch(request) {
return fetch(request);
checkUrl(this.#labels, request.url);
return this.#net.fetch(request);
}
/**
* Open a WebSocket to `url`; its messages cross the session as streams.
*
* @param {string} url `http:` or `https:`, as for a `fetch` upgrade
* @param {string[]} [protocols]
*/
async webSocket(url, protocols = []) {
checkUrl(this.#labels, url);
const headers = { Upgrade: "websocket" };
if (protocols.length) headers["Sec-WebSocket-Protocol"] = protocols.join(", ");
const resp = await this.#net.fetch(url, { headers });
const ws = resp.webSocket;
if (!ws) throw new Error(`no websocket: ${resp.status}`);
ws.accept();
const readable = new ReadableStream({
start(controller) {
ws.addEventListener("message", (e) => controller.enqueue(e.data));
ws.addEventListener("close", () => controller.close());
ws.addEventListener("error", () => controller.error(new Error("websocket error")));
},
});
const writable = new WritableStream({
write: (chunk) => ws.send(chunk),
close: () => ws.close(),
abort: () => ws.close(1011),
});
return { readable, writable, protocol: resp.headers.get("Sec-WebSocket-Protocol") ?? "" };
}
}
export default {
fetch(request, env) {
return env.BOARD.getByName("switchboard").fetch(request);
},
};
/**
* A switchboard socket: a WebSocket upgrade here starts a Cap'n Web
* session whose main object is a {@link Handle} with this socket's labels.
*/
export default class Root extends WorkerEntrypoint {
fetch(request) {
const labels = Object.freeze({ ...this.ctx.props });
const net = { public: this.env.PUBLIC, local: this.env.LOCAL }[labels.network];
if (!net) return new Response("socket has no network\n", { status: 500 });
return newWorkersRpcResponse(request, new Handle(labels, net));
}
}
+6 -4
View File
@@ -1,12 +1,14 @@
[Unit]
Description=toybox switchboard (workerd forward proxy)
Description=toybox switchboard (workerd, Cap'n Web egress handles)
Requires=toybox-switchboard.socket
[Service]
ExecStart=/opt/toybox/node_modules/workerd/bin/workerd serve --experimental /opt/toybox/switchboard.capnp -S http=3
# the local (localhost) handle goes where no other unit can open it;
# hand it out deliberately, if ever
ExecStart=/opt/toybox/node_modules/workerd/bin/workerd serve --experimental /opt/toybox/switchboard.capnp -S public=3 -s local=unix:/run/toybox-switchboard/local.sock
RuntimeDirectory=toybox-switchboard
RuntimeDirectoryMode=0700
DynamicUser=yes
# the outer fence: switchboard.capnp allows "local" for tests; deny it here
IPAddressDeny=localhost link-local multicast
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
+3 -3
View File
@@ -1,9 +1,9 @@
[Unit]
Description=toybox switchboard: the vats' only way out
Description=toybox switchboard: the public egress handle
[Socket]
ListenStream=/run/toybox/switchboard.sock
# only toybox may dial out: its service joins this group
ListenStream=/run/toybox/switchboard-public.sock
# only toybox holds this handle: its service joins this group
SocketGroup=toybox
SocketMode=0660
+2 -2
View File
@@ -4,9 +4,9 @@ Requires=toybox.socket toybox-switchboard.socket
After=toybox-switchboard.socket
[Service]
ExecStart=/opt/toybox/node_modules/workerd/bin/workerd serve --experimental /opt/toybox/config.capnp -S http=3 -S admin=4 -d state=/var/lib/toybox -e switchboard=unix:/run/toybox/switchboard.sock
ExecStart=/opt/toybox/node_modules/workerd/bin/workerd serve --experimental /opt/toybox/config.capnp -S http=3 -S admin=4 -d state=/var/lib/toybox -e switchboard=unix:/run/toybox/switchboard-public.sock
DynamicUser=yes
# to open the switchboard socket
# to open the public switchboard socket
SupplementaryGroups=toybox
StateDirectory=toybox
# no network but the sockets handed in, and the switchboard's
+10
View File
@@ -0,0 +1,10 @@
// @endo/init/debug.js, but with `overrideTaming: 'moderate'`: Node 24's
// undici assigns `message` to its errors (ClientDestroyedError when a
// WebSocket closes), which 'min' makes a TypeError.
import '@endo/init/pre-remoting.js';
lockdown({
errorTaming: 'unsafe',
overrideTaming: 'moderate',
domainTaming: 'unsafe',
});
+53
View File
@@ -0,0 +1,53 @@
/** @file The ingress's mount SQL, on node:sqlite: lookup, overlap, and mounts stored before mount_label. */
import { DatabaseSync } from "node:sqlite";
import { test } from "node:test";
import assert from "node:assert/strict";
import { LOOKUP, MOUNT, MOUNT_LABELS, MOUNT_SCHEMA, OVERLAP } from "../mounts-sql.js";
const makeDb = () => {
const db = new DatabaseSync(":memory:");
for (const statement of MOUNT_SCHEMA) db.exec(statement);
const mount = (vat, labels) => {
const json = JSON.stringify(labels);
db.prepare(MOUNT).run(vat, json);
db.prepare(MOUNT_LABELS).run(vat, json);
};
const lookup = (request) => db.prepare(LOOKUP).get(JSON.stringify(request))?.vat ?? null;
const overlaps = (vat, labels) => !!db.prepare(OVERLAP).get(JSON.stringify(labels), vat);
return { db, mount, lookup, overlaps };
};
test("a request reaches the mount whose every label it carries", () => {
const { mount, lookup } = makeDb();
mount("a", { vat: "alice", flavor: "plain" });
mount("b", { vat: "bob" });
assert.equal(lookup({ vat: "alice", flavor: "plain", page: "2" }), "a");
assert.equal(lookup({ vat: "alice" }), null);
assert.equal(lookup({ vat: "alice", flavor: "fancy" }), null);
assert.equal(lookup({ vat: "bob" }), "b");
assert.equal(lookup({}), null);
});
test("a mount with no labels matches any request", () => {
const { mount, lookup } = makeDb();
mount("only", {});
assert.equal(lookup({ anything: "x" }), "only");
});
test("overlap: a mount must conflict with every other", () => {
const { mount, overlaps } = makeDb();
mount("a", { vat: "alice" });
assert.equal(overlaps("b", { vat: "alice", n: "2" }), true);
assert.equal(overlaps("b", { app: "x" }), true);
assert.equal(overlaps("b", { vat: "bob" }), false);
assert.equal(overlaps("a", { vat: "alice", n: "2" }), false, "a vat's own mount doesn't count");
});
test("mounts stored before mount_label are filled in when the schema runs", () => {
const db = new DatabaseSync(":memory:");
db.exec("CREATE TABLE mount (vat TEXT PRIMARY KEY, labels TEXT)");
db.prepare("INSERT INTO mount VALUES (?, ?)").run("old", JSON.stringify({ vat: "old" }));
for (const statement of MOUNT_SCHEMA) db.exec(statement);
assert.equal(db.prepare(LOOKUP).get(JSON.stringify({ vat: "old" }))?.vat, "old");
assert.equal(db.prepare(LOOKUP).get(JSON.stringify({ vat: "other" })), undefined);
});
+105
View File
@@ -0,0 +1,105 @@
/** @file OCapN vats: a client on the admin socket creates a vat and passes it a remote object. */
import { mkdir, mkdtemp, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { after, before, test } from "node:test";
import assert from "node:assert/strict";
import { E, Far } from "@endo/far";
import { connectClient } from "../client.js";
import { freePort, here, startWorkerd } from "./workerd.js";
const vatSrc = (name) => readFile(join(here, "vats", name), "utf8");
let dir, port, server, client;
const startToybox = () =>
startWorkerd(
"config.capnp",
[
"-s", `http=127.0.0.1:${port}`, "-s", `admin=unix:${join(dir, "admin.sock")}`,
"-d", `state=${join(dir, "state")}`, "-e", `switchboard=unix:${join(dir, "no-switchboard.sock")}`,
],
{ port, host: "127.0.0.1" },
[join(dir, "admin.sock")],
);
before(async () => {
dir = await mkdtemp(join(tmpdir(), "toybox-ocapn-"));
await mkdir(join(dir, "state"));
port = await freePort();
server = await startToybox();
client = await connectClient(join(dir, "admin.sock"), "client-1", { verbose: !!process.env.OCAPN_LOG });
});
after(async () => {
client?.close();
await server?.stop();
await rm(dir, { recursive: true, force: true });
});
test("createVat passes a remote object in; the vat's root comes back", async () => {
const carol = Far("carol", { greet: (name) => `hi ${name}, from carol` });
const { id, root } = await E(client.vatAdmin).createVat(await vatSrc("greeter.js"), { carol });
assert.match(id, /^[0-9a-f]{64}$/);
assert.equal(await E(root).hello(), "hello from the vat");
assert.equal(await E(root).greetVia("vat"), "hi vat, from carol");
});
test("a counter vat counts; two counter vats count apart", async () => {
const src = await vatSrc("counter.js");
const { root: a } = await E(client.vatAdmin).createVat(src);
const { root: b } = await E(client.vatAdmin).createVat(src);
assert.equal(await E(a).increment(), 1);
assert.equal(await E(a).increment(), 2);
assert.equal(await E(b).increment(), 1);
assert.equal(await E(a).read(), 2);
assert.equal(await E(b).read(), 1);
});
test("a vat serves the web through the ingress it is given", async () => {
const ingress = await E(client.vatAdmin).makeIngress({ vat: "hello" });
await E(client.vatAdmin).createVat(await vatSrc("hello.js"), { ingress });
const resp = await fetch(`http://127.0.0.1:${port}/greet?vat=hello`);
assert.equal(resp.status, 200);
assert.equal(resp.headers.get("content-type"), "text/html; charset=utf-8");
assert.match(await resp.text(), /<h1>Hello, world<\/h1><p>from a toybox vat, at \/greet<\/p>/);
});
test("an ingress nobody serves is a 503", async () => {
await E(client.vatAdmin).makeIngress({ vat: "idle" });
assert.equal((await fetch(`http://127.0.0.1:${port}/?vat=idle`)).status, 503);
});
test("after a crash, a request restarts the vat that serves it", async () => {
// its own toybox: a crash breaks every session to it, the shared client's too
const home = await mkdtemp(join(tmpdir(), "toybox-phoenix-"));
await mkdir(join(home, "state"));
const port2 = await freePort();
const start = () =>
startWorkerd(
"config.capnp",
[
"-s", `http=127.0.0.1:${port2}`, "-s", `admin=unix:${join(home, "admin.sock")}`,
"-d", `state=${join(home, "state")}`, "-e", `switchboard=unix:${join(home, "no-switchboard.sock")}`,
],
{ port: port2, host: "127.0.0.1" },
[join(home, "admin.sock")],
);
let toybox = await start();
try {
const creator = await connectClient(join(home, "admin.sock"), "creator");
const ingress = await E(creator.vatAdmin).makeIngress({ vat: "phoenix" });
const { root } = await E(creator.vatAdmin).createVat(await vatSrc("hello.js"), { ingress });
assert.equal((await fetch(`http://127.0.0.1:${port2}/?vat=phoenix`)).status, 200);
await root; // its handoff, settled before the client goes
creator.close();
await toybox.stop();
toybox = await start();
const resp = await fetch(`http://127.0.0.1:${port2}/again?vat=phoenix`);
assert.equal(resp.status, 200);
assert.match(await resp.text(), /Hello, world.*at \/again/);
} finally {
await toybox.stop();
await rm(home, { recursive: true, force: true });
}
});
+179 -103
View File
@@ -1,118 +1,103 @@
/** @file toybox end to end: a real workerd, an admin socket, a public port, a switchboard. */
import { spawn } from "node:child_process";
/** @file toybox end to end: a real workerd, an admin socket, a public port, a switchboard with two handles. */
import { mkdir, mkdtemp, readFile, rm } from "node:fs/promises";
import { createServer as createHttpServer, request } from "node:http";
import { connect, createServer } from "node:net";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { after, before, test } from "node:test";
import assert from "node:assert/strict";
import { WebSocketServer } from "ws";
const here = new URL("..", import.meta.url).pathname;
// the package binary, not the .bin shim: the shim forks it, so a kill would orphan workerd
const workerd = join(here, "node_modules/workerd/bin/workerd");
import { freePort, here, startWorkerd } from "./workerd.js";
const app = (name) => readFile(join(here, "apps", name), "utf8");
const freePort = () =>
new Promise((resolve) => {
const s = createServer().listen(0, "127.0.0.1", () => {
const { port } = s.address();
s.close(() => resolve(port));
});
});
/** Resolves once something accepts connections at `opts` (a port or a socket path). */
const accepting = (opts) =>
new Promise((resolve) => {
const s = connect(opts, () => (s.end(), resolve(true)));
s.on("error", () => resolve(false));
});
/** @param {string} socketPath the admin socket; holding it is the authority to install */
const makeAdmin = (socketPath) => ({
install: (vat, src, egress) =>
/**
* The admin socket: holding it is the authority to create and upgrade vats.
*
* @param {string} socketPath
*/
const makeAdmin = (socketPath) => {
const send = (method, path, src, { egress, mount } = {}) =>
new Promise((resolve, reject) => {
const path = egress ? `/${vat}?egress=${encodeURIComponent(egress)}` : `/${vat}`;
const req = request({ socketPath, path, method: "PUT", agent: false }, (res) => {
const q = new URLSearchParams();
if (egress) q.set("egress", typeof egress === "string" ? egress : JSON.stringify(egress));
if (mount) q.set("mount", JSON.stringify(mount));
const req = request({ socketPath, path: `${path}?${q}`, method, agent: false }, (res) => {
let body = "";
res.on("data", (c) => (body += c));
res.on("end", () => resolve(body.trim()));
res.on("end", () => resolve(res.statusCode === 200 ? JSON.parse(body) : { status: res.statusCode, body }));
});
req.on("error", reject);
req.end(src);
}),
});
});
return {
/** `createVat`: a new vat, reachable only by its id here, and by its mount */
create: (src, grants) => send("POST", "/", src, grants),
upgrade: (vat, src, grants) => send("PUT", `/${vat}`, src, grants),
/** a vat from before ids */
byName: (name, src, grants) => send("PUT", `/by-name/${name}`, src, grants),
};
};
/** @param {string} base the public origin */
const makePublic = (base) => ({
get: async (path) => (await fetch(`${base}/${path}`)).text(),
put: async (vat, src) => (await fetch(`${base}/${vat}`, { method: "PUT", body: src })).status,
get: async (pathAndQuery, headers = {}) => {
const resp = await fetch(`${base}${pathAndQuery}`, { headers });
return { status: resp.status, text: await resp.text(), referrerPolicy: resp.headers.get("Referrer-Policy") };
},
text: async (pathAndQuery) => (await fetch(`${base}${pathAndQuery}`)).text(),
put: async (pathAndQuery, src) => (await fetch(`${base}${pathAndQuery}`, { method: "PUT", body: src })).status,
});
/**
* Run workerd on `config` until `stop()`; resolves once `listening` accepts.
*
* @param {string} config
* @param {string[]} args socket, directory, and external overrides
* @param {{ port: number } | { path: string }} listening
* @param {string[]} stale socket files a crash leaves behind; bind() would fail on them
*/
const startWorkerd = async (config, args, listening, stale = []) => {
for (const f of stale) await rm(f, { force: true });
const child = spawn(workerd, ["serve", "--experimental", join(here, config), ...args], {
stdio: ["ignore", "ignore", "pipe"],
});
let log = "";
child.stderr.on("data", (c) => (log += c));
const exited = new Promise((resolve) => child.on("exit", resolve));
for (let i = 0; i < 100; i++) {
// SIGKILL: a crash is the harder durability test
if (await accepting(listening)) return { stop: () => (child.kill("SIGKILL"), exited) };
await new Promise((r) => setTimeout(r, 50));
}
child.kill("SIGKILL");
throw new Error(`workerd did not start: ${log}`);
};
/** An HTTP server that answers GET with "plain ok" and echoes WebSocket messages. */
/** An HTTP server that answers GET with "plain ok", echoes WebSocket messages, and counts requests. */
const startEcho = async () => {
const server = createHttpServer((_req, res) => res.end("plain ok"));
let hits = 0;
const server = createHttpServer((_req, res) => (hits++, res.end("plain ok")));
const wss = new WebSocketServer({ server });
wss.on("connection", (ws) => ws.on("message", (m) => ws.send(String(m))));
await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
return {
host: `127.0.0.1:${server.address().port}`,
hits: () => hits,
stop: () => new Promise((resolve) => (wss.close(), server.closeAllConnections(), server.close(resolve))),
};
};
let dir, port, server, switchboard, echo, admin, pub;
let dir, port, server, switchboard, echo, echoLabels, admin, pub;
const startToybox = () =>
startWorkerd(
/**
* A toybox in `dir/<name>` that holds one switchboard handle.
*
* @param {string} name
* @param {number} port
* @param {"public" | "local"} handle which switchboard socket it gets
*/
const startToybox = async (name, port, handle) => {
const home = join(dir, name);
await mkdir(join(home, "state"), { recursive: true });
return startWorkerd(
"config.capnp",
[
"-s", `http=127.0.0.1:${port}`, "-s", `admin=unix:${join(dir, "admin.sock")}`,
"-d", `state=${join(dir, "state")}`, "-e", `switchboard=unix:${join(dir, "switchboard.sock")}`,
"-s", `http=127.0.0.1:${port}`, "-s", `admin=unix:${join(home, "admin.sock")}`,
"-d", `state=${join(home, "state")}`, "-e", `switchboard=unix:${join(dir, `sb-${handle}.sock`)}`,
],
{ port, host: "127.0.0.1" },
[join(dir, "admin.sock")],
[join(home, "admin.sock")],
);
};
before(async () => {
dir = await mkdtemp(join(tmpdir(), "toybox-"));
await mkdir(join(dir, "state"));
port = await freePort();
echo = await startEcho();
echoLabels = { host: "127.0.0.1", port: echo.host.split(":")[1] };
switchboard = await startWorkerd(
"switchboard.capnp",
["-s", `http=unix:${join(dir, "switchboard.sock")}`],
{ path: join(dir, "switchboard.sock") },
[join(dir, "switchboard.sock")],
["-s", `public=unix:${join(dir, "sb-public.sock")}`, "-s", `local=unix:${join(dir, "sb-local.sock")}`],
{ path: join(dir, "sb-local.sock") },
[join(dir, "sb-public.sock"), join(dir, "sb-local.sock")],
);
server = await startToybox();
admin = makeAdmin(join(dir, "admin.sock"));
// the tests' echo server is on localhost, so this toybox holds the local handle
server = await startToybox("main", port, "local");
admin = makeAdmin(join(dir, "main", "admin.sock"));
pub = makePublic(`http://127.0.0.1:${port}`);
});
@@ -123,68 +108,159 @@ after(async () => {
await rm(dir, { recursive: true, force: true });
});
test("a vat with no code is 404", async () => {
assert.equal(await pub.get("nobody"), "no code installed\n");
const at = (name) => ({ mount: { vat: name } });
test("nothing mounted: 404", async () => {
assert.equal((await pub.get("/?vat=nobody")).status, 404);
});
test("install, run, upgrade: the app's state carries over", async () => {
assert.match(await admin.install("alice", await app("counter-v1.js")), /^[0-9a-f]{64}$/);
assert.equal(await pub.get("alice"), "v1 count 1\n");
assert.equal(await pub.get("alice"), "v1 count 2\n");
await admin.install("alice", await app("counter-v2.js"));
assert.equal(await pub.get("alice"), "v2 count 3\n");
test("createVat returns an id and a code hash; unmounted, the vat is unreachable", async () => {
const { vat, hash } = await admin.create(await app("counter-v1.js"));
assert.match(vat, /^[0-9a-f]{64}$/);
assert.match(hash, /^[0-9a-f]{64}$/);
assert.equal((await pub.get("/")).status, 404);
});
test("install, run, upgrade: the app's state carries over; every start states its grants", async () => {
const { vat } = await admin.create(await app("counter-v1.js"), at("alice"));
assert.equal(await pub.text("/?vat=alice"), "v1 count 1\n");
assert.equal(await pub.text("/?vat=alice"), "v1 count 2\n");
await admin.upgrade(vat, await app("counter-v2.js"), at("alice"));
assert.equal(await pub.text("/?vat=alice"), "v2 count 3\n");
await admin.upgrade(vat, await app("counter-v2.js"));
assert.equal((await pub.get("/?vat=alice")).status, 404);
await admin.upgrade(vat, await app("counter-v2.js"), at("alice"));
assert.equal(await pub.text("/?vat=alice"), "v2 count 4\n");
});
test("vats don't share state", async () => {
await admin.install("carol", await app("counter-v1.js"));
assert.equal(await pub.get("carol"), "v1 count 1\n");
await admin.create(await app("counter-v1.js"), at("bob"));
await admin.create(await app("counter-v1.js"), at("carol"));
assert.equal(await pub.text("/?vat=bob"), "v1 count 1\n");
assert.equal(await pub.text("/?vat=carol"), "v1 count 1\n");
});
test("an app gets no fetch", async () => {
await admin.install("bob", await app("try-fetch.js"));
assert.match(await pub.get("bob"), /^fetch refused: This worker is not permitted to access the internet/);
await admin.create(await app("try-fetch.js"), at("bob-fetch"));
assert.match(await pub.text("/?vat=bob-fetch"), /^fetch refused: This worker is not permitted to access the internet/);
});
test("the public port can't install code", async () => {
assert.equal(await pub.put("mallory", "export class App {}"), 404);
assert.equal(await pub.get("mallory"), "no code installed\n");
assert.equal(await pub.put("/", "export class App {}"), 404);
});
test("code and state survive a restart", async () => {
await admin.install("dave", await app("counter-v1.js"));
assert.equal(await pub.get("dave"), "v1 count 1\n");
test("the path is the app's; its labels are the mount's and the request's together", async () => {
await admin.create(await app("show-labels.js"), at("lab"));
const got = await pub.get("/a/b?vat=lab&page=2");
assert.equal(got.text, `/a/b ${JSON.stringify({ vat: "lab", page: "2" })}\n`);
assert.equal(got.referrerPolicy, "no-referrer");
});
test("a label given twice is a 400", async () => {
assert.equal((await pub.get("/?vat=lab&vat=lab")).status, 400);
});
test("mounts that could both match a request are refused", async () => {
const src = await app("show-labels.js");
await admin.create(src, at("x"));
const narrower = await admin.create(src, { mount: { vat: "x", n: "2" } });
assert.equal(narrower.status, 409);
assert.match(narrower.body, /overlaps another/);
// no key in common with the {vat} mounts: ?vat=x&app=x would carry both
assert.equal((await admin.create(src, { mount: { app: "x" } })).status, 409);
assert.match((await admin.create(src, { mount: { vat: "y", n: "2" } })).vat, /^[0-9a-f]{64}$/);
});
test("a request reaches a vat only by naming every label of its mount", async () => {
await admin.create(await app("show-labels.js"), { mount: { vat: "both", flavor: "plain" } });
assert.equal((await pub.get("/?vat=both")).status, 404);
assert.equal((await pub.get("/?vat=both&flavor=fancy")).status, 404);
assert.equal((await pub.get("/?vat=both&flavor=plain")).status, 200);
});
test("routing reads no labels from headers", async () => {
const got = await pub.get("/?vat=both", { "Toybox-Labels": JSON.stringify({ flavor: "plain" }) });
assert.equal(got.status, 404);
});
test("code, state, and mount survive a restart", async () => {
await admin.create(await app("counter-v1.js"), at("dave"));
assert.equal(await pub.text("/?vat=dave"), "v1 count 1\n");
await server.stop();
server = await startToybox();
assert.equal(await pub.get("dave"), "v1 count 2\n");
server = await startToybox("main", port, "local");
assert.equal(await pub.text("/?vat=dave"), "v1 count 2\n");
});
const probe = (vat, target, ws) =>
pub.get(`${vat}?url=${encodeURIComponent(`http://${target}/`)}${ws ? "&ws=1" : ""}`);
test("a vat from before ids is unreachable until the admin mounts it by its old name", async () => {
await admin.byName("old", await app("counter-v1.js"));
assert.equal((await pub.get("/?vat=old")).status, 404);
await admin.byName("old", await app("counter-v1.js"), at("old"));
assert.equal(await pub.text("/?vat=old"), "v1 count 1\n");
});
const probe = (vat, target, { ws, constrain } = {}) =>
pub.text(
`/?vat=${vat}&url=${encodeURIComponent(`http://${target}/`)}` +
(ws ? "&ws=1" : "") +
(constrain ? `&constrain=${encodeURIComponent(JSON.stringify(constrain))}` : ""),
);
test("a vat granted a host reaches it through the switchboard: fetch and WebSocket", async () => {
await admin.install("erin", await app("egress-probe.js"), echo.host);
await admin.create(await app("egress-probe.js"), { ...at("erin"), egress: echoLabels });
assert.equal(await probe("erin", echo.host), "got: plain ok\n");
assert.equal(await probe("erin", echo.host, true), "echo: hello\n");
assert.equal(await probe("erin", echo.host, { ws: true }), "echo: hello\n");
});
test("same code, no grant: no egress", async () => {
await admin.install("frank", await app("egress-probe.js"));
await admin.create(await app("egress-probe.js"), at("frank"));
assert.equal(await probe("frank", echo.host), "no egress granted\n");
});
test("a grant is an object, not ambient: the global fetch still throws", async () => {
await admin.install("heidi", await app("try-fetch.js"), echo.host);
assert.match(await pub.get("heidi"), /^fetch refused: This worker is not permitted to access the internet/);
await admin.create(await app("try-fetch.js"), { ...at("heidi"), egress: echoLabels });
assert.match(await pub.text("/?vat=heidi"), /^fetch refused: This worker is not permitted to access the internet/);
});
test("a grant covers one host only", async () => {
await admin.install("grace", await app("egress-probe.js"), "127.0.0.1:1");
assert.equal(await probe("grace", echo.host), `got: egress to ${echo.host} not granted\n\n`);
assert.match(await probe("grace", echo.host, true), /^no websocket: 403 egress to .* not granted/);
test("the switchboard refuses what a grant doesn't cover", async () => {
await admin.create(await app("egress-probe.js"), { ...at("grace"), egress: { host: "127.0.0.1", port: "1" } });
assert.match(await probe("grace", echo.host), /^got: egress refused: .*egress to 127\.0\.0\.1 not granted/);
assert.match(await probe("grace", echo.host, { ws: true }), /^no websocket: 403 egress refused: .*not granted/);
});
test("a vat can narrow its grant, but not contradict it", async () => {
await admin.create(await app("egress-probe.js"), { ...at("ivan"), egress: { scheme: "http" } });
assert.equal(await probe("ivan", echo.host, { constrain: echoLabels }), "got: plain ok\n");
assert.match(
await probe("ivan", echo.host, { constrain: { scheme: "https" } }),
/^refused: label scheme is already set to a different value/,
);
});
test("a grant stored before labels, a bare host:port, still works", async () => {
await admin.create(await app("egress-probe.js"), { ...at("kim"), egress: echo.host });
assert.equal(await probe("kim", echo.host), "got: plain ok\n");
});
test("a grant survives a restart", async () => {
await server.stop();
server = await startToybox();
assert.equal(await probe("erin", echo.host, true), "echo: hello\n");
server = await startToybox("main", port, "local");
assert.equal(await probe("erin", echo.host, { ws: true }), "echo: hello\n");
});
test("a toybox holding only the public handle can't reach localhost, whatever the grant", async () => {
const port2 = await freePort();
const other = await startToybox("public-only", port2, "public");
try {
const admin2 = makeAdmin(join(dir, "public-only", "admin.sock"));
await admin2.create(await app("egress-probe.js"), { ...at("judy"), egress: echoLabels });
const before = echo.hits();
const got = await (
await fetch(`http://127.0.0.1:${port2}/?vat=judy&url=${encodeURIComponent(`http://${echo.host}/`)}`)
).text();
// workerd reports the public network's refusal (restrictPeers) to JS only as "internal error"
assert.match(got, /^got: egress refused: /);
assert.equal(echo.hits(), before);
} finally {
await other.stop();
}
});
+51
View File
@@ -0,0 +1,51 @@
/** @file Run real workerd processes for tests. */
import { spawn } from "node:child_process";
import { rm } from "node:fs/promises";
import { connect, createServer } from "node:net";
import { join } from "node:path";
export const here = new URL("..", import.meta.url).pathname;
// the package binary, not the .bin shim: the shim forks it, so a kill would orphan workerd
const workerd = join(here, "node_modules/workerd/bin/workerd");
export const freePort = () =>
new Promise((resolve) => {
const s = createServer().listen(0, "127.0.0.1", () => {
const { port } = s.address();
s.close(() => resolve(port));
});
});
/** Resolves once something accepts connections at `opts` (a port or a socket path). */
export const accepting = (opts) =>
new Promise((resolve) => {
const s = connect(opts, () => (s.end(), resolve(true)));
s.on("error", () => resolve(false));
});
/**
* Run workerd on `config` until `stop()`; resolves once `listening` accepts.
*
* @param {string} config
* @param {string[]} args socket, directory, and external overrides
* @param {{ port: number } | { path: string }} listening
* @param {string[]} stale socket files a crash leaves behind; bind() would fail on them
*/
export const startWorkerd = async (config, args, listening, stale = []) => {
for (const f of stale) await rm(f, { force: true });
const child = spawn(workerd, ["serve", "--experimental", join(here, config), ...args], {
stdio: ["ignore", process.env.WORKERD_LOG ? "inherit" : "ignore", "pipe"],
});
let log = "";
// WORKERD_LOG=1 echoes workerd's stderr, for debugging
child.stderr.on("data", (c) => (process.env.WORKERD_LOG ? process.stderr.write(c) : (log += c)));
const exited = new Promise((resolve) => child.on("exit", resolve));
for (let i = 0; i < 100; i++) {
// SIGKILL: a crash is the harder durability test
if (await accepting(listening)) return { stop: () => (child.kill("SIGKILL"), exited) };
await new Promise((r) => setTimeout(r, 50));
}
child.kill("SIGKILL");
throw new Error(`workerd did not start: ${log}`);
};
+59
View File
@@ -0,0 +1,59 @@
/**
* @file The runtime of an OCapN vat: SES, an OCapN peer, and the vat's own
* module (`vat.js`), which exports `buildRootObject(vatPowers, params)`.
*
* Bundled (see build.js) and loaded as the facet's main module beside the
* vat's code. The vat's designator is its id; it reaches other peers, and
* they reach it, through the vatAdmin's router (`env.router`).
*/
import "@endo/init";
import { DurableObject } from "cloudflare:workers";
import { E, Far } from "@endo/far";
import { makeClient } from "@endo/ocapn";
import { makeRpcNetLayer } from "./netlayer.js";
import * as vat from "./vat.js";
/**
* The swissnum of a vat's bootstrap: the vatAdmin starts the vat through it.
* Not exported: workerd takes every export of a main module for an entrypoint.
*/
const VAT_SWISSNUM = "vat";
export class App extends DurableObject {
#gateway;
constructor(ctx, env) {
super(ctx, env);
this.#gateway = this.#boot();
}
async #boot() {
const { vatId, router } = this.env;
let root;
const vatPowers = Object.freeze({ E, Far, storage: this.ctx.storage.kv });
const bootstrap = Far("vatBootstrap", {
/** (Re)build the root: on creation, and when the vatAdmin restarts the vat. */
start: (params) => {
root = vat.buildRootObject(vatPowers, params);
return root;
},
});
// workerd runs FinalizationRegistry callbacks outside any request, where
// sending the resulting GC message is forbidden I/O. So imports are
// never collected here; they go when their session does.
const ocapn = makeClient({
debugLabel: vatId.slice(0, 8),
swissnumTable: new Map([[VAT_SWISSNUM, bootstrap]]),
enableImportCollection: false,
});
const netlayer = await ocapn.registerNetlayer(
makeRpcNetLayer({ designator: vatId, openGateway: (designator) => router.gateway(designator) }),
);
return netlayer.gateway;
}
/** This vat's OCapN gateway, for the router. */
gateway() {
return this.#gateway;
}
}
+15
View File
@@ -0,0 +1,15 @@
/**
* A counter whose count lives in the vat's storage.
*
* TODO: a durable Exo once vatPowers offers a zone, e.g.
* `zone.exoClass("Counter", undefined, () => ({ count: 0 }), { incr() { this.state.count += 1; return this.state.count; } })`
*/
export const buildRootObject = ({ Far, storage }) =>
Far("counterRoot", {
increment: () => {
const n = (storage.get("count") ?? 0) + 1;
storage.put("count", n);
return n;
},
read: () => storage.get("count") ?? 0,
});
+6
View File
@@ -0,0 +1,6 @@
/** A vat that greets through the `carol` it was created with. */
export const buildRootObject = ({ E, Far }, { carol }) =>
Far("greeterRoot", {
greetVia: (name) => E(carol).greet(name),
hello: () => "hello from the vat",
});
+12
View File
@@ -0,0 +1,12 @@
/** Hello, world, served through the `ingress` it was created with. */
export const buildRootObject = ({ E, Far }, { ingress }) => {
const handler = Far("helloHandler", {
handle: ({ path }) => ({
status: 200,
headers: { "content-type": "text/html; charset=utf-8" },
body: `<!doctype html><title>hello</title><h1>Hello, world</h1><p>from a toybox vat, at ${path}</p>\n`,
}),
});
E(ingress).serve(handler);
return Far("helloRoot", {});
};