WIP: toybox: OCapN vats on workerd, with ingress and egress as capabilities #4

2026-10-14T02:45:22Z

dckc commented

2026-10-14 02:45:22 +00:00

toybox: vats whose code is installed at runtime, on workerd, after Agoric's vatAdmin.createVat(bundle). Base: main.

Vats: each vat is a Durable Object whose app runs as a facet on code loaded through the Worker Loader. An OCapN vat runs SES and @endo/ocapn in its facet around a module that exports buildRootObject(vatPowers, params).
vatAdmin: an OCapN peer (OCapN over Cap'n Web) on the admin socket. createVat(src, params) takes remote objects in params and returns the vat's root; passing them is an OCapN handoff, so the vatAdmin also routes connections between peers, which can't dial out.
Ingress is a capability: makeIngress(labels) reserves a mount; the vat it is passed to serves requests carrying those labels, as data. Routing reads labels from the query string only; a mount must conflict with every other (Flower's rule), and the lookup is SQL.
Egress is a capability too: a separate switchboard process, after Flower, serves Cap'n Web handles with fixed labels per socket (public, local); constrain(labels) only narrows. A vat gets its grant as an object in env; its global fetch throws.
systemd: workerd takes its sockets from socket activation (-S name=fd), so holding a socket is the authority it grants; units are in toybox/systemd/, not yet run under the system manager.

Try it: cd toybox && pnpm install && pnpm test (30 tests, real workerd processes), or pnpm start and pnpm hello, then visit http://127.0.0.1:8787/?vat=hello.

Known gaps:

A vat is restartable only if its params hold data and ingresses; another vat's or a client's objects need sturdyrefs (below).
enableImportCollection: false in the workerd peers: workerd runs FinalizationRegistry callbacks outside a request, where sending op:gc is forbidden.
@endo/ocapn doesn't break crossed hellos; one of two simultaneous sessions fails, harmlessly so far.
App vats (export class App, HTTP admin API) remain beside OCapN vats, to be removed.

To do:

carol the counter: the hello page counts visitors with a counter vat passed in as a remote object (createVat(helloSrc, { ingress, counter })). That works until the first eviction, since counter is another vat's object and the hello vat's params can't be stored. Needs sturdyrefs between vats: the vatAdmin stores counter as (vat id, swissnum), and the counter vat gives its root a swissnum that survives its own restart (the durable-Exo TODO in vats/counter.js is one way). Test: the count continues across a crash.

🤖 Generated with Claude Code

toybox: vats whose code is installed at runtime, on workerd, after Agoric's `vatAdmin.createVat(bundle)`. Base: `main`. - **Vats:** each vat is a Durable Object whose app runs as a [facet](https://developers.cloudflare.com/dynamic-workers/usage/durable-object-facets/) on code loaded through the Worker Loader. An OCapN vat runs SES and `@endo/ocapn` in its facet around a module that exports `buildRootObject(vatPowers, params)`. - **vatAdmin:** an OCapN peer (OCapN over Cap'n Web) on the admin socket. `createVat(src, params)` takes remote objects in `params` and returns the vat's `root`; passing them is an OCapN handoff, so the vatAdmin also routes connections between peers, which can't dial out. - **Ingress is a capability:** `makeIngress(labels)` reserves a mount; the vat it is passed to serves requests carrying those labels, as data. Routing reads labels from the query string only; a mount must conflict with every other (Flower's rule), and the lookup is SQL. - **Egress is a capability too:** a separate switchboard process, after [Flower](https://github.com/NuxiNL/flower), serves Cap'n Web handles with fixed labels per socket (`public`, `local`); `constrain(labels)` only narrows. A vat gets its grant as an object in `env`; its global `fetch` throws. - **systemd:** workerd takes its sockets from socket activation (`-S name=fd`), so holding a socket is the authority it grants; units are in `toybox/systemd/`, not yet run under the system manager. Try it: `cd toybox && pnpm install && pnpm test` (30 tests, real workerd processes), or `pnpm start` and `pnpm hello`, then visit `http://127.0.0.1:8787/?vat=hello`. Known gaps: - A vat is restartable only if its `params` hold data and ingresses; another vat's or a client's objects need sturdyrefs (below). - `enableImportCollection: false` in the workerd peers: workerd runs FinalizationRegistry callbacks outside a request, where sending `op:gc` is forbidden. - `@endo/ocapn` doesn't break crossed hellos; one of two simultaneous sessions fails, harmlessly so far. - App vats (`export class App`, HTTP admin API) remain beside OCapN vats, to be removed. To do: - [ ] **carol the counter:** the hello page counts visitors with a counter vat passed in as a remote object (`createVat(helloSrc, { ingress, counter })`). That works until the first eviction, since `counter` is another vat's object and the hello vat's `params` can't be stored. Needs sturdyrefs between vats: the vatAdmin stores `counter` as `(vat id, swissnum)`, and the counter vat gives its root a swissnum that survives its own restart (the durable-Exo TODO in `vats/counter.js` is one way). Test: the count continues across a crash. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dckc added 13 commits 2026-10-14 02:45:22 +00:00

capsite/toybox: vats with code installed at runtime, on workerd 88739d9495

Each vat is a Durable Object that stores code by hash and runs it as a
facet through the Worker Loader: an empty env, no fetch, storage that
carries over upgrades. Two sockets with no configured address: whoever
starts workerd hands them in (systemd socket activation), so the admin
socket is the authority to install code.

Tests run a real workerd: install, upgrade, isolation between vats, no
fetch, the public port can't install, and code and state survive a crash.
They spawn the package binary rather than the .bin shim, which forks
workerd, so killing the shim orphaned it and the test run never ended.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: a switchboard, the vats' only way out 7bf6eb44c7

A vat granted a host (PUT /<vat>?egress=<host> on the admin socket) gets
Egress as its globalOutbound, with the host in ctx.props where the vat
can't change it. Egress forwards matching requests, WebSocket upgrades
included, to the switchboard and refuses the rest with a 403; with no
grant, fetch still throws.

The switchboard is a separate workerd: a Switchboard DO that calls fetch
and returns the response, so a 101's webSocket passes through. toybox
reaches it over a unix socket in HTTP proxy style, which keeps the full
URL, scheme included. Its network service is the outer fence, with TLS
trusting the browser CAs; systemd units for it join toybox's, and only
the toybox group can open its socket.

The loaded Worker is now named by vat and incarnation, not by code hash:
the same code with a different grant must not share a Worker. The new
incarnation table sits beside the old ones; vats installed before it
read as incarnation 0 with no grant.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: a grant is an object in env, not the vat's global fetch 2820c015ee

A vat's globalOutbound is now always null. A granted host arrives as
env.egress, an Egress the app can hand to whichever of its parts should
have it; as globalOutbound it was ambient to every module in the vat.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: egress through Flower-style switchboard handles f0557d29d6

The switchboard now serves a Cap'n Web Handle on each of its sockets:
public (no loopback or private addresses) and local. A socket's labels
are fixed in config, so holding the socket's fd is holding that handle;
constrain(labels) returns a narrower one, as Flower's Constrain returns
a narrower fd. Labels only accumulate: a missing key matches anything,
and a key both sides set must agree (labels.js, shared by both).

The switchboard enforces the labels; toybox's Egress no longer checks
URLs itself. It opens a session to its one switchboard socket, constrains
the handle to the vat's grant, and forwards through it; a WebSocket
crosses the session as a pair of message streams. A vat narrows its own
grant with env.egress.constrain().

Grants are now labels as JSON. One stored before, a bare host[:port],
reads as {host, port}.

pnpm start and the systemd units give toybox the public handle; the
switchboard unit binds the local one in its own 0700 runtime directory.
The tests give their toybox the local handle to reach an echo server,
and check that a toybox holding only the public one never reaches it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: vats have ids, not names; ingress by mount 8d8900d84d

createVat (POST / on the admin socket) makes a vat with newUniqueId and
returns its id, which addresses it only there. A vat is reachable only
where it is mounted: ?mount=<labels> on each start, as ?egress= is
granted; absent means none.

The ingress reads a request's labels from its query string, plus a
Toybox-Labels header for secret ones (a URL ends up in logs and
Referers). It routes to the vat whose mount labels the request carries,
every one: stricter than Flower's no-conflict match, since an ingress
client holds no constrained handle and a label it omits must not match.
Registration keeps Flower's rule, so a mount must conflict with every
other and no request can carry two. The app keeps the whole path and
sees its connection labels in Toybox-Connection-Labels.

A vat from before ids lives at idFromName(name). PUT /by-name/<name>
reaches it on the admin socket; it stays unreachable from outside until
mounted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: OCapN vats; createVat takes remote objects 9356f63c64

The admin socket now also speaks OCapN over Cap'n Web: a WebSocket there
reaches the VatAdmin DO, an OCapN peer whose bootstrap (swissnum
vatAdmin) offers createVat(src, params). params may hold remote objects;
the vat's root comes back as one.

Each OCapN vat is its own peer: SES and @endo/ocapn run in its facet
(vat-runtime.js, bundled with esbuild), around the vat's own module,
which exports buildRootObject(vatPowers, params). Passing a client's
object to a vat, or a vat's root to the client, is an OCapN handoff, so
the receiver dials the exporter. Nothing here can dial out, so the
VatAdmin is also the router: a client registers its gateway under a
designator, a vat's designator is its id, and every peer reaches every
other through it. The netlayer is ertp-clerk's, made to work over Workers
RPC stubs and plain promises as well as Cap'n Web.

Not yet: sturdyrefs that survive a restart, an adminNode, and grants
(egress, mount) for OCapN vats. App vats (export class App) still work
beside them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: mount lookups in SQL, not a scan in JS 7fcd251261

Each request parsed every mount's labels and searched them in JS. Labels
now sit one per row in mount_label, indexed on (key, value): a lookup
joins the request's labels (json_each) on the index and keeps the vat
whose every label matched. The overlap check at registration is SQL too.

mount.labels keeps the whole set as JSON; mounts stored before
mount_label get their rows filled in when the schema runs. The SQL lives
in mounts-sql.js, tested on node:sqlite, old-shape state included.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: no encoder object in module scope 28b1c3c9d3

A module-level TextEncoder is a mutable object every importer shares.
Keep only its encode method, bound and hardened; the encoder itself is
unreachable. The code hash uses it too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: routing carries no secrets 193a126d1c

The ingress no longer reads labels from a Toybox-Labels header. A header
was no place for the webkey pattern: browsers can't set one on a
WebSocket, and the pattern puts the secret in-band anyway. A page keeps
it in its fragment, reaches the vat by public labels, and sends it in
the body of a call, which the vat checks like a swissnum.

A request still reaches a vat only by naming every label of its mount.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: an OCapN counter vat; no import GC in workerd peers f64dac4434

vats/counter.js keeps its count in the vat's storage; the test checks
that it counts and that two counter vats count apart.

Running it after the greeter test hung: a FinalizationRegistry callback
in the vatAdmin, collecting an import, sent op:gc over the client's
WebSocket, and workerd forbids I/O outside a request. The vatAdmin's and
the vats' OCapN clients now run with enableImportCollection: false, so
imports go when their session does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: TODO: the counter vat as a durable Exo 8025ec4b20

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: ingress is a capability; a hello-world web vat a984461cee

E(vatAdmin).makeIngress(labels) reserves a mount and returns an ingress;
whoever holds it serves requests carrying those labels by calling
E(ingress).serve(handler). A vat gets one in its params, like any other
remote object; nothing in a vat's module is an entrypoint. A request
reaches the handler as passable data (method, path, labels, body), and
the response comes back as data (status, headers, body). The handler is
a live reference for now: after a restart its mount answers 503.

vats/hello.js serves hello, world; pnpm hello creates it on a running
toybox (client.js is the Node OCapN client, shared with the tests).

pnpm start and pnpm switchboard never ran: systemd-socket-activate wants
absolute paths for unix sockets.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

capsite/toybox: a mount outlives its handler: the vat restarts on demand e5ed407e55

A handler is a live reference, held by the VatAdmin DO; workerd evicts
an idle DO within seconds, and with it the handler, so a vat's page went
503 almost at once.

createVat now keeps a vat's params when they hold only data and
ingresses (an ingress by its key: the VatAdmin can make it again), and
which vat was given which ingress. A request for a mount with no handler
restarts that vat with its params; it rebuilds its root, serves again,
and the request is answered. A vat whose params hold another peer's
objects isn't restartable yet.

The test crashes its own toybox (a crash ends every session to it), and
the creator waits for the root's handoff before it disconnects, as
scripts/hello.js now does too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

You are not authorized to merge this pull request.

This pull request can be merged automatically.

This pull request is marked as a work in progress.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin toybox:toybox

git checkout toybox

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dckc/captoys#4