valis / Understanding the system
valis - Architecture
What valis is
valis is the userland process the founding doctrine calls "a Lisp Machine for network protocols." Linux's eBPF makes one IP address answer on every TCP/IP port and forwards the traffic to userland; valis is the process that receives it and multiplexes a single person's data and identity across the protocols they choose to run.
valis is not a static "one machine, one person" install. It is a host-agnostic, migratable service: a person's valis can move between valis-compatible hosts anywhere on the internet, because the thing that is durable is not the process but the person's namespace. The architecture reifies Plan 9 semantics (per-process namespaces, everything-as-a-file, network-transparent resources) with the 9P protocol as intrinsic substrate rather than one protocol plugin among many. Migration, and the ability to run a person's protocol modules across more than one machine, fall out of that namespace model.
The central abstraction: every port a protocol, every protocol a plugin
The protocol surface is organised around one idea:
A protocol is a plugin. It declares the ports it answers on and knows how to handle a connection by integrating, through common interfaces, with the user's own data and identity.
This is the seed in src/:
src/protocol.lisp: theprotocolclass and thehandle-connectiongeneric function every plugin specialises. The contract every protocol plugin implements. (A long-lived protocol may instead be driven from the event loop through session callbacks and define nohandle-connectionat all; see the execution model.)src/registry.lisp: the port → protocol registry and the dispatch entry point. A connection arrives carrying its original destination port; the registry routes it to the protocol that claimed that port.src/main.lisp: process lifecycle (run / daemon / dev).
The architecture is the moat: not any single protocol, but the substrate that makes every protocol a uniform, user-owned plugin.
One caveat rides with this framing, because it is the model a reader most easily over-reads: "it declares the ports it answers on" governs dispatch (which protocol handles a connection that arrives), not reachability, whether the internet may reach that port at all. On a deployed host those are two different decisions owned in two different places, and a port's public reachability is a privileged decision at the host agent, never a consequence of a plugin declaring it. The public surface settles which ports the internet reaches and who opens them.
The keystone: the 9P namespace fabric
The durable unit is the per-process namespace: a mount table of 9P resources naming the person's data, identity, names, and the mailboxes of their running protocol modules. A protocol module is an ephemeral viewer onto that namespace, not the owner of any irreplaceable state.
Organising the system this way buys three otherwise-hard properties from a single idea:
- Migration is cheap. A module holds no state that cannot be rebuilt by re-attaching to the namespace, so moving a person's valis to another host is closer to "restart there and re-mount" than to "serialise a live process."
- Sovereignty is enforceable. Content lives in the namespace; operations only ever move the viewer. The substrate and lifecycle are operable without the operator ever touching the content and meaning of the data.
- Distribution is uniform. A module on another machine is just another 9P-reachable resource in the namespace: local threads and remote Lisp nodes are addressed the same way.
Everything below hangs off this keystone.
Data flow: how a packet reaches a protocol module
outside world
│ packets to <user-ip>:<any-port>
▼
┌──────────────────────────────────────────────────────────┐
│ privileged host agent, external to valis │
│ │
│ binds the privileged ports attaches the eBPF │
│ (:53 udp+tcp, :443, :80, sk_lookup catchall: │
│ the owner terminus) every port to one socket │
└─────────┬──────────────────────────────┬─────────────────┘
│ inherited │ steered
│ (passed in at exec) ▼
│ ┌──────────────┐
│ │ Linux/eBPF │
│ └──────┬───────┘
▼ ▼
┌──────────────────────────────────────────────────────────┐
│ valis process (unprivileged) │
│ listener: one socket, many ports │
│ │ accept; recover original destination port │
│ ▼ │
│ registry: port ──▶ protocol module │
│ │ dispatched over 9P │
│ ▼ (local thread or remote node) │
│ the user's substrate │
│ (namespace: data · identity · names · crypto) │
└──────────────────────────────────────────────────────────┘
⚠ What one picture flattens: two arrival paths, at different maturities. The inherited path on the left is what carries public traffic today, and it is the older of the two. The steered path is live: on a node whose catchall is attached it takes every TCP port at the unit's address, DNS over TCP included, and the two DNS arrival paths sets out what that costs. A node declares the set of ports it serves and they all arrive through the single steered socket, so the shape the drawing describes is the shape the code has. Both arrivals converge on the same listener, which is why they are drawn together; arrival paths treats the distinction properly.
The steering layer: sklookup (validated)
The eBPF mechanism that makes "one IP answers on every port" concrete is
BPF_PROG_TYPE_SK_LOOKUP (Linux 5.9+; kernel docs): a program that runs during the
kernel's listener lookup for every inbound connection, before port
demultiplexing. It calls bpf_sk_assign to bind the connection to one
chosen socket regardless of destination port. That socket is valis's
single listener, registered in a BPF_MAP_TYPE_SOCKMAP; the program is
attached to a network namespace via a BPF link.
This is the real steering primitive, and it had to be established by
investigation. An earlier proof-of-concept that hooked the
inet_sock_set_state tracepoint only observed connections: it could not
redirect them. sklookup is the program type that actually answers for
every port.
Written in Lisp: the substrate stays introspectable
The steering program is compiled by Whistler, a Common Lisp s-expression → eBPF → ELF compiler with a pure-CL loader (no libbpf, no clang, no bpftool). Keeping the kernel-facing layer in Lisp is the Lisp-Machine thesis applied to the substrate itself: the owner reads and reshapes the steering logic in the same language as everything above it.
sklookup support and a catch-all redirector were validated end to end on Linux 7.0: compiled, accepted by the kernel verifier, and attached to a network namespace through the pure-CL loader. Minimal shape:
(defmap redir-sockmap :type :sockmap :key-size 4 :value-size 8 :max-entries 1)
(defprog catch-all (:type :sk-lookup :section "sk_lookup" :license "GPL")
(when-let ((sk (map-lookup redir-sockmap 0)))
(sk-assign (ctx-ptr) sk 0)
(sk-release sk)) ; mandatory: see invariant below
SK_PASS)
Invariants worth remembering:
- A sockmap lookup in sklookup context returns a reference-counted
socket; the program must
sk-releaseit before exit or the verifier rejects it with "unreleased reference." - Two kernel constants that are easy to confuse: program type
SK_LOOKUP= 30, attach typeBPF_SK_LOOKUP= 36.
The Whistler additions are upstream (atgreen/whistler#43), so a stock Whistler
carries them.
The listener: one socket, many ports
A connection steered by bpf_sk_assign keeps its original four-tuple, so
the original destination port (the port the client dialed) is
recoverable in userland with getsockname on the accepted socket, with no
cooperation from the eBPF program. The recovery works identically whether
the socket arrived by sklookup or by an ordinary bind, and it is the
ordinary-bind case that keeps the userland path testable over loopback with
no kernel in the loop. On its own that arrangement would leave the sklookup
case argued rather than shown. It is shown:
proving-ground/driver/scenarios/steered-port-delivery.lisp puts an off-host
client on the far side of a real steer, and the resident dispatches each
connection on the port the client dialed, including one port that appears in
no configuration on either side.
The listener is an event-loop multiplexer, not a synchronous accept
loop. It is built on iolib's multiplexer over epoll; the event source is
kept behind valis's own interface so an io_uring backend can be slotted
in later without changing any protocol plugin. (iouring is deliberately
out of the first cut: it has no mature Common Lisp support today.) Each
accepted connection has its original destination port recovered and is
handed to the registry, which dispatches it to the owning protocol module.
The execution model: one loop, one seam, and how a protocol is driven
Everything in this section follows from a single constraint: the event-dispatch thread must never block. Stalling inside a readiness callback stalls every other source on the loop, so any protocol that can block must be run off it.
src/listener.lisp holds the one backend-selection point:
*default-multiplexer-factory* names the epoll multiplexer, and nothing else in
the tree chooses a backend. Swapping in a different one is a change to that
variable, not a change to any protocol.
src/executor.lisp is the indirection that keeps the loop free. The executor's
dispatch-connection generic schedules an accepted connection onto whatever
execution context the executor manages, and returns at once.
Three strategies exist in the tree, and which one a port gets is a property of the transport rather than of the protocol:
- Every bound edge port builds a budget executor. It is a thread per accepted connection with a per-port concurrent-connection cap: the cap-th accept deregisters the port's listening source, so further connections wait in the kernel backlog rather than being accepted and closed, and a slot freeing re-registers it. The default cap is 64 connections per port.
- The connectionless datagram path takes an inline executor, which spawns nothing and runs the exchange on the loop thread itself. A datagram arrives whole, so there is nothing to block on, and spawning per datagram would hand an off-path packet flood an unbounded thread-creation lever.
- A bare thread per connection, with no budget, is the third and is what the other two are variations on.
All three sit behind the one generic, so a pooled or coroutine executor is a further subclass and the swap touches neither the multiplexer nor any protocol handler.
A protocol may instead declare :raw-descriptor stream discipline, in which case
its connections are driven from the loop itself through the session callbacks
rather than handed to the executor. This is what makes long-lived protocols
affordable: a connected but idle session costs no thread at all, so concurrency
scales with the number of active clients and never with the number of connected
ones. Sizing for such a protocol is therefore against a rate of activity rather
than a count of peers. The connection-lifetime assumptions measure that
claim, at zero OS threads and a few kilobytes retained per idle session.
Two boundaries govern that declaration:
- An adapter that wraps or terminates the stream cannot honestly declare raw discipline, because the descriptor beneath it carries ciphertext while the wrapper holds the bytes the protocol wants. A terminating edge therefore keeps the thread shape. Driving a continuous protocol over a terminated port from the loop is unresolved: termination hands back a blocking stream bound to a thread, so such a session costs a thread even while idle. That question gates continuous protocols over TLS, and owner mail access with them.
A protocol that needs to make blocking calls while serving a session, such as a namespace round trip, must still get off the loop to make them. Connection dispatch cannot carry those workers. Its unit is a whole connection lifetime: it is keyed on a port an adopted session has already resolved, it reaches the protocol through the registry's
handle-connection, which a session-driven protocol no longer defines, and where a strategy owns that lifetime it closes the connection when the work returns. A per-session worker runs many times over one session, so the first one to return would tear the session down.Neither entry point covers work that is not a whole connection lifetime, such as a per-session worker or a timer body. A task-level dispatch that takes a thunk, owns no lifecycle and closes nothing is owed here; connection dispatch is expressible on top of it. Until it exists, each protocol spawns its own threads, and the choice between unbounded workers and a bounded pool is settled independently in each one. The two are not interchangeable: a pool smaller than the simultaneously-active set couples clients whose latency is independent today.
Arrival paths: how a wire descriptor reaches a node
Wire descriptors reach a running node two ways. The distinction is in how a descriptor arrives, never between kinds of node:
Inherited. The privileged host agent binds a port outside the node and passes the descriptor in. The reason is privilege rather than protocol: valis runs unprivileged in its namespace, so a port it may not bind is bound for it and handed down before privilege is dropped. ⚠ These are not the ports the node designates. The set a node declares and the set the agent hands down are disjoint, and the agent's firewall reads neither of them the way a reader expects: it reads the owner port and the edge port from its own config. Calling an inherited port a designated one put those two sets under one word once before, and a privileged-agent check was then written against the wrong one and refused the only value that worked for three weeks.
Four descriptors arrive this way, and the node reads exactly four flags for them (
src/main.lisp)::443for the public TLS edge,:53on TCP and:53on UDP for authoritative DNS, and the routable owner terminus on whatever port the agent's config names. Three are TCP streams, and the steer is perfectly capable of carrying them. Only a UDP reply must source from the socket the client addressed, which is why UDP is the one transport that has to be inherited to work at all; the two DNS arrival paths sets that out, along with what it costs that:53is split across both arrivals.- Not inherited, though a reader expects it to be:
:80, for the documentation every node is meant to serve in the clear. No descriptor arrives for it and no flag carries it. It is a port number like any other inside the node, so it reaches the node by declaration: name it in the set and a socket is minted and pushed for it exactly as for any declared port, with no privileged bind and nothing crossing the boundary. What ANSWERS there is a separate question and the code has not yet reached it. Minting a socket for a port is not the same act as having an adapter claim it, and today nothing claims:80. What:80is for states the design; TODO.org carries where the code stands. - Declared and pushed. The node mints a listening descriptor and hands a duplicate to the privileged agent over the existing control connection, which is what lets the agent steer every port onto that one socket. The push direction is the property that makes the arrangement safe, and the steered-fd handoff sets out why. Note separately, at the per-namespace steering constraint, that a declaration does not by itself bound the answered set on a routable address: the agent's default-deny firewall is what narrows it.
These compose: a node inherits the descriptors the agent bound for it and declares ports of its own, and a launch may use either arrival, both, or neither. A boot declares a set of ports and mints ONE descriptor for all of them, because the steer fans every declared port at the socket it was handed and the dialed port is recovered from each accepted client. Declaring stays default-closed, so a launch that only inherits behaves as it always did.
They did not compose until recently. The boot shape that adopts inherited :53
descriptors never declared its socket to the agent, so on a node booted that way
nothing entered the steering map and no other port was reachable regardless of
what the firewall admitted. That was a defect in the code rather than a second
kind of node.
The invariant the fix must restore: authoritative DNS is a service in the :names
group and nothing more. It is not a mode the node runs in, and its presence must
never suppress the general arrival path for other protocols. What that protects is
a node routing an arbitrary number of protocols, the surfaces at the edge/core seam
and others as they are adopted. Naming none of the others here is deliberate: the
model must not need to know them. If adding a protocol requires a new branch in the boot path,
the model has been drawn wrongly and the exception has been allowed to suppress
the rule.
Identity, capability, and the namespace boundary
The connection-open moment is an authentication event. When bytes arrive on the IP, valis must transparently establish who is calling, because the answer decides which namespace is mounted for that connection. The same port serves different worlds by identity: an anonymous request to the web port sees the published-files view; the owner, proven by key on that same port, sees the management view or the operator interface entirely.
This is a full capability system (authority can be minted, attenuated, and delegated to external actors who hold no identity of their own), not a yes/no on a fixed identity. Every primitive is established prior art; the combination is what is new. It composes as layers:
- Transport identity. A Noise-style handshake authenticates the channel to a peer public key (a decentralized identifier), autonegotiated and invisible by default, yielding a principal, or anonymous for a keyless public request. (Noise is also one of libp2p's secure channels, so it serves when valis speaks IPFS.)
- An auth agent, factotum-shaped. Following Plan 9's
factotum(man page), an auth agent that is itself a 9P file system, holding the owner's keys, negotiating all security interactions for other programs, and moderating which identity applies, valis keeps all key handling in one agent. Protocol modules never see keys. - Capability tokens. Authority is carried as delegable, attenuatable, public-key-rooted capabilities in the UCAN mould, where the owner is the authorization server directly: no central server, offline-verifiable (which matters under migration). Delegation chains double as provenance.
- Capabilities as names. Published data is named by a capability that embeds its own access right and key, in the manner of Tahoe-LAFS read/write/verify caps and capability URLs. Handing out the name is the grant; the recipient needs no identity of their own. This is how anonymous external actors are authorized.
- Enforcement by mounting. The resolved authority decides what is mounted into the connection's namespace. A module can only name what is mounted; sovereignty is the absence of a name, set once at namespace construction rather than checked on every access.
- No ambient authority. A module acts only on capabilities designated to it through its namespace, never on a global identity. This is the discipline that prevents the confused deputy: a semi-trusted, possibly remote module being induced to wield the owner's authority on an attacker's behalf.
Authority resolves in two phases. Key-based transport auth fixes the principal at connect time; but a capability presented inside the protocol (an unguessable URL path, say) is known only after the request is parsed. So a connect-time principal (often anonymous) establishes a base namespace, and a per-request capability may mount further within it.
The NoiseXX handshake
The connect-time authenticator for a 9P-native peer is a Noise Protocol Framework handshake in the NoiseXX25519ChaChaPolySHA256 pattern. XX is mutually authenticating without pre-shared knowledge: neither party needs the other's static public key in advance, and both recover the other's key during the exchange. That symmetry is what lets a fresh peer dial a fresh valis and walk away with a did:key for each other: no directory, no out-of-band setup.
The exchange is three messages and binds every byte into the same transcript. The initiator opens with an ephemeral X25519 public key in the clear. The responder answers with its own ephemeral and its static public key, the static encrypted under a key derived from the running transcript; the message ends with an authenticator (an AEAD of the empty string under the same key) that proves the responder also derived the same transcript. The initiator's third message carries its static the same way, again followed by an authenticator. Each derivation step folds the latest DH output into the chaining key, and every emitted byte is folded into the transcript hash, so a single bit-flip anywhere in the conversation breaks the next decryption. After the third message a Split derives two asymmetric transport CipherStates (one for each direction) that a post-handshake transport layer would use; valis's current use ends at admission and does not exercise transport-mode encryption.
Mutual authentication here means a key recovery, not a key challenge: the responder reads the initiator's static out of the third message and publishes it to the rest of the system as a principal. There is no separate "prove you hold this key" step, because the AEAD authenticator on the message a party encrypted with the keys derived from its own static is the proof; impersonating a key requires possessing it.
The cipher behind every encrypted token is RFC 8439 AEADCHACHA20POLY1305 with the IETF 96-bit nonce shape (four zero bytes followed by the counter as little-endian uint64) and the running transcript hash as additional data. The AEAD wrapper has its own RFC 8439 §2.8.2 known-answer test, so the cryptographic primitive is anchored before the protocol that uses it.
The state machine and seven-compatible auth node are in
src/identity/noise.lisp; the AEAD wrapper is in src/identity/aead.lisp.
See the Noise Protocol Framework entry in the Bibliography.
The factotum auth-conversation
The factotum is mounted at /id and is itself a 9P file tree, faithful in
shape to Plan 9's factotum(4) (src/namespace/id.lisp). It exposes three
children:
/id/ctl- key management: a Tread renders the owner's key descriptors, a Twrite carries control verbs. The interface deliberately mirrors factotum(4) so an operator (or a local module) drives it the way they would drive Plan 9's./id/rpc- the auth-conversation endpoint. Every open of/id/rpcmints a fresh conversation object holding its own auth node, so two concurrent callers cannot share state through it. The rpc surface is a generic auth-protocol multiplexer, not Noise-specific: its first message may carry a proto-selection line, and the endpoint really dispatches on it, building the Noise responder fornoiseor the challenge driver forssh-reality, withnoisethe default when a caller begins its handshake without selecting. It is the factotum's mechanism for driving whichever proof the caller wants, and the seam a further authenticator plugs into by becoming one more dispatch arm./id/proto- a list of authenticator names available behind/id/rpc, one per line. It now advertises bothnoiseandssh-reality. An architect or an auditor reads it to see which proofs this image admits without opening source.
The same machinery serves a remote 9P peer through seven's auth seam. A
Tauth against the root binds the afid to a fresh Noise auth node by way
of the node-auth method on valis-root (src/namespace/root.lisp).
Subsequent Twrite and Tread operations on that afid pump the three XX
messages through seven's "dumb pipe" auth carrier: seven never inspects
the bytes; it carries them between the peer and the node, and parks reads
until a write completes so the challenge-response ordering is correct.
When the third message lands and the node-auth predicate
(node-auth-ok-p) reports completion, seven calls
node-attach-identity on valis-root with the approved auth node;
the specialiser returns the keyed principal the Noise responder
resolved from the peer's static public key, and seven records that
return verbatim on the live 9P session before admitting the Tattach.
Two paths (a remote afid and a local open of /id/rpc) exercise the
same Noise auth node class. The symmetry is the whole point: a local
module driving the factotum and a remote peer attaching through it
authenticate against the same code.
See the factotum(4) entry in the Bibliography.
Principal resolution and the authenticator seam
Once a proof completes, it has to become something the rest of valis can
read. That conversion is a single named operation: the authenticate
generic function in src/identity/authenticator.lisp. Everything above
the seam (the connection layer, the capability layer, the namespace
mounter) only ever sees a principal; only the code below the seam
knows what proof produced it.
NoiseXX is the first method on this seam (noise-authenticator);
anonymous-authenticator is the second. Adding a future proof (an
HTTP-edge signature authenticator, for example) means writing a new
class that specialises authenticate to mint the same principal type
from a different proof. No code above the seam changes. The Noise
implementation does not need to be read or understood to add a new
authenticator; the seam is the entire contract.
The principal value type itself lives in src/identity/principal.lisp
and carries a did:key identifier and the raw public key bytes that
identifier encodes. For a Noise admission, the responder recovers the
peer's static public key from the third message, encodes it as a
did:key (multicodec 0xec01 for X25519, base58btc per the did:key
method spec, in src/identity/did-key.lisp), and constructs the
principal. At Tattach the principal lands on the live 9P session
through seven's node-attach-identity seam: valis-root's specialiser
returns it; seven stores it verbatim on the session's identity slot.
Backend code reads it back through current-identity (seven) or the
thin wrapper current-principal (src/namespace/root.lisp) for the
dynamic extent of request dispatch. A standard file node's node-read
runs synchronously on the session thread, so the principal is in scope
during node-open / node-read / node-write; a backend that parks
reads off-thread (the bus pattern) captures it at the synchronous call
site and closes over it for delivery.
For a keyless attach (an Tattach that asks no key, e.g. a NOFID
Tattach), the seam binds a distinct anonymous sentinel, never nil.
The distinction is load-bearing: nil is the unresolved state before
admission, the sentinel is the resolved state for "no key presented".
Module code can therefore test anonymous-p without conflating "not yet
authenticated" with "authenticated as nobody in particular". The
sentinel carries the literal DID string "anonymous" so it is safe to
log and dispatch on; the boundary that no key material reaches a
module is enforced on its principal-public-key slot, which is
nil. The anonymous principal is autonegotiated by the absence of a
handshake, never produced by an error path and never a login prompt.
See the W3C Decentralized Identifiers and Noise Protocol Framework entries in the Bibliography.
Key custody
The custody store in src/identity/custody.lisp is the sole holder of
private key material in the running image. The owner's master key is an
Ed25519 signing pair; the X25519 transport key Noise uses is derived from
it deterministically, so one keyfile roots both the owner's signatures
and the owner's channels. The master key is held in memory and persisted
to a mode-0600 keyfile so the owner's did:key is stable across restarts:
an in-memory-only key would mint a fresh owner DID on every boot and
break every capability rooted in it.
Real at-rest encryption (passphrase / OS keyring / HSM) belongs to a
later phase when real steering and the IETF edge land; the dev keyfile
is sufficient for the loopback substrate.
A node's identity is its did:key, derived from its genesis Ed25519 root. The
did:key, the libp2p peer ID and the iroh NodeId are three encodings of the same
Ed25519 public key. Each network's name is computed where that network is spoken,
and is never stored as the node's identity, used as a capability audience, or
signed over by a vouch in place of the did:key. I keep the identity in no single
network's encoding because binding it to one would trap the others: iroh is the
native network, and nothing may preclude IPFS.
A protocol module receives only a resolved principal: a did:key
identifier and a public key. It never calls into the custody store, never
holds a private key object, and never has a name through which it could
ask for one. This boundary is the key-custody enforcement point: the
absence of any path from a module to a private key. The custody store is
wired into the factotum's node-auth path by start-fabric
(src/fabric.lisp), and stop-fabric clears it; an architect who needs
to convince themselves the boundary is intact can read those two
lifecycle hooks and confirm that nothing else is ever handed a custody
reference.
The seam this creates around the rest of the system is the answer to the "confused deputy" risk that the capability discipline above is designed to head off: a module asked to act on a peer's behalf cannot be tricked into reaching for the owner's key, because it has no name for it.
Validation coverage
Each phase success criterion is anchored to a named test in
tests/identity-test.lisp that proves it. An auditor reading this
document can run the suite and see the claim row stand or fail without
reconstructing the coverage from source.
| Criterion | Test | File |
|---|---|---|
| NoiseXX interop: both parties derive identical transport keys | noise-xx-two-party-interop |
tests/identity-test.lisp |
| AEAD: RFC 8439 §2.8.2 test vector encrypt/decrypt | aead-encrypt-rfc-vector, aead-decrypt-rfc-vector, aead-tampered-tag-fails |
tests/identity-test.lisp |
| did:key: round-trip identity for a 32-byte X25519 key | did-key-round-trip, did-key-z6ls-prefix |
tests/identity-test.lisp |
| Noise init: h and ck are the raw protocol name bytes | noise-init-h-ck-known-answer |
tests/identity-test.lisp |
| Keyed attach: principal recorded on the 9P session, DID matches peer key, readable via current-identity | noise-handshake-end-to-end-over-seven, node-attach-identity-keyed-returns-noise-principal |
tests/identity-test.lisp |
| Keyless attach: anonymous sentinel recorded on the session, readable via current-identity, no error | anonymous-attach-nofid, node-attach-identity-nofid-returns-anonymous |
tests/identity-test.lisp |
Factotum reachability: /id tree walkable, /id/proto lists noise |
factotum-9p-reachability |
tests/identity-test.lisp |
| Key stability: owner DID unchanged after keyfile reload | key-stability-across-restart |
tests/identity-test.lisp |
| Seam decoupling: principal package imports no crypto symbols | principal-seam-decoupling-structural |
tests/identity-test.lisp |
One validation limitation is recorded here rather than papered over. The AEAD
primitive carries its RFC 8439 §2.8.2 known-answer vector, but the NoiseXX
handshake above it is anchored only by an initialisation known-answer
(noise-init-h-ck-known-answer) and a two-party self-interop
(noise-xx-two-party-interop) in which both parties are valis: that proves the
two sides agree with each other, not that they conform to the wider Noise
ecosystem. The handshake state machine is a from-scratch implementation of a
standard protocol, so before the authenticator is trusted in production it should
be checked against independent published NoiseXX vectors (Cacophony / noise-c)
and given a short cryptographic review; a bug shared by both sides would pass
self-interop silently. This weighs more heavily as the same state machine becomes
the basis other consumers in the constellation interoperate with.
Wrapping the post-attach session
The NoiseXX handshake authenticates who is calling, but the 9P session it
admits still ran in the clear: every Tread=/=Twrite=/=Tclunk after Tattach
crossed the wire unencrypted, authenticated only by the handshake that preceded
it. That leaves a transport man-in-the-middle free to read or rewrite a peer's
file traffic once admission is past. Closing that gap means every post-attach
message (request and reply) is encrypted and authenticated under the Noise
transport keys, and a tampered byte is refused.
The split that makes this auditable is the design's defining choice: all
transport crypto belongs to mercer; valis is pure plumbing. mercer, the
validated auth agent, owns the transport session: the keys, the AEAD, the
two per-direction monotone counters, the nonce construction (the sequence is
folded into the nonce and never appears on the wire), the orientation baked in
by role, and fail-closed exhaustion. valis never touches any of it. valis owns
only the keyed accept loop (it binds the socket, accepts each connection, wraps
the connected stream in a secure endpoint, and hands that endpoint to the serve
thread) and a thin delegating endpoint that holds nothing but an opaque
session handle. Its read-frame calls the session's unseal; its write-frame
calls the session's seal; it performs no cryptography of its own. The whole
question "is the transport crypto correct?" is therefore answered once, inside
mercer; valis's audit reduces to "does the plumbing route every post-attach
frame through the session, and never leak the handle?"
The session contract (seal, unseal, close-session, and a
transport-session-error condition family) is deliberately provider-neutral.
It is the general secure-transport seam, not a Noise interface. NoiseXX is its
first provider; a TLS provider for the IETF HTTP edges and ACME-issued certs is
the intended second, and the plumbing consumes the abstract contract so that
edge reuses it unchanged. Only one operation, obtaining the session from a
completed handshake, is Noise-specific, and it lives at the attach boundary
where valis installs the handle.
On the wire, each post-attach record is a cleartext 4-byte length prefix
outside the sealed body, followed by the sealed ciphertext-and-tag. The length
sits outside the seal so the reader can delimit a record before it has any key
to unseal it; the AEAD tag is computed over the exact ciphertext, so any tamper
at the record boundary fails authentication (which is why an empty additional-
data field is safe here). The 9P engine keeps its own size[4] message framing
above this layer, untouched: the wrap is a transparent record cipher beneath
the protocol, not a change to it.
The pre/post-attach flip is valis-owned plumbing. The handshake, Tauth, and
Tattach must cross in the clear: there is no session yet to seal them. valis
installs the handle at the attach boundary (node-attach-identity in
src/namespace/root.lisp), so the very first sealed record is the Rattach
reply; from there every frame is delegated. Tamper is terminal: when unseal
signals transport-auth-failed, valis tears the session down. There is no
in-band rekey and no renegotiation: an anomaly ends the session rather than
recovering it.
This is the same architecture Plan 9's 9front uses for post-auth styx
encryption: factotum holds the keys and yields an opaque secret, devtls
applies a record cipher (seal/open) under it, and exportfs speaks plain 9P
over that cipher without knowing it is there. The mapping is one-to-one onto
mercer (the keys and cipher), valis's plumbing (the record framing and the
endpoint substitution), and seven (the cipher-agnostic 9P engine). The
9front-validated rules carry over verbatim: a cleartext record header outside
the sealed body, a monotone per-direction sequence folded into the nonce and
never on the wire, no in-band rekey, and interposition done by substituting
the endpoint in the serve path.
The thin delegating endpoint is in src/transport/secure-endpoint.lisp; the
keyed accept loop is in src/transport/keyed-acceptor.lisp; the attach-boundary
install is in src/namespace/root.lisp. The transport session itself (every
byte of the crypto) lives in mercer at mercer/src/transport.
Validation coverage: transport wrap
Crypto correctness is mercer's, not valis's: the transport session's ciphertext
is anchored byte-for-byte against mercer's own round-trip and the published
Cacophony / noise-c vectors under mercer 0.3.0. valis's tests prove the
plumbing: that the production keyed path routes every post-attach frame
through seal/unseal, flips the pre/post boundary at the Rattach, refuses and
tears down on tamper, frames records with the cleartext length prefix, closes
and never exposes the opaque handle, round-trips orientation both ways, and
holds the session cap and drain. Each row is anchored to a named test in
tests/transport-wrap-test.lisp, and each test declares in its own docstring
whether it observes the real production effect (DIRECT) or an equivalent
property the production path guarantees where the live serve thread offers no
external observation seam (PROXY), so an auditor sees exactly which dimension is
proven how.
| Criterion | Test | File |
|---|---|---|
| Confidentiality on the keyed path: a post-attach frame on the wire is ciphertext, via a real keyed attach (DIRECT) | confidentiality-on-keyed-path |
tests/transport-wrap-test.lisp |
| Tamper-reject and teardown: a flipped post-attach byte fails authentication and tears the session down, no plaintext (DIRECT arm + PROXY arm) | tamper-rejected-and-torn-down |
tests/transport-wrap-test.lisp |
| Pre/post-attach boundary: handshake/Tauth/Tattach cleartext, the first post-attach frame delegated (DIRECT) | pre-post-attach-boundary |
tests/transport-wrap-test.lisp |
| Wire framing: the cleartext length prefix delimits the record outside the seal; a boundary tamper fails the tag (DIRECT) | wire-framing-length-prefix |
tests/transport-wrap-test.lisp |
| Handle lifetime: the opaque session is closed on teardown and exposed by no public accessor (PROXY) | handle-lifetime |
tests/transport-wrap-test.lisp |
| Orientation cross-check: what one side seals the other unseals, both directions, pinned to mercer's round-trip anchor (DIRECT) | orientation-cross-check |
tests/transport-wrap-test.lisp |
| Accept-loop parity: the session cap and drain behave as the prior listener's, adding no DoS surface (DIRECT) | accept-loop-parity |
tests/transport-wrap-test.lisp |
Validation coverage: over-the-wire conformance
The transport wrap above proves valis seals every post-attach frame, but it proves it against valis's own initiator talking to valis's own responder, both sides our code. A bug shared symmetrically by both would pass that test silently. The remaining question is conformance: does the handshake valis admits, and the transport cipher it then runs, match an externally-defined standard rather than merely agreeing with itself? That question is answered by anchoring valis's real accept loop to a published NoiseXX transcript (bytes authored by the spec ecosystem, not by us) and confirming the wire reproduces them. Because the handshake valis sends carries no payload, the only published transcript its wire can byte-match is the empty-payload NoiseXX25519ChaChaPolySHA256 vector, dual- referenced (noise-c and an independent reference) and vendored under mercer. The replay fixes all four handshake keys to the vector's values so the wire is determined by the transcript, not by agreement between two pieces of our code.
Each criterion below rides the genuine production path (start-fabric →
keyed-acceptor → node-attach-identity, the same loop a real peer reaches), so
the proof comes off the real socket, never a direct endpoint call. The named
tests live in tests/over-the-wire-test.lisp.
| Criterion | Test | File |
|---|---|---|
| Handshake conformance: on the real keyed wire, the handshake messages and the responder's final handshake hash match the published empty-payload NoiseXX vector byte-for-byte (DIRECT) | published-vector-rides-the-real-wire |
tests/over-the-wire-test.lisp |
First sealed record: the first post-attach record off the wire unseals over the client session to a valid Rattach, proving the real seal path carries a sealed reply end-to-end (DIRECT) |
published-vector-rides-the-real-wire |
tests/over-the-wire-test.lisp |
| Transport-cipher conformance: the session derived from the real-wire handshake, sealing the vector's transport payload at the first responder send, reproduces the vector's transport ciphertext byte-for-byte, so the transport-key derivation conforms, not just the handshake (PROXY, tied to the wire by the shared session) | published-vector-rides-the-real-wire |
tests/over-the-wire-test.lisp |
Principal-scoped sealed read: over the sealed session a Twalk=+=Tread of /id/owner returns the authenticated owner DID and the principal has not collapsed to anonymous: the initiator reaches its keyed view, not a shared one (DIRECT) |
sealed-read-resolves-owner-did |
tests/over-the-wire-test.lisp |
One verification here is out-of-band rather than automated: an end-to-end wrapped attach against a real, non-loopback peer. The in-process and loopback dimensions all have automated in-suite coverage above; only driving a wrapped attach across a real network interface (which needs a remote peer and mount privileges the test image does not hold) is a manual test the operator runs, confirming the post-attach frames are ciphertext on the wire. The runbook below makes that check repeatable.
Runbook: a real-NIC sealed attach (manual, operator-run)
This is the one out-of-band confirmation the architecture asks for: a sealed 9P attach driven across a real network interface, from a peer holding mount privileges, observed to be ciphertext on the wire and to reach the caller's own principal-scoped view. The automated suite proves confidentiality and conformance over an ephemeral loopback socket; loopback cannot stand in for a real NIC, and a remote peer with mount privileges is not driveable from the Lisp image, so the operator runs this once, by hand, to close it. Running it successfully discharges the previously-open out-of-band wrapped-attach check.
The procedure reuses the test image's existing VALIS_TEST_HOST opt-in: the
real-NIC drive stays skipped by default and engages only when the operator points
that variable at the machine's actual non-loopback address, so the manual run
never fires accidentally in an ordinary suite pass.
Steps:
- Bring up a responder on the real interface. On the responder host, start
valis in the foreground (
valis run). It binds the 9P fabric on an ephemeral port and printsvalis: 9P fabric on port N; noteNand the host's non-loopback address. Leave it running. It tears down cleanly onSIGINT. - Opt the initiator in to the real NIC. On the peer that will attach, set
VALIS_TEST_HOSTto the responder's non-loopback address (not127.0.0.1/::1; a loopback value leaves the drive skipped by design). This is the single switch that turns the otherwise-skipped real-NIC drive on. - Drive a sealed attach from the remote peer. Using the reusable sealing
initiator under
tests/support/(the same harness the automated conformance tests drive), complete the keyed handshake (Tauth→ the NoiseXX exchange over the afid →Tattach) againstVALIS_TEST_HOST:=N=, then promote the connection to the sealed client session and issue aTwalk=+=Treadof/id/owner. - Observe the two outcomes that matter. First, capture the post-
Tattachtraffic on the wire (e.g. with a packet capture on the interface) and confirm it is ciphertext: no cleartext 9P frame, no readableRattach, no readable file payload. Second, confirm the sealed/id/ownerread returns the attaching peer's own owner DID, the initiator reaching its authenticated principal- scoped view across the real network, not a shared or anonymous one.
A clean run (ciphertext on the wire plus the correct principal-scoped read off a real interface) is the out-of-band confirmation the architecture names, and completing it once discharges the open real-NIC sealed-attach item.
Capability tokens: delegation and confinement
A capability token is the delegable form of authority. It carries the
issuer's DID (the source of the authority), the audience's DID (the
grantee), a list of mount grants (each a namespace designation with a
rights set), an expiry, a nonce, and an Ed25519 signature by the issuer
over a deterministic encoding of those fields. A delegated token carries
its ancestry inline: the proof chain back to the owner's root token rides
inside the token itself, so a verifier needs no external store to walk it.
That self-containment is what makes verification offline: a property
that matters under migration, when the owner's substrate may be mid-move
and no authorization server is reachable. The value type lives in
src/capability/token.lisp; the chain walk in
src/capability/verifier.lisp.
Verification walks the chain from leaf to root. The root token must be issued by the owner and signed by the owner's Ed25519 key; every token in the chain must be unexpired and absent from the revocation store. Each delegation link is then held to four conditions: the child's rights must be a subset of the parent's, the child's designation must be a path extension of the parent's, the parent must itself carry the delegate right, and the child must not outlive its parent. Its expiry may equal the parent's but never exceed it. The third condition deliberately diverges from stock UCAN: re-delegation is a structural property of the chain, not a policy judgment made at verification time, so a grantee who was not given the delegate right cannot mint valid sub-grants at all. The fourth (expiry attenuation) is what keeps a delegate that legitimately holds the delegate right from wielding it to escape its own lifetime: because equality is the loosest a child's expiry may be, a delegate cannot mint itself a longer-lived token and outlast the grant it descends from. Extending a lifetime is therefore not a delegation act at all; it takes a fresh grant minted at the root, which only the owner can produce. Together with the issuer-to-audience continuity check at each hop, this closes the confused-deputy route through token forwarding: a chain that escalates rights, escapes its designation prefix, lengthens its own life, or passes through a non-delegable hop fails verification with a stated reason.
Both verification entry points are fail-closed: every condition raised during decoding or checking is caught inside the verifier and returned to the caller as a refusal with a reason string, never as an escaped condition. The 9P layer that fronts verification can therefore treat the verifier as total.
Capability names: bearer grants and the self-certifying name
Where a token grants authority to a principal, a capability name grants
authority to its holder. A name encodes a designation, a rights
bitmask, an expiry, and a nonce, followed by the owner's Ed25519
signature over those bytes, the whole rendered as base58btc behind the
valis: URI prefix: URL-safe, so the same string can later ride in an
HTTP path. The codec is src/capability/name.lisp; verification is the
same src/capability/verifier.lisp described above.
The name is self-certifying in the Tahoe-LAFS sense: verifying it
requires only the owner's public key and the revocation store: no
directory, no central service, no identity for the presenter. Handing out
the string is the grant. This is how anonymous external actors are
authorized without ever acquiring an identity: possession designates.
The two costs of bearer semantics are answered structurally. Replay
within the validity window is bounded by mandatory expiry: a name
without a positive expiry cannot be minted. And the revocation store is
the kill switch: every verification consults it, for names exactly as for
tokens, so a leaked name dies the moment the owner records its hash,
before expiry, not at it. The store itself
(src/capability/revocation.lisp) is an append-only set of canonical
hashes persisted beside the owner's keyfile; revocation is irreversible
by construction. Tahoe's further trick (embedding an encryption key in
the name so the storage layer never sees plaintext) is deliberately
deferred until the durable store exists for it to protect.
Capabilities as mount grants
Every verified presentation (token or name) resolves to the same
internal authority value: the mount directive
(src/capability/directive.lisp), a designation in the owner's canonical
namespace plus a closed rights set (read, write, mount, delegate). A
mount directive is an assembly instruction: it tells the namespace
assembler which 9P-served subtree to bind into the requester's session
view, with which rights. Recipients never interpret the designation
string; they find the subtree present (or absent) in their assembled
view. Authorization is thereby enforced by what is mounted, not checked
per access: out-of-scope objects are not denied, they are unnameable.
The per-session view is the view-root
(src/namespace/view-root.lisp), an attach root whose children come from
a per-instance mount table. The binding happens at the attach moment:
a bearer name presented as the 9P attach name is verified and the session
receives a view containing exactly the designated subtree; a name that
fails verification admits the session to an empty view: fail closed,
never the shared tree. A plain attach continues to bind the development
tree; the policy for what base namespace a keyed versus anonymous session
receives there, and the per-request second phase that mounts further
capabilities into a live view mid-session, is the namespace-assembly
layer's work and lands with it.
Coverage over time is intrinsic to live mounts. A mount is a live 9P binding (walks reach the serving node at access time), so new children appearing under a granted subtree are covered the moment they exist, with no reissue traffic. The converse is also deliberate: new rights verbs are never implicitly covered, because the rights set is closed and a bitmask from an older grant simply has no bit for a verb that did not exist when it was minted.
The capability service itself is 9P-addressable at /cap
(src/namespace/cap.lisp), mirroring the factotum discipline: a ctl
file mints tokens and names and records revocations (signing always
delegated to the custody store: key material never enters the
capability layer), and a verify file turns a presented token or name
into a verification verdict. Modules interact with authority through
file operations, never through direct calls.
Owner-client trust: two privilege tiers and proof-of-reality enrolment
An owner does not always stand at the host. The durable way to operate a deployed node is through an owner-facing client (ubik) that attaches over the sealed 9P terminus from somewhere else entirely, and that never witnessed the node's genesis. The problem this section settles is how such a client comes to hold authority without the owner seed ever moving to it. The answer is two privilege tiers, each anchored to a proof matched to what it can do, and both expressed as ordinary revocable, key-bound capability delegations, never a copy of the root secret.
| Tier | Authority | Anchor to obtain it | Carrier |
|---|---|---|---|
| Operator-equivalent | unrestricted; can destroy the instance | proof of reality (real host control) | a maximal, revocable OCAP grant bound to the client's key |
| Scoped client | read /mail, edit one content axis, … |
ordinary OCAP delegation over the terminus | an attenuated, revocable OCAP token |
The anchor is matched to consequence. Operator-equivalence can destroy everything, so it demands the strongest gate a genesis-less client can be held to: a demonstration of real host control, strictly stronger than mere network reach. Scoped grants are bounded and revocable, so they ride the ordinary capability path with no host-access requirement; forcing proof-of-reality on every client action would defeat the point of a delegable capability system. Crucially, neither tier moves the owner seed. Even operator-equivalent access is a delegated, key-bound, revocable capability, which is exactly what lets a client be operator-equivalent without becoming an unrevocable clone of the owner key.
Tier 2 is already the capability layer above at work: an owner (or an
operator-equivalent client bearing the delegate right) mints an attenuated
token via /cap ctl (audience the client's own DID, a specific mount
grant, a short expiry), and the client presents it over the terminus, where
the assembler binds its audience to the Noise-authenticated session
principal and verifies the chain to the owner root. No host access, no new
mechanism; the trust model simply uses it.
The operator-equivalent grant shape
Operator-equivalence is expressed as a single maximal grant, not a union
of control-surface designations. It is one owner-custody-signed capability
token whose audience is the client's self-generated OCAP DID, whose
designation is the namespace root / (which prefix-covers every subtree,
so the grant is maximal by reach rather than by enumerating axes that
would have to be maintained as new ones appear) and whose rights are the
full closed set :read :write :mount :delegate. The issuer is the owner
custody root, never the client: this is a root-minted grant, never a
client→client maximal self-mint. It carries :delegate deliberately, so an
operator-equivalent client can itself mint Tier-2 grants for other clients
without a fresh proof-of-reality each time: the proof is a one-time
bootstrap of the tier, not a per-action gate.
A grant that can destroy the instance is contained not by scoping it down but by two structural limits on its lifetime and its progeny:
- Expiry attenuation. A child token can never outlive its parent (the
fourth chain condition above). So even though the grant carries
:delegateand thechild ⊆ parentrule permits an equal-authority re-issue, an operator-equivalent client cannot self-extend: any token it mints for itself expires no later than the Tier-1 grant it descends from. Renewing the tier is not a delegation act: it requires a fresh root grant through the proof-of-reality gate, which only the owner can drive. - Cascade revocation. Every descendant chains through the Tier-1 grant to the owner root, and the verifier consults the revocation store for every token in a chain. So revoking the single Tier-1 grant hash kills it and every token minted beneath it at once: one hash retires the whole subtree of authority the enrolment created.
Its lifetime is moderate and revocation-primary: expiry is the backstop, revoking the hash is the real containment lever. (Scoped Tier-2 grants invert this: short-lived and expiry-primary, because re-issue from an operator-equivalent client or the owner is cheap.) The blast radius is thus bounded by revocability and TTL, never by pretending a scoped grant is operator-equivalent when it is not.
The proof-of-reality gate: the ssh-reality authenticator
Minting the Tier-1 grant is authorized by a demonstration of real host
control, delivered as a first-class /id/proto authenticator named
ssh-reality, driven behind /id/rpc exactly as noise is. It is the
native place for a proof-of-reality: the driver lives in valis's /id
factotum, and on a satisfied proof it mints the grant through the shared
/cap mint path: the privileged host agent grows no auth-mint policy of
its own.
The gate's defence in depth is structural, and it requires both of two independent anchors. Either alone is refused, and the driver weighs each on its own so a diagnostic can always name which one failed rather than returning a single undiscriminating boolean:
- Host presence. The enrolment material must carry the un-forgeable
provenance stamp that only fulcrum's root-only,
AF_LOCAL, never-network-reachable host-local intake can apply. A terminus-facing path can fabricate an arbitrary stand-in object, but not that stamp, so this is a structural binding, not a checked field: a leaked operator SSH key presented over the network can never satisfy it. - Pinned-key reality. The operator must sign valis's issued challenge with the one pinned Ed25519 operator key, and that SSHSIG must verify (by mercer) against the pinned key and a fixed provisioning namespace.
valis mints a single-use, time-bounded challenge nonce for each attempt;
the operator signs it out of band with ssh-keygen -Y sign; mercer verifies
the SSHSIG blob; and only when both anchors hold does the driver return a
satisfied result carrying the client's bootstrap public key, from which the
audience DID of the minted grant is derived. Because host presence and a
valid operator signature are both required, a stolen SSH key alone mints
nothing remotely, and a host-access holder without the pinned key mints
nothing either: there is no second network-reachable path to the owner
surface. The client's bootstrap keypair is one-time: its only job is proving
live possession during the challenge, and it is spent once the grant is
minted to the client's durable OCAP DID.
A fully off-host owner (one who cannot demonstrate host control at all)
is out of scope for this authenticator; that case is served later by a
separate pairing-code /id/proto authenticator, and the SSH path is not
weakened to accommodate it. ssh-reality is one proof plugged into the
seam, not the only one it will ever carry.
Where the trust material rests
The outboard trust material for this path (the pinned Ed25519 operator key, the fixed provisioning namespace every proof is checked against, and the enrolled-client records [each an audience DID paired with its minted grant's revocation hash, kept for cascade-revocation lookup]) lives in its own cache directory beside the owner root seed, under the instance state root, and is never merged into the owner keyfile. Own directory, own lifecycle: this is outboard client trust, not the root secret, co-located with the root authority only for locality. The pin is host-local config the operator sets out of band and that is never mutated over the network. Every lifetime the gate honours (the nonce window, the Tier-1 and Tier-2 token TTLs) is a named, config-tunable parameter, not a magic number buried in a check; the defaults are deliberately generous for development and the initial go-live, to be tightened once the live client interaction is observed.
Threat model and dispositions
The dispositions this design commits to, recorded so an auditor can hold the code to them:
| Threat | Disposition |
|---|---|
| Pure-network attacker at the terminus mints operator access | mitigated: Tier 1 requires host control, not reachability |
| Captured token replayed by a third party | mitigated: audience is bound to the Noise-authenticated session principal |
| Enrolment message replayed to enrol a rogue key later | mitigated: a single-use, time-bounded challenge nonce bound to the specific bootstrap key |
| Malicious client build enrols an attacker's key | mitigated: the operator drives the delivery and confirms the key fingerprint |
| Operator-equivalent grant becomes an unrevocable owner clone | mitigated: it is a revocable, key-bound OCAP token, never the seed |
| A second network-reachable auth path to the owner surface | mitigated: the enrolment intake is host-local, uid/root-gated, off the public surface |
| Leaked operator SSH key alone mints operator-equivalence | mitigated: the host-presence anchor is structural; a key over the network cannot stamp host provenance |
| Operator-equivalent client self-extends its own lifetime | mitigated: expiry attenuation caps a child at its parent; renewal needs a fresh root grant |
| Host compromise | accepted: already total compromise (the seed is on the box); anchoring Tier 1 on host access adds nothing to lose |
| Client dials a spoofed node | mitigated (client-side) by the server-DID pin; the server-side lever is exposure-gating |
Two residuals are named rather than papered over. The trust of a genesis-less client bottoms out at one owner-controlled vouch: here, host access; the design only makes that "once" require public material, reuse an existing anchor, and yield a revocable delegate. And explicit revocation is best-effort, so short token expiry is the real containment lever: bound the blast radius by lifetime, and re-issue.
Owner key rotation
The owner's identity method is did:key, which has no update or
deactivate operation: the DID is the key. Rotation is therefore
destructive and total: every grant rooted in the old key dies with it.
The procedure, in order:
- Revoke all outstanding capability hashes by writing them to the revocation store, then stop the fabric. Revoking first means a copy of the old keyfile recovered later still cannot resurrect old grants within their expiry windows.
- Delete the dev keyfile at
~/.valis/dev-keyfile. The keyfile is the only persistence of the master key; removing it is the rotation. - Start the fabric. A fresh Ed25519 master key is generated, the owner DID changes, and the derived X25519 transport key and every factotum-served descriptor follow it automatically.
- Re-mint every capability name the owner wishes to keep granting. Old names became unverifiable at step 3 (their signatures no longer match the owner key the verifier is handed), so bearer names held by external parties are invalid immediately, not merely at expiry.
- Reissue delegation tokens to every delegate. Old chains terminate at the old owner DID and fail the root-issuer check.
There is no migration utility in v1: the procedure assumes the loopback development phase, where no real authority is outstanding. When real external grants exist, rotation tooling must walk the revocation and reissue steps mechanically rather than trusting the operator's memory.
Two planes of authority, and why the system is a federation of instances
⇒ The record of how this position was reached, with the constraints that produced it and what it means for work built on valis, is The epistemology of fully connected symbolic computers. This section remains the statement of record; that one explains it.
⇒ The classical statement of this division is in docs/REPL-AXIS.org under The hierarchy, in the classical terms, and it is the one to read first: the Lisp runtime is this machine's operating system kernel, and the capability model is that machine's programming ABI. What follows here is that argument's consequence for how instances relate, which that note poses without resolving.
Authority in this system divides into two planes, and only one of them is described by the rights above.
- The data plane. Reading and editing projections of the operator's own
data. This is what
:read,:write,:mountand:delegatedescribe, and it is where substantially all ordinary operation happens. - The execution context. Root access to the running Lisp machine itself, in the sense that term carries for any machine: loading code, installing a service, changing what the node is.
⛔ A question of the form "which right is evaluation" is malformed, and the
answer is not a new bit. :write cannot carry evaluation, because :write
means editing a projection of the data. A new right alongside the others is the
same category error under another name: it would enter a vocabulary whose
central promise is that a child grant is a subset of its parent, while meaning
every right at once plus the power to mint more. A holder performing correct
attenuation arithmetic on such a right would get a wrong answer, and nothing in
the mechanism would object.
Evaluation is not attenuated. The context is.
Once a party can evaluate in a context, they hold everything in that context by definition. Evaluation reaches every symbol, every package, this capability layer, and any key in memory, so a restricted evaluator resembles a boundary without being one.
⇒ Evaluation is therefore the wrong thing to bound, and the context is the right one. The confinement this substrate applies is outward: it governs what a node may reach, never what the party holding that node may do inside it.
An instance that executes but holds no data has an execution context that is total over itself and confers exactly its outward reach. That reach is capability-mediated 9P to the instances that do hold the data, and those mounts attenuate exactly as the rest of this chapter describes: each carries a grant hash, revocation actively evicts it, an expiry applies, and a read-only grant is wrapped so the write operation has no name.
⇒ The attenuation happens at the 9P boundary, never at the evaluation boundary. A party holds total authority over a context that holds nothing, and reaches the operator's data only through grants the operator can withdraw. Holding that context was never the same as holding the data.
The consequence: cooperating instances, not one image
This is why the system is a collection of valis instances cooperating through the capability system rather than one image serving many parties. A second party is given a second instance, not a confined region of the first. A collaborator, a vendor, or a hosted service can therefore execute without any of them holding the execution context of an instance that holds data.
⚠ An in-image confinement is not an alternative that has been left unexplored. A restricted evaluator, a sandbox, a package boundary, or an audited language subset each resemble a boundary and none of them is one, for the reason above. A better answer to the irreducibility of evaluation would change this design and would be welcome; the shape here is what the constraint currently admits.
The operational shape
Three layers, carrying very different traffic:
- The owner-facing viewer to an instance, over capability-mediated 9P. This is the overwhelming majority of all operation.
- The execution context of each instance, reached individually by real host control, for update and service installation. It is a maintenance channel rather than a user-facing capability, which is why anchoring it to host access is proportionate: whoever holds the machine already holds everything the channel would grant.
- Administration and service installation across instances, intended to be mediated by consensus over material signed by other instances holding operator authority, including instances a vendor maintains for the benefit of its clients. What crosses that boundary is an attestation that an artifact is legitimate; what never crosses it is access to the context that installs the artifact. This layer is direction rather than built shape.
⚠ The seam for the third layer already exists. Module admission runs through a
module's identity, a publisher's vouch for it, and an owner signature carrying the
:admit right. Federating administration changes the signer of the owner's half
from the owner alone to a quorum, and needs no new mechanism. :admit belongs to the execution context,
not to the data rights: no door that mints data rights will mint it.
Code admission has its own door. I never let :admit share a token with
any other right, because loading code into a running node changes what the
node is. It is minted only at /cap/admit, the dedicated admit ctl file, over
exactly one /module/ designation, and a grant minted there carries :admit
and nothing else. /cap/ctl refuses :admit as an unknown right. The admit
file is bound into the owner's own session namespace when that namespace
is assembled, and is not a child of the shared /cap: any other session
finds no such file, rather than a refusal, and a grant naming /cap or
/cap/admit cannot hand the door to a delegate, because grants resolve
against the shared tree. You reach it without Slynk, so admitting code is
a governed act, not something written into a REPL. This follows Plan 9,
which hides power at the level of the file and the namespace, never per verb.
Validation coverage: capability layer
As with the identity layer above, each capability success criterion is
anchored to a named test an auditor can run. All tests live in
tests/capability-test.lisp.
| Criterion | Test | File |
|---|---|---|
| Delegation chain verifies offline against the owner's key | token-chain-valid |
tests/capability-test.lisp |
| Attenuation: child rights must not exceed the parent's | token-chain-child-exceeds-parent-rights |
tests/capability-test.lisp |
| Re-delegation requires the delegate right | token-chain-no-delegate-right |
tests/capability-test.lisp |
| Expired tokens are rejected | token-chain-expired |
tests/capability-test.lisp |
| Revoked tokens are rejected within their expiry window | token-chain-revoked |
tests/capability-test.lisp |
| Name round-trips designation, rights, expiry, nonce, signature | cap-name-encode-decode-round-trip |
tests/capability-test.lisp |
| Valid name yields its mount directive; bad signature or expiry refuse | cap-name-verify-valid, cap-name-verify-bad-sig, cap-name-verify-expired |
tests/capability-test.lisp |
| Revoked names are rejected within their expiry window | cap-name-verify-revoked |
tests/capability-test.lisp |
| The name is the grant: attach with it, walk the designated subtree, out-of-scope axes unnameable | cap-name-attach-and-walk |
tests/capability-test.lisp |
| The capability service is reachable over live 9P | cap-9p-ctl-reachable |
tests/capability-test.lisp |
| Pure capability packages import zero crypto symbols | directive-imports-no-ironclad, token-imports-no-ironclad, name-imports-no-ironclad |
tests/capability-test.lisp |
Two-phase namespace resolution
A session's namespace resolves in two phases. At connection time, the principal
established during the Noise handshake (or the anonymous sentinel for keyless
connections) determines the base namespace the session receives. This is the
connect-time assembly in src/namespace/assembler.lisp, the function
build-base-view. A connection whose principal matches the owner's key receives
the full canonical namespace: all axes present, all of /proto, /bus, /id,
/cap, and /ctl named and reachable. The anonymous sentinel receives the
read-only /pub published view and nothing else, the same surface an
unauthenticated HTTP GET resolves to (see The anonymous published view across
both edges, below). A keyed peer who is not the owner receives an empty
namespace: the connection is admitted, but there are no names to reach for.
Sovereignty is the absence of a name.
After the base namespace is established, a second phase runs when a capability is
surfaced alongside a request. Writing a capability to the session's /ctl node (or
calling the in-image assembler API directly) mounts the designated subtree into the
live view. The capability is verified through the same path as bearer-name attaches:
the designation is resolved, rights are applied, and an audience check is made for
audience-bound tokens. A read-only grant produces a deep read-only proxy (see below);
a write-capable grant mounts the raw node. The mount persists until the session
detaches or the capability is explicitly unmounted.
Base-namespace policy
The base policy has exactly three cases, decided once at Tattach time:
- Anonymous connection (NOFID or keyless): the read-only
/pubpublished view, and only that./pubis published (public by design), so an anonymous mount names it and traverses the published items beneath it, but the floor is wrapped in a read-only projection: every write, create, and publish is refused. The credential-gated axes (/proto,/bus,/id,/cap,/edge) have no name to reach for. If the publication substrate is not up the floor is simply empty: fail-closed, never an error. Capabilities mounted in the second phase still widen the view from here. - Keyed connection whose principal is not the owner: empty namespace. Knowing who a
peer is does not grant them anything; authority arrives only as capabilities, not as
identity alone. A non-owner keyed principal does not receive the
/pubfloor: only the anonymous sentinel does. - Connection whose principal's key matches the owner's key: full canonical namespace. The owner is the root of all authority: for the owner, every axis has a name by definition.
The policy is expressed as a function in src/namespace/assembler.lisp. The identity
check compares the connecting principal's did:key identifier against the owner's DID
from the key custody store (src/identity/custody.lisp). If the custody store has not
been initialised (for example, in tests that exercise individual nodes without
starting the full fabric), the owner-DID check is skipped and the connection receives
an empty namespace: a deliberately fail-closed default.
Bearer-name attaches bypass the base policy: a capability presented as the 9P attach name is verified by the capability service and mounts exactly the designated subtree, regardless of the connecting principal's identity. The name is the grant.
The anonymous published view across both edges
The /pub published view is the one thing an unauthenticated caller can see, and it
looks the same whichever edge they arrive through. An unauthenticated HTTP GET resolves
to the read-only /pub view via the anonymous edge grant the seam holds; an
unauthenticated 9P mount (access=any, no credential) resolves to the same read-only
/pub floor through build-base-view's anonymous branch. Both are projected read-only,
so neither path can write or publish. The credential-gated axes (/proto, /bus,
/id, /cap, /edge) stay absent for the anonymous principal on either edge: an
out-of-scope name fails the walk rather than returning a refusal, so a caller cannot
even confirm the name exists. The owner's full frame is built by
assemble-canonical-frame; the anonymous floor and the owner frame are the two ends of
the same build-base-view policy, with the non-owner keyed principal resolving to
absence between them.
Union directories and bind semantics
A session namespace is an ordered list of mount entries. When two grants cover the same leaf name, the entries form a union: walking the name tries entries in list order and returns the first successful resolution. Directory reads concatenate entries from all union members without deduplication: both entries appear in a listing even when their names collide, because the collision is resolved at walk time, not at list time. This follows the Plan 9 and 9Front kernel's union-directory semantics.
Three bind flags govern how a new mount is added to an existing name:
- Replace: the new entry becomes the sole entry for that name; any prior members are removed.
- Before: the new entry is prepended to the member list; it is searched first on walk.
- After (default): the new entry is appended to the member list; it is searched last on walk. The existing members keep precedence on name collision.
The bind flag is a choice made by the caller at mount time: it belongs to the presentation, never to the grant. A grant carries a designation and rights only; the flag is supplied separately in the ctl grammar or the assembler API.
When a create request targets a union name, the first union member that has write rights receives the create. This is a simplified reading of the Plan 9 MCREATE rule, adequate for the current namespace structure where grant rights are known at assembly time.
The union semantics are implemented in src/namespace/union-node.lisp and exercised
by the assembler; the seven server never interprets union structure: it invokes the
backend's node-walk and node-entries methods and the union node responds
appropriately.
Read-only projection enforcement
When a grant carries only read rights, the assembler wraps the designated subtree in a
read-only projection node at mount time (src/namespace/projection.lisp). The
projection is a deep proxy: walking through it returns further projections, so every
node in the subtree seen through the projection is also read-only. Write, create,
remove, and wstat operations on a projected node fail at open time: the write path
has no name, not merely an error response.
The projection node passes through the inner node's QID unchanged. A QID identifies a resource object, not a view of it, so two sessions projecting the same underlying node present the same QID on the wire. This is correct 9P2000 behaviour: the projection is transparent at the protocol level.
Enforcement is by construction: the projection is what was mounted; there is no policy lookup on the hot path. The decision is made once when the grant is assembled, and thereafter no per-access check is needed or present.
Revocation eviction and live sessions
When a capability is revoked, the revocation takes effect immediately in all live
sessions that have mounted it. The revocation path (src/capability/revocation.lisp)
persists the revoked hash, then calls the assembler's eviction sweep. The sweep walks
the live-session registry and removes every mount entry whose stored grant hash
matches the revoked hash.
The registry is a weak-reference table keyed by the live namespace root, constructed
via make-hash-table with :weakness :key (an SBCL extension). What it maps that
root to is a live-view-entry, and the entry carries considerably more than the
session's principal: the session id an operator has to be able to type, the auth
conversation the session was admitted through, the Unix seconds at which it was
admitted and at which it was last used, a fenced marker that is nil for a live session
and the second of the fence otherwise, and a per-view lock. The id exists because the
registry is keyed by object identity, which is unspeakable outside the image, so an
operator looking at a session they want stopped needs a name for it. The same entry is
also held on the view itself, so the request path stamps its activity mark from a slot
read rather than a lookup on a weak table.
The lock ordering is a discipline rather than an aside. A sweep snapshots the registry
under *live-view-registry-lock* and releases that lock before it takes any per-view
lock, which maintains one acquisition order throughout: registry lock, then per-view
lock. Two paths that take the same two locks in opposite orders deadlock the fabric,
and a deadlocked fabric answers nothing at all. Within that order a sweep holds the
session's own lock across the removal, so a session's namespace is consistent before
and after; no session sees a partially-evicted state.
Mount entries created by the base-namespace policy carry no grant hash and are never affected by the revocation sweep. Revocation targets only capability-backed mounts established through the second phase. That immunity is correct for what the sweeps do and is depended on elsewhere, and it is also why withdrawing an owner's live authority needs a mechanism of its own: see owner session lifetime below.
Weak references are what make an ended session safe to forget. A session whose view root is no longer reachable by anyone is collected without an explicit deregistration hook, and that is a settled arrangement rather than an outstanding one. The gap worth naming is a different thing and it concerns sessions that are still live: a handle already open below the root of a session that has been fenced goes on working. The next section states that limit and what follows from it.
Owner session lifetime: fencing, idleness, and authority transfer
The owner was the one principal whose live authority could not be withdrawn. Authority that arrives by presenting a capability carries a grant hash, so both eviction sweeps can find it and take it back. Authority that arrives by proving who you are is base-namespace policy, carries no grant hash, and both sweeps pass over it by design. The owner's frame is base policy from end to end, so it was precisely the case the sweeps could not reach: short of restarting the fabric, an owner session admitted once went on naming the full frame for as long as its transport lived.
Fencing a session empties its view outright, base policy included. That asymmetry with the two eviction sweeps is deliberate: a mechanism that respected the usual immunity could not touch the very session an operator is trying to stop. What is left is a session that can name nothing, which is how this namespace refuses in general. An out-of-scope object has no name to reach for rather than a name that answers no.
A fence does not tear down the sealed transport, and it is not meant to. The connection stays up and the session on it can reach nothing, which is the same enforcement model the rest of this document describes: absence rather than denial.
A session is stopped three ways.
- By name, at the identity door. An operator reads the live sessions, picks the one they want, and writes a fence command naming it. Fencing the session issuing the command is allowed and does exactly what it says, because an operator who has lost control of a session should not first have to work out whether they are about to cut themselves off, and an attacker sharing that session should not keep it by virtue of being the one still holding it.
- By idleness, checked lazily. The bound is enforced by the operation that would have used the session, not by a sweeper. A dormant session issues no requests and reaches nothing; the case worth catching is someone picking up an abandoned one, and that arrives as an operation.
- By an authority transfer. When write authority moves to another locus, every live owner session on the surrendering instance is fenced. This is a separate step from the capability sweep that runs beside it, and it decides by principal rather than by generation, because an owner's namespace is base policy and the sweep cannot reach it. Without this step the instance that gave up authority would go on serving its owner after an evacuation.
The idle bound is unset by default, and the default is a decision. Applying a bound to a deployment that never asked for one is a lockout nobody requested arriving with an upgrade. Fifteen minutes is the recommended value, short enough that an unattended browser tab stops being a live management console over a lunch break and long enough that ordinary editing is never interrupted, but it is advice a launcher or an operator adopts explicitly rather than a value that applies on its own.
Getting back in means authenticating again, and that is where the seam earns its place. A re-attach on the same transport is refused by conversation, so a client cannot simply ask for a fresh frame and hand itself back what was taken; re-running the handshake yields a conversation the fence has never seen, and that succeeds. This connects directly to the evaporate-and-recondense architecture described later in this document: an owner whose instance has relocated has to be able to ask whether the instance now answering is the one they meant, and a session that silently outlives a relocation is what makes that question unanswerable. Cutting the session is what forces it to be asked.
The limit: an open handle survives the fence
A fence does not revoke a handle already open below the session root. The read path serves an open object without re-consulting the root, so emptying a view stops walks and listings from the root while a client already holding a handle keeps using it. The same limit applies to idle expiry: a client working purely through handles it already holds never reaches the check that would fence it.
Two things follow, and both matter more than the limit itself.
The first is a rule for anyone checking a fence: walk from the session root. A probe that reuses a handle it opened before the fence will observe that handle still working and report a functioning fence as broken. The behaviour under test is reachability by name, so the test has to exercise naming.
The second is where the remedy belongs. The 9P library needs a generic per-session handle revoke, and it needs to know nothing about owners, fences, or capabilities: valis decides which session and why, and the library carries out the revocation and raises a distinct error to the client. The placement test is one line, and it settles questions like this one throughout the constellation: could another consumer of this library use the mechanism with no valis present? Here the answer is yes, so the mechanism belongs there.
That mechanism now exists upstream, as a forcible revocation of a session's handles with its own error class and a stable client-facing message string. valis does not yet call it, so the limit above is live in valis today and is described as it behaves, not as it will behave once the two are wired together.
The door an operator touches
The surface is a pair of files on the identity axis, following the split the capability axis already uses: one file performs and its sibling reports. It sits under identity rather than capability because a session is identity and transport, not a grant.
The listing is read-only and rendered afresh on every read. It advertises no length, because any length it advertised would be a number that was true a moment ago. Beside it is a control file that performs, and the result of a command is held per open, so one caller's result never surfaces on another caller's read.
It is deliberately not an evaluation surface. Command lines are split on whitespace with no reader involved, an unknown verb is ignored rather than diagnosed, and a malformed argument to a known verb produces an error line readable on the handle instead of a signal into the 9P layer. ⚠ The evaluation axis proposed in docs/REPL-AXIS.org gives that property up by its nature, which is exactly why it must be its own axis and must never be reachable from a grant that reaches a control file.
Enforcement validation coverage
Each success criterion for the enforcement-by-mounting layer is anchored to a named test that proves it. An auditor reading this document can run the suite and see each claim stand or fail without reconstructing the coverage from source.
| Criterion | Description | Test | File |
|---|---|---|---|
| Absence not refusal | Out-of-scope resource has no name on the wire; walk raises an error as if the name never existed, not a permission-denied | sc1-absence-of-name-over-socket |
tests/namespace-test.lisp |
| No per-access guard | Walk and read paths contain no verifier calls (structural check of source) | sc2-no-per-access-guard-structural |
tests/namespace-test.lisp |
| Mid-session widening | Writing a capability to /ctl widens the live view; the name is absent before and present after |
sc3-ctl-mount-widens-live-view |
tests/namespace-test.lisp |
| Base policy: anonymous published floor | Anonymous mount names read-only /pub, not the gated tree; empty and fail-closed when /pub is down |
anonymous-mount-sees-read-only-pub-floor, base-policy-anonymous-empty-view |
tests/substrate-test.lisp, tests/capability-test.lisp |
| Base policy: owner canonical | Owner-keyed attach yields the full canonical namespace | base-policy-owner-yields-canonical-frame |
tests/capability-test.lisp |
| Revocation eviction | Revoking a hash removes matching live mount entries; base-policy entries with no hash survive | revocation-evicts-live-mount |
tests/namespace-test.lisp |
| Self-narrowing unmount | A session can shed grants; the /ctl node is irremovable by design |
self-narrowing-unmount |
tests/namespace-test.lisp |
| Projection denial | A read-only grant's write path signals at open time | projection-open-write-mode-denied, projection-write-denied |
tests/namespace-test.lisp |
| Union walk order | First member wins on name collision; directory reads concatenate all members | union-node-walk-order, union-node-walk-first-wins, union-node-entries-concatenated |
tests/namespace-test.lisp |
| Session has a nameable identity | A live session carries an id an operator can read off the listing and type back | session-carries-a-nameable-identity |
tests/fenceable-sessions-test.lisp |
| Fence reaches base policy | Fencing empties the session's view including base-policy mounts, which no eviction sweep touches | fence-removes-base-policy-mounts |
tests/fenceable-sessions-test.lisp |
| Fence is not undone by re-attach | A second attach on the same auth conversation does not restore the frame; re-running the handshake does | fenced-session-is-not-restored-by-attaching-again, authenticating-again-restores-the-frame |
tests/fenceable-sessions-test.lisp |
| Lazy idle expiry | The operation that wakes an idle session fences it; with no bound configured, sessions are left alone | idle-session-is-fenced-by-the-operation-that-wakes-it, an-unconfigured-idle-bound-leaves-sessions-alone |
tests/fenceable-sessions-test.lisp |
| Authority transfer fences owners | Surrendering write authority fences live owner sessions and leaves non-owner sessions untouched | authority-transfer-fences-owner-sessions, authority-transfer-leaves-non-owner-sessions-alone |
tests/fenceable-sessions-test.lisp |
| Fencing leaves the sweeps alone | The capability eviction sweep still spares base-policy entries after the fence path exists | the-capability-sweep-still-spares-base-policy |
tests/fenceable-sessions-test.lisp |
| The door reports and performs | The listing names the live sessions, a fenced session reads as fenced, and a malformed command is refused rather than signalled | the-door-lists-the-live-sessions, a-fenced-session-reads-as-fenced-in-the-listing, the-door-refuses-a-malformed-command |
tests/fenceable-sessions-test.lisp |
| Fence over a real socket | Over a sealed client connection the door names a live session, the fence empties the session read from its root, and it takes only the session it names | the-door-names-a-live-session-over-the-wire, a-fence-empties-a-live-session-read-from-its-root, a-fence-takes-the-session-it-names-and-leaves-the-other |
tests/session-fence-over-the-wire-test.lisp |
| Publication placed on a running node | An owner-keyed client places a publication on a node already running, including a body spanning several messages | publish-places-a-publication-on-a-running-node, publish-carries-a-body-larger-than-one-message |
tests/owner-publish-test.lisp |
Publication substrate and wire-format adapters
The publishing category of the substrate begins here: one publication object, owned by the person, with the wire protocols as projections over it. HTTP and Gopher are the first two projections, chosen deliberately as a pair: two protocols with nothing in common but the object they render prove the object is protocol-agnostic in fact, not by assertion.
Publication object model
A publication is a directory node in the 9P namespace whose children are
named attribute files (title, author, published-date,
content-type, and an optional gemtext) plus a body file. The
object is shaped as a Plan 9-faithful directory-of-files: each attribute
is individually walkable, individually grantable, and independently
readable by an adapter that needs only some fields. The five core
attributes are the web-minimal metadata set; gemtext is a per-protocol
representation used by the Gemini adapter when a publication wants to
serve native gemtext while the generic body carries HTML for HTTP. The
adapters are required to consult all present attributes when rendering:
this faithfulness contract is what makes the projections equivalent views
rather than competing copies.
Absent attribute files are tolerated at read time. The node layer
returns defined defaults (Untitled for title, an empty author, a date
derived from the publication's modification time, text/plain for
content-type, an empty body), so a half-authored publication stays on
the wire during creation instead of erroring until every field exists.
The defaults are applied by the node layer, not the store: a store
backend reports absence (a null attribute, an empty body), and the
caller decides what absence means. The node classes live in
src/substrate/publication.lisp; the /pub axis that roots them in the
namespace is src/namespace/pub.lisp.
Protocol-agnosticism is an import invariant, not a convention. The store and publication packages import no protocol or registry package, and the adapter packages import no substrate package: an adapter's only path to a publication is a real 9P attach, walk, and read. A future adapter must preserve this boundary: if it can name a substrate symbol, it has bypassed the capability seam the design depends on.
Placing a publication on a running node
A publication reaches the published axis at any time. An owner-keyed client walks the axis over the sealed loopback fabric, creates the slug when it is not already there, and writes the content type and the body; a slug that already exists is replaced rather than refused, so a page can be revised as often as its author likes. The reason this matters is plain in the operator's terms: a node built before a page existed must still be able to receive that page.
Genesis is not a gate on content
Seeding the branded landing publication at genesis is one behaviour with two callers and no policy gate. There is no absence check, no reseeding at boot, and no record anywhere of whether something has been placed. The consequence is the point rather than an oversight: if the operator deletes a publication it stays deleted.
The principle generalises well past publications. Everything here is meant to be mutable by the operator while it runs, which is a large part of why the substrate is written in a language whose systems are patchable live. A system that restores what you removed is not one you control, and an ordering gate at genesis would contradict the whole arrangement.
The write discipline, and one case that still lacks it
The sealed session negotiates a message size of 8192 bytes, so a body larger than one message arrives as several writes at advancing offsets. Such a body must be accumulated across the writes and committed when the handle closes. Before that was done each write replaced the last, and an 11114-byte body read back as 2945 bytes, exactly its trailing remainder. Committing at close also makes the replacement atomic from a reader's side: the previous body is served until the new one is whole.
Any writable node whose content can exceed that ceiling needs the same discipline, and one does not yet have it. Attribute writes have the identical shape and persist each write on its own. Most attributes are far too short to reach the ceiling, but the per-protocol gemtext representation is stored as an attribute and could exceed it. No path writes one today, so the case is latent rather than live, and it is recorded here so that it is found by reading rather than rediscovered as a surprise.
Store seam
The store protocol (src/substrate/store.lisp) is the seam between the
namespace node layer and backing storage. It is a narrow CLOS contract:
eight generic functions (list, attribute read and write, body read and
write, create, existence, and modification time) each taking a store
instance as its first argument. There are no default methods; a backend
that misses a method is a missing-method error, never a silently
papered-over default. Reads are tolerant by contract: absence comes back
as a null attribute, an empty octet body, or a null mtime, and the
calling node layer applies the documented defaults. The filesystem
backend (src/substrate/fs-store.lisp) is the v1 implementation; a
future content-addressed store implements the same eight functions and
no node class above the seam changes. This seam is the designated
replacement point for the durable-storage work.
The filesystem layout mirrors the namespace: one directory per publication, one file per attribute, the slug as the directory name. The layout is inspectable with ordinary tools and is the same tree a kernel v9fs mount would see.
Writes are atomic via rename-into-place: the value is written to a
temporary file in the publication's own directory, then renamed over the
destination, so a reader never observes a torn write: the rename is
atomic because source and destination are on the same filesystem.
Temporary names are unique per write (process id plus an atomically
incremented counter), so concurrent writers never collide on a shared
temp path. Writes are durable as well as atomic: the temporary file's
data is flushed to stable storage (fdatasync) before the rename, and
the containing directory is flushed (fsync) after it, so a write that
has returned survives a crash or power loss; creating a publication
flushes the data directory the same way, so the new directory entry is
itself durable. A write whose flush or rename fails signals
store-error instead of reporting success. The cost is two device
flushes per write (tens of milliseconds on rotating storage) which
suits operator-paced publication writes; *durable-writes* can be
bound to nil where durability is deliberately traded for throughput,
such as bulk imports.
Slug validation is the path-injection guard, and it is enforced
fail-closed at both ends of the object's life. A slug is a non-empty
string of lowercase ASCII [a-z0-9-], checked at the create path
(node-create on the /pub root, and again inside the store backend
before any directory is made) and at the read path (node-walk refuses
a non-conforming name even when a matching directory exists on disk).
Because the walk side validates independently, no adapter bug or
encoding trick can turn a wire request into filesystem traversal: the
HTTP adapter additionally percent-decodes each path segment before
validating it, so an encoded .. is rejected as the decoded text it
really is, and the Gopher adapter rejects .. segments in the raw
selector. The shared predicate lives in src/namespace/pub.lisp.
Durable content-addressed store backend
The durable backend (src/substrate/store-store.lisp) is the second
implementation of the store seam, the content-addressed analog of the
filesystem store. It implements the same eight generic functions over a
content-addressed block stack, a Merkle tree codec, and a
generation-fenced head, so no node class above the seam changes and the
/pub path behaves identically to the filesystem backend. The seam, not
a backend choice, is what the rest of the system sees; this backend exists
so publications can become crash-consistent and content-addressed without
disturbing the publication object model that sits on top of it.
The on-tree layout is the same shape the filesystem backend writes to
disk, expressed through the tree codec rather than through directory
entries. The head names a root tree; that tree carries a pub directory
child; each publication slug is a subdirectory under it; each attribute is
a file child named for the attribute; the body is a body file child.
The publication node layer's slug / attribute / body model therefore
carries over unchanged: the durability mechanism shifts, the addressing
model does not.
Durability here is structural rather than a flush discipline bolted onto
a rename. A block is content-addressed and written (and flushed)
before the head names it, so the head never references a block that is
not already durable. Advancing the head is a generation-fenced
compare-and-swap, and that single advance is the only linearization point
for a write: a write that carries a superseded generation is rejected at
the commit and surfaces as store-error, never a silent overwrite and
never a retry that could lose a concurrent change. This is the
durable-before-referenced ordering the rest of the store stack already
relies on, reused at the publication seam.
Writers within one image are serialized by an in-image lock the backend
holds for the read-modify-write cycle, so the only way the
compare-and-swap can fail is an out-of-band advance from outside this
image, and that case fails closed as store-error. The lock removes the
need for an unbounded retry loop; the compare-and-swap remains the
authority on what actually committed.
One place the POSIX assumption behind the seam leaks through: the seam exposes a publication modification time, and there is no filesystem timestamp to report. The durable backend synthesizes that value from the head generation, which is monotone, so a later write always reports a later time. The value is used only as the published-date display fallback when a publication carries no explicit published-date attribute; it is a display proxy, not a wall-clock guarantee.
The backend is selected when the fabric starts (src/fabric.lisp,
start-fabric). The durable store is the wired default; the filesystem
store is retained as a selectable operator fallback for a conservative
rollout or for continuity with an existing on-disk publication tree. A
fresh durable store serves an empty /pub on first boot: there is no
in-band migration of existing filesystem publications into the durable
store, so an operator who needs continuity with a filesystem publication
tree selects the filesystem backend until a later migration path lands.
The durable store roots at its own data directory, distinct from the
filesystem publications tree, resolved in a fixed order: an explicit
operator-supplied store directory wins; otherwise the store roots at a
sibling of a supplied publication directory; otherwise it falls to the
XDG data home default. Both the durable store's data directory and the
selected store are cleared on a failed start and on stop, so a fabric
that fails to come up or that has been stopped leaves no dangling seam
state behind.
Two boundaries make this backend auditable. First, it is the one file that bridges the store seam and the lower block, head, and tree codecs, and it deliberately reaches no identity, capability, namespace, or protocol symbol: the seam-crossing is one-directional and structurally enforced, so a capability bug cannot reach storage and a storage bug cannot reach the capability layer. Second, the store holds opaque blocks: it never interprets publication content and carries no key bytes. The durability layer sees ciphertext-or-cleartext blocks as bytes to address and name, never as material to read: the sovereignty boundary that keeps key custody out of the storage path holds here as it does everywhere else in the substrate.
Adapter attach and projection-over-capability
The wire-format adapters (the adopted c3po-http and c3po-gopher
modules) are ordinary valis protocol plugins: each registers on its port and
specialises handle-connection. What makes them adapters rather than
owners is how they reach content. At fabric start
(src/fabric.lisp), each adapter is handed a freshly minted bearer
capability designating /pub with read-only rights, and opens a
long-lived 9P session by attaching over an in-process channel, the same
codec and the same enforcement path a remote 9P client would traverse,
with no TCP loopback. The assembler resolves the capability to a mount
directive and builds a view containing exactly one entry: a read-only
projection wrapping the publication root. The adapter can walk and read;
a write-mode open is denied at the server by the projection wrapper, not
by the adapter declining to write. Because the attach path is the
in-process form of the real transport, the adapter is structurally
identical to a sandboxed out-of-process edge module. Moving it out of
the image changes the transport endpoint and nothing else: capability,
view assembly, and projection enforcement are already in their final
shape.
Each wire request maps one-to-one onto a walk in the adapter's mounted
subtree. A missing name is a walk failure, and the adapter surfaces it
as the protocol-native absence: HTTP 404, Gopher error entity type 3.
A principal with no /pub grant has no /pub name at all, so absence
reaches the wire by construction, never by a permission check on an
admitted request.
Responses and requests are size-bounded. Directory listings and file
bodies are read with hard ceilings (one megabyte for a listing, sixteen
megabytes for a body), so a pathological publication cannot balloon a
response buffer. Request lines are byte-capped on read (4096 bytes for
an HTTP request line, 1024 for a Gopher selector), and overflow is
answered with HTTP 413 or a Gopher type-3 error rather than unbounded
buffering. Connection occupancy is bounded the same way memory is: each
adapter arms a per-connection read deadline when it accepts a request
(*http-read-deadline-seconds* and *gopher-read-deadline-seconds*,
ten seconds by default), so a client that connects and sends nothing
(or dribbles bytes slower than the deadline) cannot hold a handler
thread past it. Expiry is treated exactly like a malformed request:
HTTP answers 408 Request Timeout, Gopher closes without a response per
its one-transaction model, and the executor reclaims the thread and the
socket. The mechanism lives behind the connection seam: the adapter
arms the deadline through a transport-blind generic, and the epoll
backend (the one component allowed to name the socket library) wakes
the blocked reader by shutting down the socket's read side, which
surfaces to the handler as ordinary end-of-file while leaving the write
side open for the parting response. In-memory connections, and any
transport with nothing to time out, accept the call as a no-op.
Adapter teardown is complete: stopping the fabric closes both ends of each adapter's in-process channel, and close means drain-then-EOF: frames already written still deliver, then both sides observe end-of-file. The teardown order is deliberate. The port is unregistered first, so no new connection lands in a dying session; the live fid handles are clunked while the channel still delivers; the channel is then closed, which wakes the server's blocked dispatch loop and lets it return; finally the stopped adapter joins its server thread. The client-side reply reader exits on the same EOF, with any outstanding waiters drained by a transport-closed condition rather than left blocked. The join is bounded: a server thread that fails to exit within the timeout is a regression in the channel's close path, and it surfaces as a loud warning naming the leaked thread rather than a stop that hangs forever: teardown must never trade a thread leak for a deadlock.
Validation coverage
Each success criterion and security property of the publication
substrate is anchored to a named test. The substrate tests prove the
object model and the 9P session path; the render tests drive
handle-connection end to end against an in-memory connection (the
same entry point a real TCP client reaches), so the request-parse, walk,
and response-render paths are exercised exactly as shipped.
| Criterion | Test | File |
|---|---|---|
| Store backend round-trips every store protocol operation | substrate-store-protocol |
tests/substrate-test.lisp |
| Non-conforming slugs rejected fail-closed on create and on walk | substrate-slug-charset-fail-closed |
tests/substrate-test.lisp |
| Absent attributes read as defined defaults | substrate-tolerant-defaults |
tests/substrate-test.lisp |
| Owner creates and writes publications via 9P | substrate-authoring-create-write |
tests/substrate-test.lisp |
| One publication object backs both adapters | substrate-publication-shared-object |
tests/substrate-test.lisp |
| HTTP adapter reads only through the read-only projection | substrate-http-read-through-projection |
tests/substrate-test.lisp |
| Gopher adapter reads through the read-only projection | substrate-gopher-read-through-projection |
tests/substrate-test.lisp |
| No grant means no name: capless session sees absence on the wire | substrate-no-mount-no-name |
tests/substrate-test.lisp |
| HTTP listing renders end to end and links the publication | http-render-directory-listing |
tests/adapter-render-test.lisp |
| Listing links below the root carry the parent path | http-render-nested-listing-links |
tests/adapter-render-test.lisp |
| HTTP serves the publication body end to end | http-render-file-body |
tests/adapter-render-test.lisp |
| Percent-encoded traversal segments are rejected with 404 | http-render-rejects-encoded-traversal |
tests/adapter-render-test.lisp |
The /pub node refuses traversal names regardless of adapter |
pub-walk-refuses-traversal-slug |
tests/adapter-render-test.lisp |
| Duplicate create surfaces as a precise 9P error, not an internal one | pub-create-duplicate-signals-clean-error |
tests/adapter-render-test.lisp |
| Gopher menu renders end to end with the RFC 1436 terminator | gopher-render-menu |
tests/adapter-render-test.lisp |
| Gopher serves the publication body end to end | gopher-render-file-body |
tests/adapter-render-test.lisp |
| An idle client is cut off at the read deadline, its thread reclaimed | idle-client-cut-off-at-read-deadline |
tests/read-deadline-test.lisp |
| A prompt request is served normally with the deadline armed | prompt-request-unaffected-by-armed-deadline |
tests/read-deadline-test.lisp |
| HTTP answers a deadline cut-off with 408 | http-deadline-expiry-answers-408 |
tests/read-deadline-test.lisp |
| Gopher closes a deadline cut-off without a response | gopher-deadline-expiry-closes-silently |
tests/read-deadline-test.lisp |
| Fabric stop reaps session threads and clears every seam variable | adapter-lifecycle-reaps-threads |
tests/adapter-render-test.lisp |
The bus: dispatch over 9P
Inter-module and inter-node dispatch rides 9P itself. 9P is already a network-transparent RPC over a namespace (it is a message bus), so a protocol module's mailbox is a file, and routing a connection to a module running in a local thread or on a remote Lisp node is the same file operation either way. A separate message-bus transport (e.g. ZeroMQ) is admitted only where a pub/sub fan-out pattern genuinely beats file semantics, not as the default carrier.
There is no usable Common Lisp 9P implementation in the wild, but 9P2000 is
a small protocol (~13 message types), so a native CL client+server is
tractable. Because the Linux kernel ships a 9P client (v9fs), a valis 9P
server can additionally be mounted by the host kernel, bridging the
namespace into ordinary Linux tools.
Migration: light process, durable namespace
A person's valis is migratable (it can be stood up on any valis-compatible host) because the durable thing is the namespace, not the process. Migration is therefore reassembly, not transfer: a fresh instance reconstructs the filesystem semantics from a small manifest of mounts plus the owner's keys, and the destination host's agent re-steers the designated ports to it. This demands a continuous discipline: keep the process boundary light, externalising anything reconstructable, so standing up an instance is cheap. The model is Plan 9's: a per-process namespace (assembled, local) distinct from a shared authoritative file server.
Where the namespace lives: tiered by access pattern and secrecy
- Bulk and cold data is content-addressed and immutable: a block's hash is its name (the Venti model; iroh carries it between nodes). Immutable content is location-independent: an instance pulls it from wherever it is served.
- Published data is content-addressed and unencrypted, served from many places at once. Because it lives outside the mutable store, high-fanout public reads never touch the private substrate.
- Mutable state lives on a backing store the instance mounts; the mutable head is a small hash pointer over the immutable tree, in the manner of fossil over Venti or git refs over objects.
- Secret keys are owner-held or held in a separate trust domain; they are never content-addressed and never live solely on a host that might be evacuated.
- A hot working set is served by a read-side caching tier in front of the store, so a coincident instance keeps its reads off the single-writer head while still observing every committed write. It is two caches and a coherence primitive, each of which gets its correctness from a fact the store already carries (a block's hash is its name, and the head's generation is its version), so the cache adds speed without adding a way to be wrong. The read-side caching tier below is the mechanism.
The reassembly path in code: where to follow it
The reassembly is not a single function but an ordered handshake whose composition
site is src/fabric.lisp → start-fabric (the :evacuative t arm drives the
security regime; see the two regimes below). An auditor can follow
it through these named seams.
A fresh instance comes up only if it holds the owner's key out of band: the gate is
src/fabric.lisp → owner-key-required, which fails closed when the keyfile is
absent rather than standing up an unauthenticated substrate; the key itself is loaded
through the custody seam (src/identity/custody.lisp → load-or-create-keyfile), so
the key bytes never live on the durable store the instance mounts. With the key in
hand, the instance reconstructs the filesystem semantics at
src/namespace/assembler.lisp → assemble-canonical-frame, the single reassembly
entry that rebuilds the view from the manifest of mounts; an unreachable mount is a
fail-closed stand-up (reassembly-unreachable), never a silent partial view. The
assembler reads the authoritative mutable head through an injected seam,
*durable-head-reader* (bound at fabric start, src/fabric.lisp), so the same
reassembly code path serves whether the head is read from the durable store or from a
test double: the durability boundary is one named indirection, not a scattered set of
direct reads.
The mutable head carries the generation that is both the consistency token and the
fence (the two regimes section traces its security role). It advances
through one linearization point: src/store/head.lisp → advance-head, a
compare-and-swap against the expected generation. A late writer that expected a
superseded generation loses the swap and is told so by generation comparison alone:
no wall clock, no lease deadline. The pre-commit fault seam *advance-head-pre-commit-hook*
(src/store/head.lisp) lets the out-of-band crash harness freeze a writer exactly at
the swap to prove the rejection survives a real pause.
Consistency across coincident instances
valis instances can be temporally coincident: several at once, to absorb load. Authority over the mutable namespace is a single writer per subtree (the fossil model), not a leaderless or eventually-consistent scheme: a sovereign namespace wants one authoritative truth, not merge. For one person's modest write load a single writer is ample; if write scaling ever bites, the namespace (already a tree of mounts) is partitioned into per-subtree authorities (the Ceph dynamic-subtree-partitioning model). Coincident instances each assemble their own namespace view and interpose caches; the authoritative writer serialises mutation; public read load is offloaded to the unencrypted content-addressed publishing path.
The read-side caching tier
A coincident instance must be able to read fast without consulting the single writer on every access, yet must never serve a stale answer once a write has committed. The read-side tier achieves both by deriving each cache's correctness from a value the store already carries, so caching adds no new way to be inconsistent. It has three parts.
The first is an immutable-block cache (src/store/block-cache.lisp), a
read-mostly tier in front of the content-addressed block device, one per store
handle and never a process-global. Because a block is named by the SHA-256 of
its bytes, the name is the validator: a clean hit returns the admitted bytes
verbatim and never re-hashes, yet can never be stale, because the same name
always means the same bytes. Admission is verify-then-admit: the bytes are
re-hashed against their requested name before they enter the cache, on the
store's single digest seam, so a substituted or bit-rotted on-disk block is
rejected at admission and can never become a hit. The corollary is that a
cached hit stays correct even if the on-disk block is later corrupted: the
content is immutable, so the in-memory copy is as authoritative as the disk
ever was. Growth is doubly bounded (an explicit byte budget and an
independent descriptor/slot cap, so a flood of tiny blocks cannot exhaust
entries while staying under the byte budget) with CLOCK (second-chance)
eviction, the right policy for an immutable read-mostly tier where eviction
order does not move the hit rate. A reader that must hold a block across an
operation pins it: with-pinned-block refcounts the slot and releases the
pin on every exit path via unwind-protect, and eviction skips a pinned slot
rather than force-evicting it: force-evicting a block a reader still holds
would be a use-after-free of its byte vector.
The second is a generation-revalidated head-decode cache, one fixed record
per store handle (src/substrate/store-store.lisp). Decoding the head's pub
directory is the expensive part of a read; caching the decoded result would
normally reintroduce staleness. It does not here, because the head already
carries a monotonic generation token (the same token that fences a
superseded writer), and that token doubles as the cache validator. Every read
re-reads only the generation through a narrow reader (current-head-generation,
src/store/head.lisp) that integrity-verifies the 124-byte HEAD record but
decodes no tree, reuses the cached decode when the generation is unchanged, and
reloads on any change. The generation probe, the compare, the decode, and the
recorded generation all happen inside one critical section under the store
lock, so the cache is always stamped with the generation that was actually
current: a coincident writer cannot advance the head between the probe and the
decode and leave the cache mislabelled. The staleness bound is therefore zero:
the moment another handle commits, the next read here sees a moved generation
and reloads. The write path is untouched (it reads the current head directly
under the lock and never consults this read cache), so the cache cannot
interfere with the compare-and-swap that is the sole linearization point.
The third is content-addressed qid.version, the coherence primitive a
correct cache above the store needs. A served node's 9P qid.version is the
low 32 bits of its content score (store-node-score / score-low-word), with
0 remapped to 1 so genesis, an absent slug, and an unwritten leaf all read
1, never 0 and never an error. The score is a pure function of the node's
durable content (the Merkle property: a directory's score changes iff its
listing changes, a leaf's iff its bytes change), so the version is correct
across instances by construction: two instances over one data directory
compute the same version for the same node with zero coordination, and a write
to one slug never perturbs another slug's version because their scores are
independent. The override is per node, with no shared mutable version counter,
so there is nothing to serialize. This is what lets a 9P or HTTP cache one tier
up validate correctly: the version it keys on is itself the content hash, which
is exactly the change-detector a cache validator wants.
The three compose into the property the coincident regime needs: reads run off the per-handle caches at content-addressed speed, the generation token keeps the head-decode cache coherent with staleness zero, and the content-derived version lets every cache above the store (including an ordinary HTTP cache at the edge) track live content without a coordination protocol.
- Validation coverage: the read-side caching tier
Each property of the caching tier is anchored to a named test. The block-cache tests drive the cache directly; the store-store tests drive the generation-revalidated decode and the content-addressed version over the real durable store.
Criterion Test File A clean cache hit returns the device bytes verbatim, with no re-hash block-cache-hit-returns-device-bytestests/store-block-cache-test.lispAdmission verifies bytes against their score; a corrupt block is never admitted block-cache-verify-then-admittests/store-block-cache-test.lispA cached hit stays correct even after the on-disk block is corrupted block-cache-immutability-under-ondisk-corruptiontests/store-block-cache-test.lispResident bytes never exceed the byte budget block-cache-honours-byte-budgettests/store-block-cache-test.lispThe descriptor count never exceeds the slot cap block-cache-slot-captests/store-block-cache-test.lispA pinned block is never evicted while it is held pinned-block-never-evictedtests/store-block-cache-test.lispThe pin is released on every exit path from with-pinned-blockwith-pinned-block-releases-on-every-exittests/store-block-cache-test.lispThe cache layer reaches no capability/namespace/identity symbol block-cache-import-cleantests/store-block-cache-test.lispThe narrow generation reader verifies the HEAD but decodes no tree current-head-generation-narrow-readtests/store-head-test.lispAn unchanged generation reuses the cached head decode store-store-unchanged-generation-reuses-decodetests/substrate-store-store-test.lispCoincident handles over one store read each other's committed writes store-store-coincident-handles-read-your-writestests/substrate-store-store-test.lispThe head-decode cache is per handle, never shared across instances store-store-head-cache-is-per-instancetests/substrate-store-store-test.lispA node's version is non-zero after a write store-store-version-nonzero-after-writetests/substrate-store-store-test.lispA write to one slug does not move another slug's version store-store-version-per-file-isolationtests/substrate-store-store-test.lispTwo instances over one data directory compute the same version for a node store-store-version-cross-instance-sametests/substrate-store-store-test.lispGenesis and an absent slug read version 1, never0store-store-version-genesis-absent-slug-is-onetests/substrate-store-store-test.lisp
Two regimes, and evacuation under attack
The mutable head carries a monotonic generation token, which yields consistency and fencing from one mechanism. There are two migration regimes:
- Coincident (scaling): several instances share the store; the generation/lease serialises who may advance the head.
- Evacuative (security): fleeing a computational substrate under attack, assuming the host is compromised. Standing up the new instance bumps the generation, which fences the old, possibly-compromised instance: the store rejects writes from a superseded generation, the same shape as revoking a superseded capability. Because keys never lived solely on the evacuated host, the new instance comes up without trusting the old one.
In-flight connections drop and re-establish by default, acceptable because the
namespace, not the process, is the source of truth. Live carry-over of
established TCP connections (the kernel's TCP_REPAIR path) is a possible
enhancement for protocols that cannot tolerate a reconnect, not a dependency.
Re-steering the established sessions cleanly (so the old instance's clients
reconnect to the new one rather than waiting on a passive TCP drop) is the job of
a session-drain seam on the steering host's agent (the chartered drain-sessions
boundary, proposed in the steering agent's repository). That seam is a latency and
user-experience improvement on handover, never the fence: the fence is the store's,
and it holds whether or not the old host cooperates, because the threat model
assumes the old host may be compromised.
The fence and the sweep: where to follow them
The bump fences the superseded instance through the same one revocation mechanism
that revokes a capability. The fence write is src/capability/revocation.lisp →
fence-epoch-hash (the content-addressed name of a fenced generation) entered into
the durable revocation set by revoke-hash (fdatasync-durable, so a fence survives
a crash the instant it is written). A write capability is rejected if its bound
generation is fenced: the verifier consults the revocation set at
src/capability/verifier.lisp → %write-cap-fenced-p, on the same path that checks
an ordinary revoked token. Each capability carries the generation it was minted under
as src/capability/token.lisp → token-bound-generation, so the consult is a pure
membership test against the fenced generation: a superseded writer is rejected by
construction, not by a host that must be trusted to police itself. Finally, live
write-capable grants still held under the fenced generation are dropped by
src/namespace/assembler.lisp → evict-by-fence-epoch, so a fenced instance keeps
no live authority even in memory.
Evacuation walkthrough: one migration, end to end
One full evacuation traces these seams in order. An operator (or the new host's agent) stands up a fresh instance over the same durable data directory while the old, possibly-compromised instance is still running.
- Owner-key gate. The fresh instance refuses to start without the owner's key,
held out of band (
src/fabric.lisp→owner-key-required, loaded viasrc/identity/custody.lisp→load-or-create-keyfile). Because the key never lived solely on the evacuated host, the new instance comes up without trusting the old one. - Reassembly. It rebuilds the namespace from the manifest of mounts
(
src/namespace/assembler.lisp→assemble-canonical-frame), reading the authoritative head through the injected*durable-head-reader*seam: reassembly, not transfer. - Generation bump. It advances the head past the running instance's generation
(
src/store/head.lisp→advance-head), the single compare-and-swap that is the store's only linearization point. - Fence. The bump writes the superseded generation's
fence-epoch-hashinto the durable revocation set viarevoke-hash(src/capability/revocation.lisp),fdatasync-durable. - Sweep. Live write-capable grants under the fenced generation are evicted
(
src/namespace/assembler.lisp→evict-by-fence-epoch), so the old instance retains no live authority in memory. - Rejection of the late writer. The old instance's next write loses on both paths:
its capability is rejected at the verifier consult because its
token-bound-generationis fenced (src/capability/verifier.lisp→%write-cap-fenced-p), and its compare-and-swap is rejected because it expected a superseded generation (advance-headreturns the stale sentinel). The rejection is clock-free: it holds across an arbitrarily long pause. - Serving resumes. The new instance serves the reassembled namespace at the pinned generation; in-flight connections to the old instance drop and re-establish.
The whole handshake is composed at src/fabric.lisp → start-fabric with the
:evacuative t arm, the one site an auditor reads to see the steps wired together,
rather than re-implemented in a test body.
Opaque-block audit: why an operations path reaches no content
The store holds opaque blocks, addressed and named by the hash of their bytes, never
interpreted. The durability cluster (store-store, block-device, block-cache,
block, head, tree, manifest) imports zero capability, namespace, or identity
symbol, so a storage-side or operations-side path cannot become a confused deputy that
reaches content or keys: the absence of the import is structural, proven over the whole
cluster by store-cluster-imports-no-content-symbols, not a runtime check that could be
skipped.
The table below audits the operations paths an operator would use to move or observe a substrate. None of the backup or observability subsystems named here exists yet: the audit covers the seam each would build on and shows that the seam reaches octets, not meaning. The store handles opaque octet vectors and is encryption-ready, not encrypting: it sees ciphertext-or-cleartext blocks as bytes to address, never as material to read.
| Operations path | Seam it would use | Why it reaches no content | Anchoring test |
|---|---|---|---|
| Migrate / evacuate a substrate | start-fabric :evacuative t → reassembly + bump |
Moves the head pointer and the addressed blocks; never decodes a block, never holds a key | migration-reassemble-then-evacuate-end-to-end |
| Back up the durable store (subsystem TBD) | the block-device / head / tree codecs | Copies hash-named opaque blocks and the 124-byte head record; no content/key import path | store-cluster-imports-no-content-symbols |
| Observe / meter the store (subsystem TBD) | the store handle's counters and the narrow head reader | Reads generation and block counts; the cluster imports no namespace/capability symbol | store-cluster-imports-no-content-symbols |
| Revoke an evacuated instance's authority | revoke-hash over fence-epoch-hash |
Writes a content-addressed fence into the revocation set; the set holds hashes, not data | two-instances-only-one-write-accepted-fail-closed |
| Sweep live grants under a fenced gen | evict-by-fence-epoch |
Drops in-memory write grants by generation; touches no block bytes and no key material | bump-sweeps-live-write-grants |
Validation coverage: migration and evacuation
Each success criterion of the migration-and-evacuation story is anchored to a named
test. The end-to-end and fence tests drive the real start-fabric :evacuative t path
over a durable store; the out-of-band proofs run in the standalone crash harness, where
a real process is paused or killed across the bump.
| Criterion | Test | File |
|---|---|---|
| A fresh instance reassembles and serves; the evacuated instance is fenced | migration-reassemble-then-evacuate-end-to-end |
tests/migration-end-to-end-test.lisp |
| and its live write grants are swept | ||
| Live write-capable grants under the fenced generation are swept by the bump | bump-sweeps-live-write-grants |
tests/store-fence-revocation-test.lisp |
| A write paused across the bump is rejected at the verifier and at the CAS | two-instances-only-one-write-accepted-fail-closed |
tests/store-fence-revocation-test.lisp |
| The store cluster imports no content, key, or namespace symbol: a confused | store-cluster-imports-no-content-symbols |
tests/migration-end-to-end-test.lisp |
| deputy is impossible by construction | ||
Out of band: an old process SIGSTOP'd across the bump cannot land its CAS |
proof-evacuation-under-pause |
scripts/store-crash-test.lisp |
| Out of band: a recovered head never names a generation past an unfenced one | proof-fence-precedes-head-advance |
scripts/store-crash-test.lisp |
The mail transport spine
valis terminates mail as a first-class axis of the sovereign namespace. The wire engines that speak SMTP and IMAP to the hostile internet live in a separate process at the IETF edge; valis owns the substrate the mail moves through: the authenticated submission door, the durable transport queue, the router that decides local-versus-relay, the per-DID Maildir inbox, and the seam a wire adapter drains relayed mail through. The whole spine moves an opaque message body end to end: every layer below names and addresses the body by the hash of its bytes and never interprets it.
This chapter is the auditor's entry point to that spine. It traces one message end to end through the named code, states what the open-relay guard guarantees and why it cannot be turned off, and shows that no operations path reaches the content of a message. The wire is stubbed for the proof: what is proven is the substrate's end-to-end movement, not delivery onto a real network.
Mail flow walkthrough: one message, end to end
One submission traces these seams in order. The owner stands at /mail/outbox
holding a write capability minted through the real /cap/ctl path; an anonymous
or unauthenticated peer never reaches this door.
- Authenticated submission (the clunk commit). A capability-mounted 9P write
into
/mail/outboxstages octets; the clunk is the atomic commit (src/namespace/mail-outbox.lisp→node-closeon the outbox-submission file). The envelope and the submit provenance are stamped at the clunk, never parsed from the body; a fid abandoned without a clunk lands nothing: complete or absent by construction. - The live seam.
/mailis present in the running owner frame only becausesrc/fabric.lisp→start-fabricbinds the three mail seams (*mail-dir-root*,*local-delivery-did-resolver*,*local-domains*), each gated on a durable store-backed substrate and reset fail-closed on teardown. The axis is mounted bysrc/namespace/assembler.lisp→%ensure-mail-mountand reached atsrc/namespace/root.lisp→node-walkon "mail". - Ingest. The clunk hands the assembled octets, derived envelope, and stamped
provenance to
src/mail/seam.lisp→land-message, the single ingestion door through which both owner-submitted and peer-landed mail enter. - Route.
src/mail/router.lisp→route-messageclassifies every recipient throughroute-recipient, which tests locality against*local-domains*and admits relay only on proven submit authority (see the open-relay guard below). - Queue. Each routed recipient lands a durable per-recipient record via
src/mail/transport-queue.lisp→place-incoming: the ack follows thefdatasync, so the queue is durable before the submitter is told "committed". The relay path is admitted into a bounded active window withadmit-activeand the slot is returned on a terminal outcome withrelease-slot. - Local delivery. A local recipient is written into the owner's Maildir at
/mail/inbox/<did>/new/bysrc/mail/local-deliver.lisp→local-deliver, keyed under the live owner DID the*local-delivery-did-resolver*returns from custody. - Relay drain. A relay recipient is drained by
src/mail/drain.lisp→start-mail-drain/drain-active-once, which hands the opaque message to a registered mail-adapter. The wire engine is stubbed here; the adapter records a disposition and the drain frees the active slot.
The same path is driven against the live fabric (outbox capability, real router,
real queue, real delivery) in tests/mail-end-to-end-test.lisp, so the
walkthrough above is exercised, not merely asserted.
The open-relay guard: fail-closed by capability, not by flag
route-recipient reaches the relay path by exactly two yes-paths: the recipient
is local (recipient-local-p true against *local-domains*, so the message is
delivered into a local inbox and never relayed), or the message carries
authenticated-submit provenance, an owner who stood at /mail/outbox under a
keyed session. A peer-landed message (non-authenticated provenance) for a
non-local recipient matches neither path: route-recipient signals
mail-open-relay-refused and lands nothing, never silently forwarding.
What this guarantees is that valis cannot become an open relay. Why it holds by
construction: there is no allow-relay flag to misconfigure. Relay authority is a
capability the submitter either presents or does not; the refusal is the absence
of that authority, not a policy toggle a host could flip. An auditor confirms the
guard is live, not vestigial, in
unauthenticated-non-local-relay-is-refused-fail-closed.
The content boundary: the body is an opaque block
A message body is an opaque block, content-addressed octets the store names by
the hash of their bytes and never interprets, exactly like every other block in
the durability cluster. Sovereignty here is absence: the operations-facing view
of the mail spine has no name for any body. A read grant on /mail/queue resolves
a body-free projection (src/namespace/mail-queue.lisp → %synthetic-name) that
exposes envelope and status only; it has no walkable name for any message body and
none for /mail/inbox. The body score that keys a status record is an address,
never invertible to content.
The table below audits the operations paths an operator would use to move or observe the mail spine. None of the backup or observability subsystems named here exists yet: the audit covers the seam each would build on and shows that the seam reaches octets, not meaning. The store handles opaque octet vectors and is encryption-ready, not encrypting.
| Operations path | Seam it would use | Why it reaches no content | Anchoring test |
|---|---|---|---|
| Observe the mail queue (operator) | the /mail/queue body-free projection (%synthetic-name) |
Names envelope + status only; no walkable name is a body, and none is /mail/inbox |
ops-queue-grant-resolves-no-message-body-name |
| Any ops path over the queue projection | the queue projection package's imports | Imports zero body-octet read (read-block / decode-file): a confused deputy is impossible |
ops-queue-projection-imports-no-body-read-symbol |
| Back up the mail store (subsystem TBD) | the block-device / head / tree codecs | Copies hash-named opaque blocks; no content or key import path | store-cluster-imports-no-content-symbols |
| Observe / meter mail flow (subsystem TBD) | the queue projection's status records and counters | Reads envelope, status, and counts; the body score is an address, not content | ops-grant-on-queue-has-no-name-for-the-other-leaves |
| Relay-drain a message (the transport itself) | drain-active-once → a registered mail-adapter |
Hands opaque octets to the wire adapter; classifies and moves, never decodes for ops | authenticated-relay-submission-drains-to-the-stub-adapter |
Validation coverage: the mail transport spine
Each success criterion of the mail spine is anchored to a named test. The
end-to-end tests drive the real start-fabric seam, outbox capability, router,
queue, and delivery over a durable store with only the wire engine stubbed; the
sovereignty tests prove the content boundary both behaviorally (over a really
queued message) and by construction (a do-symbols import firewall).
| Criterion | Test | File |
|---|---|---|
| A local-recipient outbox submission delivers into the owner inbox byte-identical | local-recipient-submission-delivers-into-the-owner-inbox |
tests/mail-end-to-end-test.lisp |
| An authenticated non-local submission drains to a stubbed adapter; the slot frees | authenticated-relay-submission-drains-to-the-stub-adapter |
tests/mail-end-to-end-test.lisp |
| An unauthenticated non-local relay is refused fail-closed and lands nothing | unauthenticated-non-local-relay-is-refused-fail-closed |
tests/mail-end-to-end-test.lisp |
An ops grant on /mail/queue resolves no name for any message body or the inbox |
ops-queue-grant-resolves-no-message-body-name |
tests/mail-sovereignty-audit-test.lisp |
| The ops-facing queue projection imports no body-octet read by construction | ops-queue-projection-imports-no-body-read-symbol |
tests/mail-sovereignty-audit-test.lisp |
An ops grant on /mail/queue has no walkable name for the other axis leaves |
ops-grant-on-queue-has-no-name-for-the-other-leaves |
tests/mail-axis-test.lisp |
The DNS substrate: authoritative names off the answer hot path
valis terminates authoritative DNS as a first-class axis of the sovereign
namespace, the same way it terminates mail. Two siblings sit between the wire and
the data: the engine that speaks the RFC 1035 binary protocol on port 53 to the
hostile internet (a separate process at the IETF edge), and runciter, the DNS
service module (the logical nameserver that composes answers and holds the
serving index). valis owns the data substrate those answers come from (the zone
data held in PostgreSQL as the operator's system of record, the read seam runciter
loads its index through, and the capability gate that admits a DNS wire module),
and it dispatches a steered connection to the registered service module. valis
holds no nameserver answer logic and no serving cache: it is the authoritative
ndb DATA owner, and runciter is the nameserver. PostgreSQL is the operator's
system of record, never a per-query dependency: runciter answers from its own
in-memory index and keeps serving the last good index across a database outage,
so the names do not go dark.
This chapter is the auditor's entry point to that substrate. It states the data read seam runciter binds over, the narrow answer-composition contract the registered serving handler honours, traces a decoded query end to end through the named code, locates the availability and answerability invariants (now runciter's, over the cutover facets valis owns), and shows the capability gate a DNS wire module passes before it can bind. The wire is stubbed for the proof: what is proven is the substrate's end-to-end movement of an answer, not encoding onto a real network.
The seam contract: a decoded query, a record set, and a result code
The contract a bound DNS wire engine sees is deliberately narrow and versioned.
The engine hands a decoded query (a name, a type, and a class) to the
registered serving handler, and the handler returns a record set and a result
code. That handler is runciter's: valis registers (make-dns-service-handler
<pg-zone-source>) as the adapter handler, so the answer is composed by the
nameserver over valis's data, not by valis. The result code is one of three
keywords: :noerror (the record set is the answer), :nxdomain (the name does
not exist), or :nodata (the name exists but carries no record of the asked
type). The contract keeps :nodata distinct from :nxdomain because they are
different facts about the zone, and a correct resolver caches them differently;
the wire maps :nodata onto a NOERROR message with the authority section carrying
the zone's apex SOA.
Negative answers carry authority data, not just a code. On both :nxdomain and
:nodata, the handler returns the zone's apex SOA record in the record set, so
the wire has everything it needs to emit a spec-correct negative response and to
compute the negative-cache lifetime as the minimum of the SOA MINIMUM field and
the SOA record's own TTL (RFC 2308). A name outside every authoritative origin is
:nxdomain with an empty record set (not-authoritative, no apex SOA), kept
distinct from an in-zone negative. runciter owns what authority data accompanies
each outcome; the wire stays a pure codec that encodes what it is handed.
Name synthesis is the nameserver's job, never the wire's. For an aliased name the returned record set already contains the CNAME record plus any in-zone records the alias resolves to (one in-zone, single-hop chase); for a name covered by a wildcard owner the returned set is the synthesized answer. The wire encodes what it is handed and never chases names or reaches below the seam for zone data. Everything that is purely a property of the wire stays wire-side and is never surfaced at the seam: EDNS0/OPT and the DO bit, truncation, response-rate limiting, recursion flags, the message ID, name compression, the authoritative-answer bit, and the transport-generated result codes (FORMERR/SERVFAIL/NOTIMP/REFUSED). Delegations and DNSSEC are not in the v1 contract; they are additive later through a contract-version bump, gated by the same fail-closed version check described next.
The version check is fail-closed at register time. src/plugin/dns-adapter.lisp
holds the seam: the *dns-adapters* registry, register-dns-adapter, and a
contract-version constant +dns-adapter-contract-version+ (1) with the set of
admitted versions in *supported-dns-contract-versions*. An engine that declares
a contract version outside that set is refused before any registration mutates the
table: register-dns-adapter signals dns-adapter-version-mismatch and binds
nothing, so a wire engine built against a future, incompatible contract can never
silently skew against this core.
The data read seam: valis's PostgreSQL zone source
valis exposes its zone data to the nameserver through a narrow read protocol, not
a serving cache. pg-zone-source (src/operator-state/zones.lisp) implements
runciter's zone-data-source protocol over the dns_zone / dns_record rows:
zone-version (the zone's SOA serial as an opaque RFC-1982-comparable token),
zone-record-set (the runciter resource-record structs of a zone), and
lookup-records (records owned by a name). Two further reads let the nameserver
build its serving index and gate answerability without a serving view living in
valis: source-origins (the held-inclusive set of canonical origins valis holds
authoritatively now) and zone-cutover-status (a zone's (cutover-state,
expire-deadline): the facets a secondary's answerability turns on). Each method
queries the ambient pooled connection, so a caller wraps a read in
with-operator-state-connection; the methods open no per-query socket on any hot
path because the hot path is runciter's, not valis's.
The in-memory serving index and the answer composition live in runciter, the
nameserver. runciter loads its index from this read seam, holds it as an immutable
snapshot, and composes every answer (the RFC-2308 decision tree: exact match,
one in-zone single-hop CNAME chase, single-label *.<parent> wildcard synthesis,
NODATA-versus-NXDOMAIN against an existence set, apex SOA on in-zone negatives,
no apex SOA on a not-authoritative name) over its own snapshot. valis composes no
answer and caches no zone. This is the Option-A split: the serving cache lives
with the nameserver; valis is the authoritative DATA owner over PostgreSQL.
Availability is runciter's, by construction over this seam. runciter rebuilds its
index build-then-swap under a lock and serves last good: a rebuild whose source
read raises (PostgreSQL unreachable, a row malformed) is contained, leaving the
prior snapshot answering, so a database outage degrades to serve-last-good rather
than to an empty answer. PostgreSQL is the operator's system of record, never the
per-query dependency. valis's contribution to that guarantee is the data seam
itself plus the cutover facets the nameserver gates on; the serve-last-good
property is verified in runciter's suite (failed-rebuild-keeps-prior-index,
source-outage-after-load-keeps-answering).
The commit fires a change signal; the nameserver pulls
valis installs nothing into a serving view on commit: it owns no view. The
capability-gated :names door (src/namespace/names.lisp) takes a zone's master
text and commits an atomic full-zone replace at the 9P clunk: node-close on the
submission file is the commit, exactly as the mail outbox clunk is. After the
import resolves the new serial, the commit fires the in-process zone-change signal
(fire-zone-change, origin and serial) so an in-process listener (the outbound
zone-change feed, and a wire feed) observes the change. The nameserver refreshes
by pull: runciter re-checks zone-version per origin (and source-origins for
set changes) on a throttled, query-triggered poll and rebuilds its index when the
version is newer, so a committed write is picked up without valis pushing into
runciter. The committed write is durable in PostgreSQL regardless; runciter's next
successful pull reflects it. (A lower-latency optional force-refresh seam exists on
the runciter side, refresh-dns-service, that a commit hook may call; valis
leaves it unwired by default, the throttled poll being the correctness floor.)
The record boundary: valis owns the envelope, runciter owns the rdata
valis stores zone records in typed PostgreSQL columns (owner, TTL, class, type,
and the record's rdata as presentation text) and is the authority on that
envelope. It is not the authority on the per-type rdata grammar. Parsing,
canonicalizing, and rendering the rdata of each record type is runciter's job, the
sibling that owns the DNS protocol knowledge. Only the rdata crosses the boundary:
on the write path record->rdata-columns renders a record's rdata to canonical
text through runciter:render-rdata, and on the load path row->resource-record
(src/operator-state/zones.lisp) builds the resource-record envelope from
valis's own typed columns while handing the rdata text to runciter:parse-rdata
to type it. valis re-parses no resource-record syntax of its own. This keeps a
single authority for the wire grammar (the same authority the nameserver composes
through and the real wire engine encodes through), so the bytes the nameserver
serves and the bytes a zone transfer would carry cannot drift apart. The published
master-file text an operator
reads back is rendered from these same structs through runciter's canonical-line
renderer, so a round trip through the substrate is byte-faithful.
Admission: a DNS wire module is a capability decision
A DNS wire engine does not bind by being named in a config file; it binds by
passing the same two-signature admission gate every other pluggable
module passes. admit-and-bind-dns-module (src/edge/dns-controller.lisp)
delegates admission whole to the module-admission chokepoint (the module's
identity, the publisher's vouch, the owner's :admit signature, and the
revocation/fence check) and introduces no new
trust path of its own. Only on a successful admission does it bind the adapter
over valis's data source; an unvouched or revoked hash is refused fail-closed, the
module loader is never called, and the adapter registry is left byte-identical to
its pre-call state.
The bind registers the nameserver's serving handler: bind-dns-view
(src/edge/dns-controller.lisp) takes valis's pg-zone-source, builds runciter's
handler over it via (make-dns-service-handler <source>) (runciter's builder
symbol resolved late by find-symbol in its package, the same late-binding idiom
the serving seam uses, so valis carries no hard import of a serving symbol), and
registers that handler under the adapter name. The serving contract version is
asserted equal to valis's +dns-adapter-contract-version+ at bind, fail-closed:
a missing builder or a version skew signals rather than binding a wrong-shaped or
absent handler. Because the handler holds its own source and pulls its own
refresh, the bind is a single registry mutation (there is no separate serving
view to offer), and it preserves the no-partial-bind discipline on that mutation:
a post-bind self-check funcalls the handler once, and on any failure
bind-dns-view restores whatever adapter held the bound name before the attempt
rather than blind-deleting it, while admit-and-bind-dns-module restores the whole
registry it snapshotted before admission. The result code from a refused admission
is the absence of a binding, not a half-built one. Handler construction runs
inside a pooled operator-state connection when the pool is armed (the eager index
build reads PostgreSQL) and bare otherwise, so an in-memory test bind needs no
database.
A decoded query therefore traces these seams in order: the wire adapter is fetched
from *dns-adapters* by name and its handler invoked with (name, type, class);
the handler is runciter's serving handler, which composes the answer over its own
index (loaded from valis's pg-zone-source) and returns (record-set, rcode);
the adapter hands that back to the wire for encoding. No port 53 and no binary
codec runs inside valis on that path, and no answer logic runs inside valis at all:
the proof drives the whole chain through the registered runciter handler in
dns-bind-resolves-query-through-runciter-handler, so what is exercised is the
production wiring through the real nameserver, not a view in isolation.
Forward scope: what is stubbed and what is open
The binary wire engine (the RFC 1035 codec, port-53 serving with response-rate
limiting and non-recursion, zone-transfer and NOTIFY, and the resolver client) is
the sibling DNS engine's responsibility and is stubbed here; the conformant wire
transcript is that engine's acceptance gate, not this substrate's. The nameserver's
index warms at handler construction (an eager build over source-origins +
zone-record-set) and is kept current by its throttled pull; valis's job is to
keep the data seam answering and to fire the change signal on commit. Deeper
wildcard closest-encloser synthesis, delegations with glue, and DNSSEC signing are
additive through a contract-version bump and the fail-closed version check,
deliberately out of the v1 flat-authoritative-zone scope.
Validation coverage: the DNS substrate
Each property is anchored to a named test. valis-side coverage proves the data
read seam, the cutover facets answerability turns on, the capability gate and its
no-partial-state unwind, and that the bind dispatches a decoded query through the
real runciter handler. The answer-composition decision tree and the
serve-last-good availability property are runciter's (composition relocated to
runciter's compose-answer parity suite [parity-checked branch-for-branch against
valis's prior zone-view-lookup before the shed], serve-last-good to runciter's
refresh suite) and are cited below so the auditor can follow the data/answer split
to its other half.
valis-side:
| Criterion | Test | File |
|---|---|---|
The PG source surfaces a held zone's cutover-state as :held |
held-zone-surfaces-held-state |
tests/cutover-status-source-test.lisp |
| A secondary surfaces its persisted absolute epoch expire-deadline | answer-as-secondary-surfaces-absolute-epoch-deadline |
tests/cutover-status-source-test.lisp |
| A primary forces a NIL deadline (it never expires) | primary-zone-forces-nil-deadline |
tests/cutover-status-source-test.lisp |
An unstored origin reads (nil nil): fail-closed |
unstored-origin-fails-closed |
tests/cutover-status-source-test.lisp |
source-origins includes held zones (the held-inclusive authoritative set) |
source-origins-includes-held-zones |
tests/cutover-status-source-test.lisp |
A held secondary commit reads :held and so is never answerable |
held-secondary-commit-reads-held-cutover-status |
tests/secondary-ingest-test.lisp |
| A vouched DNS module admits through the chokepoint and self-registers | vouched-dns-module-admits-and-self-registers |
tests/dns-admission-controller-test.lisp |
| A refused module leaves no partial adapter state | refused-dns-module-leaves-no-partial-state |
tests/dns-admission-controller-test.lisp |
| The controller-bound adapter resolves a query through the runciter handler | controller-binds-admitted-adapter-and-resolves-query |
tests/dns-admission-controller-test.lisp |
| A bind failure reverse-unwinds to the exact pre-attempt registry | forced-bind-failure-restores-pre-attempt-state |
tests/dns-admission-controller-test.lisp |
| An adapter declaring an unsupported contract version is refused at register time | dns-adapter-version-fail-closed |
tests/dns-adapter-registry-test.lisp |
| A decoded query resolves end to end through the registered runciter handler | dns-bind-resolves-query-through-runciter-handler |
tests/dns-adapter-registry-test.lisp |
Relocated to runciter (the nameserver's suite), cited for the auditor:
| Criterion | Home |
|---|---|
| The full RFC-2308 answer-composition decision tree (present/absent/NODATA/NXDOMAIN, CNAME chase, wildcard synthesis, not-authoritative) | runciter compose-answer parity suite (parity vs valis's prior zone-view-lookup) |
| A failed rebuild keeps the prior index answering (serve-last-good) | failed-rebuild-keeps-prior-index (runciter tests/dns-service-test.lisp) |
| A source outage after load keeps the nameserver answering | source-outage-after-load-keeps-answering (runciter tests/dns-service-test.lisp) |
The substrate and the engine: where the cutover logic lives
The primary-zone story above is valis answering names it authored. The substrate
also answers names it did not author (a legacy zone fed in from an upstream master
during a migration onto valis) and feeds its own zones out to downstream
secondaries. That secondary operation is a clean split between two owners. valis owns
the durable state (the zone rows, the provenance and lifecycle facets, the
crash-safe timer deadlines) and the operator door that flips a zone through its
lifecycle. The sibling library runciter owns the pure logic (whether a
transfer is admissible, what the served-type set is, how an SOA refresh-timer
advances, which cutover transitions are legal, and whether an entry may answer
right now) and the serving index that answers from it. valis never re-implements
any of that.
valis reaches the logic through a single late-binding seam,
src/operator-state/runciter-serving-seam.lisp, and never through a runciter symbol
directly. The seam declares the contract (accept-zone-transfer,
advance-refresh-timer, cutover-transition-legal-p, may-answer-p,
effective-status, out-of-set-records) and resolves each real runciter symbol at
call time, delegating directly. runciter is a hard dependency that delegates every
target live, so the seam carries no valis-side fallback: a pinned-but-missing symbol
fails closed (runciter-serving-unavailable) rather than serving a stale local answer.
The point of the seam is that the DNS-semantic vocabulary has exactly one home: the
secondary door and the cutover door speak to this package, so the day a newer
runciter ships a sharper policy the substrate picks it up with no caller change and
no second copy of the rule to drift.
The general naming database: a zone is the first record class
:names is not a DNS table that happens to be called a naming database: it is a
general naming database (a Plan 9 ndb analog) whose first record class is a DNS
zone. The cross-cutting facets that describe any named entry (who controls it versus
who authored its data [=authority-state=, provenance-kind=], the lifecycle position
[=cutover-state=], the monotonic version, and the crash-safe SOA refresh-timer
state) live on the =ndb-entry anchor (src/operator-state/ndb-entry.lisp), keyed
by (name, kind) and agnostic to the record class. The DNS-typed dns-zone
projection is the typed face of those facets for kind "dns-zone": its origin is
the anchor's name.
This is a deliberate altitude choice, not incidental layering. Modelling provenance,
write-authority, and version at the ndb-general altitude rather than as DNS-only
columns means a later record class (an identity or federation entry) attaches to
the same anchor with no schema migration. A one-time backfill seeds every existing
zone as a primary / operator anchor, so the general model is populated from the
DNS class that motivated it without a flag day.
The cutover lifecycle: held, answering, primary (operator-gated)
A migrated zone moves through three lifecycle states (:held,
:answer-as-secondary, :primary), and valis flips it only on an explicit operator
write. The two forward transitions ride the capability-gated :names /ctl control
surface (src/namespace/names.lisp): an axis-level declare-secondary verb
bootstraps a cold held zone (creating the held shell and the master allowlist entry in
foreign-key order so the very first transfer can self-authorize), then a per-zone
answer-as-secondary verb makes the zone answer from the upstream's data, and finally
promote-to-primary makes valis the system of record for it. Each verb is an ordinary
owner-gated 9P write whose clunk is the commit, exactly like a zone import.
Legality is runciter's, not valis's. commit-cutover-transition asks
cutover-transition-legal-p through the seam whether the requested flip is a legal
forward step from the persisted state; an illegal or out-of-order flip signals
illegal-cutover-transition and persists nothing, and valis never advances a zone's
state on its own initiative. A held secondary never answers: it surfaces
:held through zone-cutover-status, and the nameserver refuses to answer it; the
gate that enforces this is structural, described next, not a convention each caller
must remember.
Answerability and expiry: the safety core
Two invariants keep a non-authoritative zone from ever answering when it should not, and under the data/answer split they straddle the seam: valis owns the facts, runciter owns the gate, and the flip door owns a write-time refusal.
The data valis owns is the cutover-status pair. zone-cutover-status surfaces a
zone's (cutover-state, expire-deadline): :held for a not-yet-answerable
secondary, :answer-as-secondary with its absolute epoch-second expire-deadline,
or :primary with a NIL deadline (a primary never expires; valis is the system of
record). The deadline is persisted as an absolute instant (the ndb-entry timer
columns, written through persist-refresh-timer-state), never an uptime offset, and
the read is fail-closed: an unstored origin, and a secondary with no recorded
deadline, surface no answerable deadline. The nameserver gates on this (runciter's
may-answer-p treats a :held zone, an expired secondary, or one with no deadline
as not answering, and an expired owning zone as not owning the name at all), so a
:held or expired secondary cannot answer regardless of which path loaded it. These
choices are load-bearing: a restart must not silently extend a secondary's lifetime
past the expiry it inherited from its last transfer, and a master gone unreachable
must eventually cause the secondary to stop answering rather than serve stale data
forever. The valis side of this guarantee is verified in
tests/cutover-status-source-test.lisp (the :held, absolute-deadline, primary-NIL,
and fail-closed cases); the gate itself is runciter's may-answer-p parity.
The flip door adds a write-time refusal: a flip to :answer-as-secondary is
refused (secondary-timer-not-ready) unless a populated, non-expired deadline
already exists, checked against the live clock (*zone-view-clock*, the surviving
serve-time absolute clock in src/operator-state/zone-view.lisp, Unix
epoch-seconds), so a secondary can never even be made answerable with a stale or
missing timer.
The outbound zone-change feed: one signal off the shared commit
valis is also a source of zones for downstream secondaries, and that outbound side
is pure in-process data. src/operator-state/zone-feed.lisp holds an in-process
listener registry (*zone-change-listeners*, mirroring the DNS-adapter registry with
the same register-time fail-closed contract-version check) that fires off the shared
zone-commit point: commit-zone-records invokes its post-commit hook after a durable
commit, and the feed installs fire-zone-change onto it. Because both the operator
text-import path and the secondary-ingest path converge on that one door, a committed
change on either path emits a single change signal with no per-door wiring; a cutover
flip emits the same signal from its own commit point. A downstream secondary-peer
allowlist (zone-feed-peers, the secondary_peer table) gates the data side, so the
feed never offers a zone to an un-allowlisted peer.
The registry is fault-isolated by construction. fire-zone-change runs each listener
under its own handler so a listener that signals is warned and skipped, its siblings
still fire, and the error never unwinds out of the post-commit hook: an out-of-tree
consumer throwing cannot turn a durably committed transfer into an apparent failure.
This is strictly the data side: whether a given change warrants an actual wire NOTIFY,
and the binary AXFR-out that carries the zone, are the sister DNS engine's concern,
not the substrate's.
Boundary hygiene: what this substrate is not
The secondary substrate holds no wire and no secret. There is no port-53 serving loop,
no binary AXFR codec, and (by construction) no transport-secret or TSIG column
anywhere in its schema: every reference it stores is a public master or peer name or
address, and the transport secrets that authenticate a real transfer are the
factotum's domain (mercer), never :names's. The record boundary is the same one the
primary path observes: the typed envelope (owner, TTL, class, type) is valis's, the
per-type rdata grammar is runciter's, and only the rdata text crosses between them. A
record whose stored rdata is malformed fails closed at the node read boundary:
%render-zone maps runciter's typed rdata-parse error to a clean 9P protocol error
scoped to that one zone, rather than letting an arbitrary Lisp error escape past the
read into the serve path.
Validation coverage: secondary operation and cutover
Each guarantee of the secondary substrate is anchored to a named suite. Together they pin the authorization boundary, the held-never-serves invariant, serve-time expiry, the outbound feed's fault isolation, restart-safe deadlines, and the read-path typed guard.
| Suite | Property it pins |
|---|---|
tests/secondary-ingest-test.lisp |
A transfer is authorized on the master allowlist fail-closed (never the session); an out-of-set type is refused and the offender named; a held secondary commits and reads :held cutover-status (so the nameserver never answers it). |
tests/zone-serial-test.lisp |
RFC 1982 serial monotonicity on both the text and the secondary path: an advancing serial is honoured, a stale one refused as a regression, an equal one auto-bumps. |
tests/operator-state-fence-test.lisp |
A secondary ingest under a superseded generation writes nothing: the split-brain fence stops the write. |
tests/operator-state-zone-audit-test.lisp |
The ndb-general facet columns pass the confused-deputy audit and no operator-state table carries a secret column; the existing zone backfills as a primary anchor. |
tests/cutover-lifecycle-test.lisp |
The held -> answering -> primary flips are operator-gated and observable; an illegal or unknown verb is refused; a non-owner cannot flip; declare bootstraps a cold transfer. |
tests/secondary-timer-test.lisp |
Timer deadlines persist as absolute instants so a restart does not extend expiry; a held secondary never answers; a missing anchor signals rather than dropping the deadline. |
tests/cutover-status-source-test.lisp |
The cutover facets the nameserver gates on: a held zone reads :held, a secondary surfaces its absolute expire-deadline, a primary forces NIL, an unstored origin is fail-closed. (The serve-time gate over these [held/expired/no-deadline never answers] is runciter's may-answer-p parity.) |
tests/zone-feed-test.lisp |
The change signal fires from both ingest paths; a throwing listener is isolated from its siblings and the durable commit; only an allowlisted peer is fed. |
tests/names-import-test.lisp |
An operator zone imports, lands rows, reads back, and re-imports/refuses under the shared commit door; a fenced write commits nothing. |
tests/names-axis-test.lisp |
Writes are owner-gated while reads are open; a malformed master file and a corrupt stored record each return a clean Rerror at the node boundary. |
Privilege: unprivileged valis, external steering
The eBPF program load, the sockmap population, and the netns attach are privileged operations performed by a host agent external to valis. valis itself runs unprivileged and simply serves on the listening socket it is given. Holding no privileged kernel state is the same property that makes valis migratable: there is nothing host-specific for it to carry.
The one host-coupled obligation is a contract: a valis unit declares the designated ports it answers, and the host's privileged agent undertakes that those ports reach the unit's socket. It does not undertake to steer only those. The steer is a catchall and carries every port at the unit's address, so the narrowing is the agent's firewall rather than the declaration (the public surface). That contract is what "valis-compatible infrastructure" concretely means, and on migration it is the destination host's agent that re-establishes both halves so the arriving unit answers its ports again.
The IETF edge and the sovereign core
A core promise of the doctrine is that a person's valis answers the existing internet protocols on their designated ports (SMTP on 25, HTTP on 80 and 443, DNS on 53, IMAP on 143) and services them as a faithful, standards-compliant server. Those callers are the anonymous, hostile internet; they know nothing of valis's namespaces, capabilities, or 9P. This sits in natural tension with valis's evolution toward its own identity-first, capability-scoped, 9P-native services. valis resolves the tension by carrying the split inside one migratable unit, as two tiers that meet at the namespace:
- The IETF edge is port-addressed. A connection's recovered destination
port selects the protocol module that answers it: this is exactly what the
registry (
src/registry.lisp) does. Edge modules face hostile input and are anonymous-first: a foreign peer with no proven identity is the anonymous principal, reaching a deliberately narrow capability-scoped namespace (an inbound mail spool, a public file view). - The sovereign core is name-addressed. Native services are reached by 9P paths in the namespace, never by port. They are identity-first and grow as valis evolves.
The IETF protocols are first-class native modules, not a compatibility skin bolted onto a "real" inner system: keeping them first-class is how the doctrine's promise is kept. An edge module reaches the core only by carrying its connection's principal and capabilities into the namespace, exactly like any other module; the tier distinction is one of trust posture and hardening, not of mechanism.
The seam is 9P-shaped, so isolation is a transport choice
The hostile edge is the part eating untrusted input, so the long-term target is to run edge modules as sandboxed OS processes that reach the core only over 9P with their narrow capabilities: a compromised edge process then holds nothing but its own anonymous-scoped authority. The near-term implementation keeps edge and core co-located in one image with a logical boundary (fast to build), hardening the anonymous path first.
The durable commitment that makes this an evolution rather than a rewrite is that the edge↔core boundary is a 9P/capability interface from the start, never a shared-memory assumption. Co-location is then 9P over an in-process transport; isolation is the same 9P over a pipe between processes. This is the same discipline applied to the event multiplexer (epoll now, iouring later) and to module placement (local thread or remote node): design the seam at the eventual boundary, implement the cheap version first.
The edge↔core seam contract
The seam is a single module (src/edge/seam.lisp) and it is the only place an
edge module reaches the sovereign core. An adapter never touches the namespace
directly; it asks the seam for the view its connection is entitled to and walks
that. The entitlement is the connection's principal and its request-scoped
capabilities, carried across a real 9P interface: the same attach, walk, and
read a remote 9P client performs, with no privileged shortcut.
The seam holds one live attach per principal class, not per connection. Every anonymous connection shares a single cached attach to the public view, because they are all the same principal and opening a fresh 9P session per request would be ruinous on a busy port; a keyed principal, when keyed edge service arrives, gets its own attach through the identical mechanism. The cache key is the principal class rather than the principal object, because the anonymous sentinel and a keyed principal are the same Lisp type: only the class of authority distinguishes the slots.
A second, deliberately separate path serves the request-scoped phase: the
with-request-capability entry point mounts a capability surfaced inside a
request into a fresh private per-request view, runs the request body bound to
that view, and unmounts on the way out, including on a non-local exit. The
private view is not the shared cached attach, and the reason is a concrete leak
it forecloses: mounting into a shared view would make one connection's
request-scoped grant briefly visible to every other connection on the same
principal. A per-request view is structurally incapable of that cross-connection
disclosure: the mount lives and dies inside one request's dynamic extent.
The transport under the seam is a rebindable factory: *edge-channel-factory*
is a function returning the two endpoints of a channel, defaulting to an
in-process channel served against the assembler root. Rebinding it swaps the
transport with no change to any adapter: the co-located case (an in-process
channel), a TCP loopback to the fabric's own 9P listener, and a real
inter-process socket pair have all been driven through this one point, each
serving the same publication byte-for-byte. An adapter names no transport symbol
at all; it imports only view-for from the seam. That import boundary is what
makes "co-located now, sandboxed later" a binding rebind rather than a rewrite.
The anonymous-grant lifecycle and self-healing
The public edge runs on exactly one bearer grant. At fabric start
(src/fabric.lisp), start-fabric mints a single edge-wide capability name
designating the public view with read-only rights and installs it as the
anonymous grant; the seam attaches against it on first use. There is no
per-adapter grant and no per-connection minting: one name is the entire public
authority, and an auditor can find its single mint site and its single
designation.
Because that one attach is long-lived, the seam has to survive losing it, and it discriminates why the view was lost before deciding what to do. The discriminator is membership in the revocation store. If the grant's hash is in the store, the loss was a deliberate unpublish: the view goes dark and stays dark (walks fail, adapters serve absence) until the operator re-establishes the public view. The seam never silently re-mints a revoked grant. Any other loss (a closed session, a dropped channel, an expired grant) self-heals: the heal path consults the revocation store first and re-checks it on every retry (so a revocation landing mid-heal still wins), re-attaches with a bounded inline retry, and re-mints through the grant's mint hook only on genuine expiry. The re-mint can therefore never resurrect a revoked grant, because the store is consulted before the mint is reached.
Sovereignty on the wire is the absence of a name, not a policy refusal. When an anonymous caller names anything outside the public view, the walk simply fails, and the adapter surfaces that failure as the protocol's own absence response: an HTTP 404-class answer, a Gopher type-3 error item. There is no "permission denied" code path at the edge to disclose that a thing exists but is forbidden; the owner's axes have no name in the anonymous view, so the only honest answer is that they are not there.
Edge wiring and the fail-closed bind order
The edge controller (src/edge/controller.lisp) binds and unbinds the edge
ports through start-edge / stop-edge, riding the listener's already-running
event loop rather than owning a listener of its own. Dispatch is purely
port-addressed: a connection's recovered destination port selects its protocol
module through the registry, exactly as the central abstraction promises.
The bind order is fail-closed, and the gate is a real resolution, not a presence
check. Before start-edge binds a single port it asserts the fabric is up, the
anonymous grant is wired, and a real view-for for the anonymous principal can
attach and walk the public view through the seam. A grant string that decodes but
no longer resolves (revoked, or pointing at a view that has gone dark) fails
this gate, so the edge refuses to come up serving a public face that cannot
actually be reached. No namespace means no open port. Binding a port here opens it
only for dispatch, though: whether that port is reachable from off the host is a
separate, privileged decision at the host agent's firewall, not a consequence of
the edge binding it: see The public surface. Teardown runs in strict
reverse: stop-edge closes the ports before the seam resets, so the wire stops
feeding a session before that session is torn down, and a mid-start failure
unwinds whatever bound in reverse order.
The one construction point a later phase rebinds is named and isolated:
*edge-source-constructor* builds each port's event source, defaulting to a
plain bind, and is the seam where a steered file descriptor handed over by the
privileged host agent arrives. The resident boot already rebinds it
(src/main.lisp), so the edge binds a source around a passed-in fd instead of
opening its own, with nothing else in the controller changing. Alongside the wiring, the controller publishes a read-only /edge
status subtree (src/namespace/edge.lisp): a directory per bound port carrying
live connection, budget, and deadline-cut counters rendered fresh on every read,
plus an anonymous file reporting whether the public view is unwired, dark,
undetermined, or serving. Undetermined is kept distinct from dark because the
two want different repairs: dark means the grant was found revoked and is
answered by minting a fresh one, while undetermined means the node could not
establish whether the grant is revoked at all, refuses to serve on that, and
would be no better off with a fresh grant. The subtree is observable as
ordinary files, and it is named only in the
owner's canonical frame: an anonymous attach cannot walk to /edge at all, so
the diagnostic surface is itself sovereign.
The edge abuse posture
The edge faces the open internet, so abuse defense is layered, and the layering is deliberate: each defense lives at the tier that can express it most cheaply, and none of it lives in the sovereign core.
At the adapter tier, each connection carries a read deadline and a hard cap on input size: a client that connects and sends nothing, or dribbles bytes slower than the deadline, cannot hold a handler thread, and an oversized request line is refused rather than buffered. At the edge tier, each bound port carries its own concurrent-connection budget. When a port reaches its budget the controller stops accepting on that port: it deregisters the listening source, so overflow connections wait in the kernel's listen backlog and no valis thread or write is ever spent refusing an over-budget client. A connection's whole lifetime is wrapped in one place in the controller (never in adapter code), so the live count is decremented on every exit path (normal, error, or deadline cut) and the port re-arms its accept as soon as a slot frees. A flood on one protocol's port cannot starve another, because the budgets are per-port. The third tier (per-IP tracking, rate limiting, and volumetric or amplification defense) belongs to the privileged host agent that sits in front of valis, outside the process entirely.
The rationale for the split is the sovereignty invariant. Per-IP and volumetric defense need a view of the network the unprivileged, migratable valis process deliberately does not hold, and the sovereign core must carry no abuse logic at all: the core moves and mounts views, it does not reason about hostile traffic. Pushing the outermost defenses to the host agent and keeping only connection-occupancy and per-port budgets inside valis keeps the core clean while still bounding what a single open port can consume.
Connection lifetime: what the edge assumes, and what it no longer does
Nearly everything the edge serves today is request shaped: a client connects, says what it wants, is answered, and goes away. The connection exists to carry one exchange and has no reason to outlive it. For that traffic the shape described above is the right one and is deliberately unchanged. A thread per connection is cheap when the thread lives as long as one short exchange; a per-port concurrency budget is a true measure of work in flight when every live connection is work in flight; and a wall-clock deadline is a sound liveness test when silence past the deadline means a client that was never going to finish.
A protocol whose connections are long lived, mostly idle, and written to unprompted breaks all three at once, and it breaks them for one reason rather than three. The thread cost, the per-port budget and the one-shot deadline read as separate limits, but each is an expression of a single assumption nobody had written down: a connection lives exactly as long as the request that created it. The table below states that assumption and its relatives plainly and disposes of each. The disposition is rarely a removal. Most are SPLIT: the request-shaped path keeps the shape it has, and a protocol that declares itself continuous gets the other one. A protocol declares which it is; the seam never guesses.
| Assumption | Where it is made | Disposition | Reason |
|---|---|---|---|
| A connection lives exactly as long as the request that created it | src/protocol.lisp:52, where handle-connection runs to completion and the exchange ends with it |
REPLACED | A continuous session is opened by an exchange it outlives, so its lifetime belongs to the session rather than to a handler's dynamic extent. |
| Each accepted connection costs one OS thread for its whole life | src/edge/controller.lisp:312, one thread spawned per accept |
SPLIT | Kept for a request-shaped connection, where a thread that lives as long as one short exchange is the cheapest place to hold its state. Replaced for a continuous session, whose thread is idle for nearly all of its life, so the cost is paid per connection instead of per unit of work. |
| Only a listening socket is an event source | the two source classes in src/backends/epoll.lisp (:350, :456) |
REPLACED | An established connection becomes a registered source as well, which is what lets the loop drive it without a thread of its own. |
| Readiness means a connection is waiting to be accepted | source-ready in src/multiplexer.lisp:171 |
REPLACED | Readiness now also means an established connection has bytes to read or room to write, so one notification serves two kinds of source. |
| A source is monitored for readability only | the :read registration in src/backends/epoll.lisp:268 |
REPLACED | A queued write that could not complete needs writability notification to finish; without it, a slow reader can only be served by parking a thread on it. |
| Output needs no buffering because a handler writes straight to the stream | no output queue exists anywhere in the edge layer: src/protocol.lisp:52 hands the adapter the connection with nothing in between |
REPLACED | A queue is the only place a bound can be enforced, so the queue has to exist before a cap can be placed on it. Without one, a peer that reads slowly is served out of whatever the handler happens to allocate, which is bounded by nothing. |
| Liveness is a wall-clock one-shot armed by the handler | set-read-deadline in src/connection.lisp:249 |
SPLIT | Kept for a request-shaped exchange, where silence past the deadline really does mean a client that will not finish. Replaced for a continuous session, where silence is the normal state and a clock alone cannot separate a healthy idle peer from a dead one; there the exchange itself has to supply the evidence. |
| A live connection is in-flight work, so one per-port count bounds abuse | the default budget at src/edge/controller.lisp:158 |
REPLACED | A session that is idle by design holds a slot while consuming nothing, so a count of live connections stops measuring work and starts refusing legitimate peers long before any resource is scarce. |
| Bytes move through the connection's buffered stream view | connection-stream at src/connection.lisp:193, vending the accepted socket's own buffered stream |
SPLIT | Kept for a request-shaped exchange, where reading ahead is exactly what makes line and header parsing cheap. Replaced for a connection driven from the event loop: the buffered view reads ahead into a userspace buffer, so bytes already drained out of the kernel are invisible to readiness notification, and a connection driven from the loop must therefore never take that view. |
| The worker closes the connection when its handler returns | src/executor.lisp:107 and src/edge/controller.lisp:263 |
SPLIT | Kept where the handler's return really is the end of the connection. Replaced for a session that outlives the exchange which opened it: closing becomes an explicit act ordered after the last queued bytes have left, and the party that closes is whoever ends the session, not whoever ran the last handler. |
| Every adapter can be driven the same way, because one budget executor serves them all | src/edge/tls-serve.lisp:450, src/edge/mail-serve.lisp:286, src/edge/dns-serve.lisp:1005 and the controller's own ports all build the same executor |
REPLACED | An adapter is driven from the event loop only if it positively declares that it drives the raw descriptor and hands nothing above it. Anything that terminates, decodes or wraps the stream keeps the thread shape, because readiness is asked of the descriptor while the wrapper holds transformed bytes above it, and the two disagree. This is a safety property rather than a performance note: on a port where the stream is terminated, driving the raw descriptor would serve the connection in cleartext. |
| A protocol module never holds a socket or a descriptor | the module contract itself, held by every adapter and by no single file | KEPT | This is what makes a module killable and rebuildable from durable state plus a client reconnect. A module that held a descriptor could not be rebuilt without the connection dying with it, and the substrate would stop being movable. |
| Every outbound connection leaves through the one dial verb | the single egress seam | KEPT | Nothing this work adds opens a socket of its own. A continuous session's own reconnection, where it has one, is an outbound connection like any other and leaves the same way, so the egress policy that governs the process keeps governing it. |
The dispositions above leave a set of design questions that each have exactly one answer. They are recorded here so the implementation does not re-decide them one file at a time.
- Where the send-queue cap is tested
- The cap is tested against the length the
queue would have once the message is added, before the message is buffered.
The reason is arithmetic rather than taste: testing the already-queued length
bounds the queue at the cap plus one maximum message, and no maximum message
exists on this path, so that placement bounds nothing at all.
src/transport/client-secure-endpoint.lisp:148is the precedent already in the tree, rejecting an over-long declared length before any body is read. - A message larger than the whole cap
- A single message that exceeds the entire cap is refused outright rather than split across several sends. The reason: splitting does not reduce the bytes the queue holds, and the seam has no producer it can pause part way through a message, so splitting would relabel an unbounded hold rather than bound it. That refusal is reported distinctly from an ordinary over-budget refusal, because an over-budget send can succeed on a later attempt once the queue drains and this one can never succeed at all.
- Who closes on a breach
- Closing a connection that breaches the cap is the edge's act and never the protocol module's. The reason: the queue and the cap belong to the edge, and a component cannot enforce a bound it does not hold. What this leaves with the module is an obligation rather than a power. It must notice that a send was rejected, and must never assume a send lands.
- The cap is not the only backpressure
- A bulk producer can read how full the queue is and suspend itself at a fraction of the cap, which makes the close a last resort instead of the only one. The reason: a producer that can pause is far cheaper to stop than a connection is to lose.
- A registration timeout is not an idle timeout
- These are two different mechanisms and are kept apart deliberately. The registration timeout runs from the moment the connection was established and is not reset by activity, while the idle timeout runs from the last activity and is paired with a protocol-level probe that asks the peer to prove it is still there.
- A session with no liveness probe
- A session whose protocol offers no way to probe the peer gets no idle cut at all, and only the registration timeout applies to it. The reason has to be stated out loud because the opposite is the tempting default: cutting an idle session that cannot be probed reinstates the wall-clock cut on exactly the connection that is idle by design, which is the mechanism this work exists to replace. Such a session ends when its peer goes away or when the edge tears its port down, never on a clock.
- The send path copies
- What a caller hands the send path is copied at the moment it is handed over, and never retained by reference. The reason: the per-session read buffer is reused on the next read, and a short write leaves queued bytes waiting across callback boundaries, so a caller echoing straight from a read into a send would otherwise ship whatever arrived next. The cost is one allocation per enqueued message, and it is reported as a measured per-unit figure rather than left implicit.
- Every sizing constant here is provisional
- Each sizing constant this seam introduces says so in its own docstring, and each is derived from a bound measured in this tree rather than adopted from another implementation. The reason: asymptotic shape is a design-time decision while constants come from profiling this system, so a number borrowed from a daemon carrying different message sizes would be a guess wearing the authority of someone else's measurement.
What the shape costs, per unit
A design that cannot say what one unit costs cannot say how many units a machine holds, and that question was among the grounds on which the dispositions above were chosen. The figures below are what the shape actually costs. They are produced by a named probe in the suite rather than reasoned from the source, and that probe reports every reading whether or not it asserts on it.
| Unit | Measured cost |
|---|---|
| One instance at rest, a loop up and no sessions | 132,603,808 bytes resident |
| Bytes retained per idle session | 4,802, of which 4,096 is the session's own read buffer |
| The same figure gross, with the peer end held in the measuring image | 6,488 |
| OS threads per idle session | 0, against one per connection on the request-shaped path |
| The two liveness clocks a session arms | 504 bytes per session at 16 sessions, 536 at 32 |
| Allocation per queued and drained message, 256 octets of payload | 304 bytes |
At 4,802 bytes a session, ten thousand quiet peers hold about 46 MiB of session state. That is why the per-port ceiling on live sessions is a density decision rather than an abuse bound, and it is the figure a sizing pass should start from.
The per-message figure includes the copy the send path makes of the caller's bytes at the moment it accepts them. That copy is the decision recorded above, not queue overhead, and a sizing pass that read the figure as pure overhead would re-derive a question this document has already answered.
Conditions, without which none of the figures means anything. They were taken
at commit f464a62, in a freshly started image with a warm compilation cache,
nothing else running in it, one process at a time. The instrument's own noise
floor, measured first from repeated readings of the instance at rest, is 256
bytes, and every per-unit figure above stands at least thirty times clear of
it. Each figure is the median of three readings, and each session reading is a
delta against an at-rest reading taken moments before it rather than against a
baseline read at the start of the run. The figures move by tens of bytes
between runs, so a reading differing from these in its last two digits is the
same reading.
- What the probe detects, and what it does not
- It detects a per-session cost that grows worse than linearly as sessions accumulate. That is the defect invisible both to reading the code and to a green suite, and it is the one demonstrated: an injected cost growing with the number of sessions already adopted drove the cost of a doubling to 5.90 and 4.64 times against an asserted ceiling of 3, and the probe went red. It does not detect a constant that simply grew. Raising the per-session read buffer sixteenfold moved the per-session figure from 4,816 bytes to 66,288 and left every ratio at 2.00, so every assertion stayed green while the cost rose by a factor of fourteen. A grown constant shows only by comparing the figures one run reports against an earlier run's, which is why the probe prints every reading rather than only asserting on it.
- How far the probe has been falsified
- The assertions do not all carry the same weight, and the difference matters more than the figures do. Two have been observed failing: the ceiling on the cost of a doubling, and the whole thread-growth control, the latter by adopting the connections so that no worker was ever spawned, which is exactly how the instrument would stop discriminating in practice. The liveness and queued-message bands, and every structural assertion, have only ever been observed passing. Two are weaker still and are named here so that nobody credits them with more than they do. The lower bound on a doubling can fail only if a doubling costs less than 1.25 times as much, which no defect in this seam produces, so it is a rail against a broken measurement rather than a defect detector. And the liveness bound compares a figure against twice one measured moments earlier in the same test, so a defect inflating both counts equally passes it: it catches per-unit growth between 16 and 32 sessions and nothing else.
- What would falsify these figures
- A run of the same probe at the same commit, in a freshly started image with nothing else live, reporting a per-session cost outside the band the measured noise floor allows.
- What is not covered at all
- The cost of a namespace read parked on behalf of a session. That belongs to the namespace layer and was measured there, in an image rather than over a socket held open for a long period, so nothing here says what such a read costs once the connection under it has been quiet for hours.
- Every sizing constant here is still provisional
- None of them has been profiled under load, and the figures above are what a later profiling pass will move them against. The per-session read buffer of 4,096 octets, the send-queue cap at four times the carrier frame ceiling, the suspend fraction of one half, the idle, probe and registration intervals of 120, 30 and 30 seconds, and the per-port ceiling of 400 live sessions are each provisional and each says so in its own docstring. A number recorded with no statement of what it is provisional against reads as settled to the next person.
Validation coverage: the edge↔core seam
Each edge success criterion and security property is anchored to a named test an auditor can run. The acceptance suite drives plain loopback TCP clients against fully shipped wiring; the seam and controller suites exercise the seam internals and the budget machinery directly.
| Criterion | Test | File |
|---|---|---|
| Recovered destination port alone selects the protocol module | port-alone-selects-protocol-module |
tests/edge-acceptance-test.lisp |
| An anonymous caller names nothing outside the public view; absence is the wire answer | anonymous-names-nothing-outside-public-view |
tests/edge-acceptance-test.lisp |
| The same adapter serves byte-identical over a second transport (TCP loopback) | same-adapter-serves-over-second-transport |
tests/edge-acceptance-test.lisp |
| The same adapter serves byte-identical over an inter-process socket pair | same-adapter-serves-over-socketpair-transport |
tests/edge-acceptance-test.lisp |
| Adapters name no transport symbol; the transport split is seam-internal | adapter-code-unchanged-between-transports, adapters-name-no-transport-symbols |
tests/edge-acceptance-test.lisp, tests/edge-seam-test.lisp |
| A request-scoped capability never leaks across connections on the shared principal | request-capability-never-leaks-across-connections, request-capability-unmounts-on-nonlocal-exit |
tests/edge-seam-test.lisp |
| A revoked grant stays dark; loss self-heals; expiry re-mints only after the store check | revoked-grant-stays-dark, view-loss-self-heals, expired-grant-re-mints-after-store-check |
tests/edge-seam-test.lisp |
| The edge refuses to bind without a resolvable public view | start-edge-refuses-without-namespace, start-edge-refuses-dark-anonymous-view |
tests/edge-controller-test.lisp |
| A mid-start failure unwinds every bound port | mid-start-failure-unwinds-bound-ports |
tests/edge-controller-test.lisp |
| Teardown closes ports before the seam resets | stop-edge-closes-ports-before-seam |
tests/edge-controller-test.lisp |
| The connection budget stops accepting at the cap; a queued connection is served when a slot frees | budget-cap-stops-accepting, queued-connection-served-after-re-register |
tests/edge-controller-test.lisp |
| The live count is restored on every exit path and never leaks a slot | counter-decrements-on-error-exit |
tests/edge-controller-test.lisp |
The /edge status surface reports live counts and the anonymous dark state, and is unnameable anonymously |
edge-status-reports-live-counts, edge-status-tracks-anonymous-dark-state, anonymous-view-cannot-name-edge-status |
tests/edge-controller-test.lisp |
The owner-proof flow: a live signature, never a token
The edge faces the anonymous internet, but one caller on the same port is not
anonymous: the owner. The owner reaches a management view by proving possession
of the custody master key on every request, not by presenting a token. Each
request the owner makes is an HTTP Message Signature (RFC 9421): the owner signs
the canonical request material with the same Ed25519 custody key that roots all
valis authority, and the edge verifies that signature against the owner's public
key. This is the deliberate choice that separates owner authentication from the
anonymous grant. A bearer token is replayable by anyone who holds it (which is
exactly the anonymous /pub model, an unscoped grant pointed at a public view),
and pointing a bearer model at the management view would let a token-holder act
as the owner without ever holding the owner's key. A live per-request signature
cannot be lifted and reused, and it carries no server-held session state to leak
or to outlive its purpose: the proof is the key-possession itself, re-demonstrated
each time.
Verification routes through the same single seam the rest of identity admission
already uses. The authenticate generic in src/identity/authenticator.lisp is
the one named operation that turns a completed proof into a principal; the Noise
handshake is the first method on it and the anonymous resolution is the second.
The seam was built reserving exactly a third method for an HTTP-edge proof, and
this is what fills it: a signature authenticator specialises authenticate to
mint the owner principal from a valid signature, with no code above the seam
changing: the connection layer, the capability layer, and the namespace mounter
still see only a principal and never learn what proved it. The verifier itself
lives in src/identity/http-signature.lisp: it reconstructs the RFC 9421
signature base over the covered components (the request method, the request
authority, and the request path) through the same base builder the owner's
signer used, so the bytes signed and the bytes verified are identical by
construction and cannot drift apart into a canonicalization mismatch. It checks
the signature's key identifier against the owner DID and verifies the Ed25519
signature against the owner public key.
The custody boundary holds at the edge exactly as it does everywhere else: the
edge module relays only the signed request material (the method, the authority,
the path, and the two signature header values) to the seam, and never sees a key.
Verification needs only the owner public key, which is already in the image on
the factotum side; the private custody key never crosses into the edge tier, and
no ambient authority is granted to a module that eats hostile input. The replay
defence is twofold and lives with the verifier: a bounded acceptance window over
the signature's created / expires stamps rejects a signature presented too
early, too late, or long after it was minted, and a bounded, expiry-pruned nonce
cache rejects a second presentation of a nonce already seen inside the window. A
signature that is absent, malformed, signed by the wrong key, expired, or replayed
does not raise an error and does not reach management: every such failure resolves
to the anonymous principal, so the management view is reachable only by a valid,
live, first-use signature. Falling open to anonymous, never to an error and never
to elevated access, is the fail-closed posture carried to the auth path.
Identity-selected views on one URL space
Anonymous and owner walk the same URLs on the same port. What differs between
them is not the path they ask for but what is mounted under it. The anonymous
caller resolves to the published view: the /pub scope, the same public face the
anonymous grant has always served. The owner, once a valid signature has resolved
to the owner principal, resolves to the full sovereign canonical frame
(/pub /id /cap /edge /bus) rendered as a browsable read projection. The
discrimination happens at one site: view-for (src/edge/seam.lisp) hands every
non-anonymous principal to build-base-view (src/namespace/assembler.lisp),
which dispatches the owner DID to assemble-canonical-frame server-side; the seam
re-implements no identity comparison of its own. The HTTP adapter then walks an identity-dependent
root: the anonymous request walks under the /pub scope, the owner request walks
the canonical frame directly, because for the owner that frame is the root. The
principal is re-resolved for each request and never cached on the connection, so
under a persistent connection a signed request and a later unsigned request on the
same socket each resolve their own principal and their own view; a signed request
cannot infect an unsigned one that follows it.
This is sovereignty expressed as absence, carried now to the HTTP wire. An anonymous caller does not receive a permission refusal when it names a management axis: it has no name for one. The management names are simply not mounted in the anonymous view, so a walk toward them fails the way a walk toward any non-existent name fails, and the adapter surfaces that failure as the protocol's own not-found answer. There is no code path at the edge that says a thing exists but is forbidden, because such a path would itself disclose that the thing exists. The honest answer to an anonymous caller reaching for the owner's axes is that, in its view, they are not there.
The HTTP/1.1 conformance surface
The published view is served by a real HTTP/1.1 server, not a thin compatibility
skin, so an ordinary client treats valis as an ordinary origin. The surface is
deliberately bounded to the read path. A Host header is required and validated:
an HTTP/1.1 request that omits it is answered 400, per the messaging
specification. GET and HEAD are served, with HEAD returning the computed
headers and no body; any other method is answered 405 with an Allow header.
Connections are persistent, framed by Content-Length, with Date and Server
on every response and the real status taxonomy
(400 / 404 / 405 / 408 / 411 / 505 alongside 200 / 304).
Conditional requests are answered from the substrate's own version stamp: the 9P
qid.version of the addressed leaf becomes a strong ETag, and an If-None-Match
that matches it yields 304 Not Modified with the validator and no body. That
mapping is the substrate-alignment win: an ordinary HTTP cache works correctly
against the sovereign namespace because the namespace already carries the exact
per-node version a validator needs. That version is now derived from the node's
content score (the low 32 bits of its Merkle hash; see the
read-side caching tier), so the ETag is itself a
content hash rather than an opaque counter, the content-addressed norm a strong
validator is meant to be. A body mutation changes the leaf's score, so its
ETag changes with it, and a conditional GET that no longer matches gets a
fresh 200 carrying the new validator. The write-path long tail of HTTP/1.1
(chunked request bodies, =Expect=/100-continue, request pipelining) is out of scope
by design: it bites almost entirely on request bodies and writes, and this is a
read server.
The conformance surface composes cleanly with the edge abuse posture documented above rather than weakening it. A persistent connection holds exactly one of its port's concurrent-connection budget slots for its whole lifetime, while the per-request read deadline re-arms at the top of every request on that connection. The two together give the right behavior under keep-alive: a slow or dribbling client is cut by the deadline on the request it is starving, without a healthy persistent connection being dropped between requests, and the per-port budget continues to bound a flood of connections regardless of how many requests each one carries. The existing guards remain in force on the new surface as well (every response header value passes through the response-splitting guard before it is emitted, and every request path passes through the traversal-safe decode before it is walked), so the broadened method, header, and conditional-request handling adds no new injection or traversal surface.
Validation coverage: the HTTP edge
Each HTTP-edge criterion is anchored to a named test an auditor can run. The conformance cases drive plain loopback HTTP clients against fully shipped wiring; the owner-proof cases drive an in-image RFC 9421 signer over the same loopback; the signature unit verifies the cryptographic round-trip directly.
| Criterion | Test | File |
|---|---|---|
A faithful HTTP/1.1 GET returns a well-formed response over the published view |
ordinary-get-returns-wellformed-http11 |
tests/http-conformance-test.lisp |
HEAD returns the headers with no body |
head-returns-headers-no-body |
tests/http-conformance-test.lisp |
A Host-less HTTP/1.1 request is answered 400 |
missing-host-is-400 |
tests/http-conformance-test.lisp |
| Keep-alive serves two requests on one connection, each re-arming its deadline | keep-alive-two-requests-one-connection |
tests/http-conformance-test.lisp |
Conditional GET (If-None-Match against the qid.version ETag) yields 304 |
conditional-get-if-none-match-304 |
tests/http-conformance-test.lisp |
A GET after a body mutation returns 200 with a fresh, differing ETag |
conditional-get-200-after-mutation-bumps-etag |
tests/http-conformance-test.lisp |
An unsupported method is answered 405 with Allow |
post-returns-405-with-allow |
tests/http-conformance-test.lisp |
| Same port, identity-selected views: anonymous reaches the published view, a valid signature reaches management | same-port-anonymous-vs-signed-views |
tests/http-owner-proof-test.lisp |
| The anonymous byte stream names nothing of the management axes; absence is the wire answer (404, not 403) | anonymous-names-nothing-of-management |
tests/http-owner-proof-test.lisp |
| A valid owner signature reaches the management view | valid-signature-reaches-management |
tests/http-owner-proof-test.lisp |
An unsigned, invalid, expired, or replayed request resolves to anonymous; management is absent and /pub still reads |
unsigned-request-is-anonymous, invalid-signature-is-anonymous, expired-signature-is-anonymous, replayed-signature-is-anonymous |
tests/http-owner-proof-test.lisp |
| The signature base round-trips: the signer and verifier reconstruct identical bytes and a valid signature verifies | signature-base-round-trip, verify-accepts-valid-rejects-tampered |
tests/http-signature-test.lisp |
| The replay window rejects a replayed or stale signature | replay-window-rejects-replay-and-stale |
tests/http-signature-test.lisp |
The steered-fd handoff: explicit consent, not reach-in
The fail-closed bind order above names *edge-source-constructor* as the one
construction point that is rebound to a steered descriptor. That descriptor
arrives by a deliberate trust choice, and the choice is the load-bearing security
property of the whole steering arrangement: valis hands a descriptor out; the
agent never reaches in.
valis creates and owns its listening socket (it is the only party that calls
bind and listen), and the socket reaches the LISTEN state inside valis's own
unprivileged process. To let the privileged host agent steer traffic to that
socket, valis sends the agent a duplicate of that one descriptor as an
SCM_RIGHTS ancillary message over the very Unix-domain connection the 9P
control session already runs on. There is no second channel: the same
AF_UNIX transport carries both the in-band 9P frames and the out-of-band
descriptor. valis chooses exactly which descriptor crosses the boundary, and
nothing else of valis's descriptor table is exposed. This is consent expressed
as a capability: possession of the passed descriptor is the entire grant.
The rejected alternative is what makes the choice legible. The kernel also offers an agent-pull path: the privileged process could open valis's process descriptor and lift a descriptor out of its table directly. That was refused on purpose: it would grant the agent the standing authority to reach into an unprivileged process and take any descriptor, a far broader and more durable trust than "valis decided to send you this one socket." The push model grants the minimum; the pull model grants a reach-in. The seam where the inherited socket lands wraps it without re-binding (an already-LISTEN descriptor cannot be bound again), so the steered source is the same event source the bind path builds, recovering the dialed destination port exactly as the ordinary listener does (see Edge wiring and the fail-closed bind order, and the four-tuple preservation the listener relies on in the steering layer).
The descriptor crossing is also strictly sequenced against the 9P stream, not concurrent with it. valis declares its port set over 9P first; the control service advances a handoff gate to a ready state only after a valid declaration; then the standalone descriptor message crosses while no frame is in flight. The order is what keeps the zero-data ancillary message from racing the length-framed 9P bytes on the shared socket, and it is why the agent never holds a descriptor it has no port contract for.
The per-namespace steering constraint
sklookup selects a socket per network namespace. The kernel runs the steering program only for traffic in the namespace the program's link is attached to, and it will only select a socket that lives in that same namespace. valis's listening socket and the agent's link must therefore share one namespace, or the steer silently never fires: load succeeds, the socket map is populated, the link attaches, and connections are still refused, with nothing in any log to point at.
This constraint is also the unit's boundary of authority, though it bounds what the steer can offer rather than what the unit answers. The steering program is a catchall: it fans every port on the unit's IP out to valis's one socket, so the namespace and the IP it carries fix the traffic that can be offered at all. That is not the answered set. A valis unit declares the ports it intends to serve, and on a routable IP the catchall does not narrow what it offers to that declaration. What narrows the reachable set back to the ports the agent admits is a separate control, the agent's default-deny firewall inside the namespace, and not the namespace itself. On a loopback-only island the two coincided and the namespace was the whole boundary; once the namespace carries a routable address they are two mechanisms, and the surface control is the firewall. This is the subject of The public surface, including why the ruleset must stay the agent's and never a module's. The first-cut topology resolves the shared-namespace requirement the simplest correct way: the agent creates the namespace and the unit's IP, then valis is launched into that namespace so its socket is born in the right place.
Privilege drawn as a repository boundary
Every capability that needs the kernel's trust lives in the external host agent
and only there. CAP_BPF to load and attach the steering program, and
CAP_NET_ADMIN (with CAP_SYS_ADMIN) to create the namespace and assign the
unit's address, are held by the agent, a separate component in a separate
repository. The migratable valis unit holds none of them. The privilege boundary
is therefore not a runtime check that could be misconfigured; it is a boundary
drawn in the filesystem, where an auditor who wants to know what runs with
elevated privilege reads one repository and a compromised valis unit cannot reach
for authority it was never granted.
This is the same property described in
the edge abuse posture (the outermost, network-aware
defenses belong to the agent because the unprivileged unit deliberately cannot
see what they need), stated here as a privilege invariant rather than an abuse
one. The steered socket needs only LISTEN state, nothing privileged: valis binds
loopback or an ephemeral high port inside the agent's namespace and never binds
the designated public ports. That valis holds no privileged kernel state is the
very property that makes it migratable, because there is nothing host-specific for
it to carry; on migration the destination host's agent re-establishes the steer so
the arriving unit answers its ports again. The dialed port survives the steer
because the kernel preserves the original four-tuple under bpf_sk_assign, so
getsockname on the accepted socket recovers the real port the client dialed,
the same recovery the listener already proves over loopback (see
the fail-closed bind order and the four-tuple
commitment in the listener section).
Validation coverage: the steered-fd handoff
The steering data path cannot be exercised from the Lisp image, since it needs a
real kernel, a network namespace, and CAP_BPF=/=CAP_NET_ADMIN. The image-testable
glue (descriptor framing, port declaration, the seam's recovered-port wrap) carries
cold in-suite coverage. Past that there are two out-of-band gates rather than one,
and they cross different boundaries. make steer-test runs in the agent's
repository on the privileged side, signed off out of band exactly as the v9fs mount
interop test is. The proving ground's steered-delivery gate builds a node and then drives
it from a separate host, which is what puts a real off-host client on the far side
of the steer and lets it read the delivery off accept queues rather than off an
answer.
The agent repository's docs/steering.org carries the same coverage table from the
privileged side.
| Criterion | Test | File |
|---|---|---|
| The steered seam wraps an inherited LISTEN descriptor and recovers the dialed port | steered-fd-recovers-dialed-port |
tests/steered-fd-test.lisp |
| A received descriptor is validated as a LISTEN-state socket before it is steered | received-listen-fd-is-validated-and-returned, non-listening-fd-is-rejected-fail-closed |
fulcrum tests/handoff-test.lisp |
| The descriptor handoff is refused until a valid port set is declared (sequencing) | handoff-refused-before-ready-for-fd, handoff-advances-only-after-valid-port-set |
fulcrum tests/handoff-test.lisp, fulcrum tests/control-test.lisp |
| A declared port set is recorded exactly; an out-of-range port is rejected fail-closed | valid-port-set-is-recorded-exactly, out-of-range-port-is-rejected-fail-closed |
fulcrum tests/control-test.lisp |
| The full steer lands the connection, recovers the dialed port, proves valis never bound it, and confines the steer to the namespace | make steer-test (operator-run privileged gate) |
fulcrum spike/steer-test.{sh,lisp} |
| An off-host client's connection is delivered through the steer and dispatched on the port it dialed, including a port named in no configuration on either side | steered-port-delivery scenario (operator-run gate) |
proving-ground/driver/scenarios/steered-port-delivery.lisp, proving-ground/drive-steered-delivery-gate.sh |
| TCP on the served port enters the steered socket's accept queue while the inherited listener's queue stays at zero, which is what separates the two hypotheses a DNS answer cannot | steered-port-delivery scenario (operator-run gate) |
proving-ground/driver/scenarios/steered-port-delivery.lisp |
The substrate as namespace
The doctrine's eight categories are not eight flat peers; they are a dependency stack, exposed as subtrees of the user's 9P namespace that protocol modules reach through their capabilities.
Semantic objects, not protocol silos
The substrate models the semantic object (a message, a contact, a publication, a contract), and the wire protocols are projections over it. This is the mechanism behind the doctrine's claim that "messaging is one thing": SMTP, IMAP, and NNTP are not three stores but three skins over a single message substrate; a message is the same object whether it arrived by SMTP or was posted by NNTP. Publishing (web, forum, and social protocols) likewise projects one publication substrate. The substrate de-duplicates reality; protocols are views onto it. This is how the protocols are "given back": the user owns the object and chooses which protocols answer for it.
A protocol module (the edge tier) is therefore a wire-format adapter between an IETF protocol on its port and a semantic subtree of the namespace, gated by the connection's capabilities. The substrate is protocol-agnostic; the protocol is storage-agnostic; they meet at the namespace.
The dependency stack
- Foundation - crypto (1), names (2), identity (3). Every other category reaches these. Crypto is the autonegotiated, invisible-by-default layer the capability model already rests on: the factotum-shaped agent, content encryption, transport security, key custody. Names are the user's sovereign namespace of people, places, and things; identity is the user's own, canonical and transient. Identity is where the principals the connection layer authenticates come from, and names and identity are who capabilities are granted to.
- Data - data (4): the relational and reactive store over the migration backing store; the queryable substrate the semantic domains build on.
- Semantic domains - messaging (5), publishing (6), contracts (7). Each is one substrate with many protocol projections. Publishing's public face is the unencrypted, content-addressed path served from many places; contracts (services and payments) is the most forward category, capability- and crypto-adjacent.
- Composite - composite protocols (8) such as IPFS and GNU Social are assemblies of the layers below (content addressing; identity, publishing, and messaging, federated). Their place at the top confirms the stack.
Content-addressed storage doubles as how a migratable instance stays coherent across hosts, and iroh carries it between cooperating nodes. IPFS is in scope as a protocol valis speaks: a node talks to an IPFS network running the reference implementation. There is no native Common Lisp libp2p/IPFS stack; the intended path is CFFI bindings to a libp2p built as a C-ABI shared library, with an external-daemon HTTP bridge as an interim stopgap and a fully native implementation as a long-horizon ideal.
What is framed and what is open
Framed: the layering, and that the substrate models semantic objects with protocols as projections. Open, and recorded as research questions: how richly a semantic object must be modelled so a protocol projection is faithful rather than lossy (email threading, NNTP groups, social-graph semantics); what the data engine actually is in Common Lisp; whether names is one model with two faces (a sovereign address book that also answers DNS); and the shape of the contracts category. These are named so each domain is built against shared substrate interfaces rather than re-inventing storage and identity per protocol, the discipline that keeps "their computer, their data" true.
Plugin composition: protocols as content-addressed, capability-gated modules
The central abstraction promises "every protocol a plugin." This section gives
the second half of that sentence a concrete noun: a plugin is a module (a
registered, lifecycled unit of code), and a protocol is a module that fills the
edge-adapter role. The mechanism is Shinmera's modularize (a library that makes
a package first-class: its own metadata store and load/delete hooks), adopted as
valis's plugin lifecycle. The fit was validated empirically before it was
committed: a throwaway module proved, cold, that modularize composes with
valis's package-inferred discipline, and surfaced the one trap that would
otherwise read as incompatibility (the grouping form is not idempotent under
ASDF's compile-then-load and must be guarded). Composition governs what code runs
and what it may reach within the substrate; whether a module's port is reachable
from off the host is a separate, privileged decision at the surface, never
derived from the module (The public surface). Three decisions define the model.
A plugin is a module, and a module is one combined system
A protocol module is a single ASDF system that is both an
asdf:package-inferred-system (valis's one-package-per-file discipline, where
each file's defpackage declares its own dependencies and the build graph
derives from :import-from) and a modularize:virtual-module, a named,
discoverable unit with a metadata store and a teardown lifecycle. The two are not
rivals: one class is both, package-inferred derivation still fires, and the
module's :packages grouping gathers the inferred sub-packages under one name.
valis-core stays an ordinary system; only the pluggables become modules.
The core's one structural change is to stop naming concrete adapters. The edge
controller's hard-coded adapter list (%adapter-specs) becomes an
*edge-adapters* registry that modules install into on load, the same
"downstream installs the hook" idiom valis already uses for
*edge-source-constructor*, *ctl-node-factory*, and *bearer-view-decorate-fn*.
A module declares its adapter in one line of module storage (an :edge-adapter
spec); a load hook registers it; a delete hook drains then retires it.
Drain-before-delete is not optional: a live protocol has in-flight connections,
so the delete hook runs the ordered edge teardown (deregister the source, drain
in-flight work, unregister) before the package is unbound. That drain is the same
primitive evacuation uses: the session drain on the 9P listener. Two correctness
disciplines fall out of the lifecycle: registration must be idempotent (the
grouping form runs twice under compile-then-load), and a protocol's macros stay
module-local: cross a module boundary with functions, not macros, because a
reloaded producer leaves a consumer's already-expanded macro call site running
the stale expansion, silently and without error.
Module load is a capability decision, never a hash fetch
Content addressing answers which module: a module package, held in the content-addressed store or fetched from a cooperating node over iroh, is named by its hash, so an instance pulls exactly the bytes it asked for and verifies them on read. But a hash answers identity, not trust. Loading a module means compiling and running code, and code pulled by hash from the network is, unqualified, a supply-chain hole: running code validated only by its hash is a recipe for pain. So which modules an instance will load and compile is a capability decision, gated by the same capability machinery that gates every mount: a module is admitted only when a publisher you have filed has vouched for its identity and you have signed for it, exactly as a capability vouches for a subtree (whose vouch admits a module). The hash is the name; the owner's signature is the permission. This keeps the "evaporate and condense elsewhere" malleability without turning a recondensing instance into an open code-execution endpoint, the discipline that matters most before the system is opened to outside contributors.
Where modules come from, and whose vouch admits them
Installing a module is dynamic. When you request a module,
the node takes it from a registry of modules, or from a third party who publishes it
through git with defined release semantics, and inserts it into the running image
with modularize. I want a new module to be a request, not a
redeploy. Installing valis itself is the genesis case of condense
(Deployment); installing a module is runtime
insertion into a node that already stands.
A request names a module by its identity. You ask for a module by the identity its publisher signed, and a node never turns a name into an identity: the name only says which system to load, and there is no name index and no latest version. I made it so because a name can be claimed by anyone, and a name that chose bytes would let whoever claimed it decide what runs in your image. The bytes come from the private dist, from a git tag, or from the node's own store when that identity is already there. A git release is marked by its tag and nothing more, and the tag only says where a release is, because a tag can be moved or mistyped. The node fetches exactly the tagged tree, computes the identity from the bytes it fetched, and refuses them unless they are the identity you requested, before they reach the module's directory. Trust comes only from the publisher's vouch over that identity, which travels in the tree beside the bytes and is left out of the identity, and from your own grant at install: a tag, a repository or a dist location confers none. A content-addressed lookup such as the DHT would be one more place to find bytes by identity, and would change none of this.
valis mints a module's identity. It is a self-describing multihash, so the name
carries its algorithm: SHA-256 today, with BLAKE3 identities alongside when iroh
lands, and a vouch survives the change. It is computed over the sorted entries of
each file's relative path, size and SHA-256. A vouch names it as
/module/ followed by the lowercase hex of the multihash, and valis refuses a
designation that names an algorithm it does not mint, or a digest of the wrong
length, so you cannot present the same bytes under a weaker hash. Each network's
name for a module (an iroh hash, an IPFS CID) is a locator computed at publish or
fetch time, where that network is spoken, and never recorded; fetched bytes are
checked against the identity. In a Kademlia DHT a
module is placed at the SHA-256 of its multihash, where IPFS looks for it.
The same path carries live update. I want a changed version of a module the node already carries to supersede it in the running image, and a change missing either signature to be refused without disturbing what the node already serves. The resident HTTP and Gopher pair is compiled into the image, so it is not replaced this way (the scope of resident-set parity).
Admission takes two signatures: the publisher's and the owner's. The publisher's says the module is what it claims to be and is fit to run. In the first instance the publisher is DeepSky, and the same authority carries a running node's core foundations forward through updates. The owner's says "install it here", so you stay sovereign over your own node: the owner's signature is the permission (module load). I intend publisher vouches to widen later: DeepSky's carries the greater default weight, and a third party's is weighted by a reputation system built from a publisher's steadfastness over time and the votes of valis operators. The capability system must let you install a third party's module for a service DeepSky does not provide. The module manifest carries no vouch (the record shape).
A publisher's vouch is a chain from its creating identity. You are to file the publishers your node trusts, each named by the did:key of its creating identity. That identity never signs a vouch itself. It delegates to a signing key it can revoke, and the signing key vouches for a module by granting admission over that module's identity, so a publisher's vouch arrives as a capability-token chain of at least two links. I made the chain the only form a publisher's vouch takes so that revoking a signing key reaches every vouch it signed without touching the publisher's root: put the delegation's hash in your revocation store and every vouch through it is refused. A chain rooted anywhere you have not filed, your own key included, admits nothing.
The owner's signature can be a standing grant rather than a prompt, because a fix for a security failure in the core has to reach a node at once. A standing grant is your signature given once for a publisher: it names the publisher and covers everything that publisher vouches. Every admitted module runs in the kernel, so there is no lower tier of update that could safely take less consent; your choice is which publishers stand in for a per-module signature, and criticality only orders what you are shown. Withdraw a standing grant and it admits nothing new, while the modules it already admitted remain eligible when the node re-admits its set. Revoke it and re-admission is to refuse them, which is how you back out everything a publisher deployed under it, and the node is to list the modules a grant admitted, so you know what that takes away. The grant is how you give your signature, never a way around it. Your own half is never delegable: admission refuses an owner credential that arrives as a chain. The admit door does not yet mint a standing grant, and filing a publisher or a grant has no owner verb yet.
The node keeps both signatures as it received them. For each module designation, the owner's vouch store has a place for the publisher's vouch and the owner's grant exactly as they arrived, together with the time the module was admitted. It is to hold them once a module is admitted through it; nothing files them yet. It also holds the most recent revocation list the publisher signed, as filed, the publishers you have filed, and your standing grants, each marked active or withdrawn. I keep the credentials rather than minting a fresh one at each use because a fresh mint carries a fresh nonce, and so a fresh canonical hash: revoking one mint would never reach the next. A credential held as received keeps one hash for as long as it is held, so revoking it is a lasting act. The store only locates: it verifies nothing and judges no expiry. I intend every use of a stored credential to verify it again, against the owner's custody and the revocation state as they stand at that moment. Admission reads the filed publishers and standing grants from it; nothing yet reads a module's stored credentials back to admit it. The module manifest still names only which modules run and carries no credential (the record shape). That is why the store may exist at all: nothing in it is a trust decision frozen into durable state.
Revocation reaches a node from two directions. You can revoke a publisher's vouch by its canonical hash in your node's own revocation store, which is your override as the node's sovereign, and revoking the delegation to a signing key refuses every vouch that chains through it. Admission checks every link of the chain against that store today. The other direction is not built yet: the publisher is to reach every node at once through a revocation list it signs, which is what an incident in a publisher's own infrastructure needs, and I intend admission to consult both. The node already keeps the list as filed; nothing reads it yet. An entry is to name a single vouch by its hash, or a signing key by its did:key, which refuses everything that key signed whatever delegation it arrived under; revocation never works by replacing the publisher's root. An entry is to mark its target either revoked or withdrawn: a withdrawn version refuses a fresh install and stays where it already runs, while a revoked one is refused at install and at restart alike. A revocation your node has seen is never to be forgotten, even when a later list leaves it out; undoing one means reissuing. A publisher is to drop a revoked entry only once its target has expired and can no longer admit a fresh install, so the list stays bounded to the revocations that still matter, and a node that joins later or was offline across lists still learns every one it needs. The list is to carry a signed issue time and a sequence number that must rise, so that a node will refuse a list older than the one it holds: integrity comes from the publisher's signature, any relay may carry the list, and no relay can hide a revocation by replaying an old one. A new install or a supersede is to need a list no older than seven days; re-admitting modules at restart, and falling back, are to use the last list held however old, and to tell you its age. A dark feed is never to take an existing service down. A node that has never received a list is to install nothing, so you seed the first node by filing a publisher-signed list, which may be empty, by hand.
Expiry governs admission; revocation governs presence. A grant or a publisher vouch is to admit a module only while it is unexpired. When a node re-admits its recorded modules at restart, or falls back to an earlier set, I intend it to judge each credential's expiry at the recorded admission time, while every signature must still verify now and nothing may be revoked now. The store has a place for the admission time, but nothing records it yet and nothing reads it. The admission time is a fact about when, never a trust decision. An installed module is therefore never to lapse on the clock: it leaves only when it is revoked or you take it out, and you need a fresh grant only to install or supersede. The one-year limit on a grant is to be a backstop on admission, enforced when the grant is filed, not a clock on how long a module stays; no such check exists yet.
⚠ How the publisher's revocation list reaches a node other than by the owner filing it is still to be designed.
Repair is broader than restart. A node is to re-admit its recorded module set on restart, judging each module afresh; today a restart refuses any installed module, because nothing yet supplies the publisher's vouch back to it.
None of what follows is built yet. Rolling a module back to a known-good state, and repairing a node while it runs, belong to module lifetime management. The registry this implies is ordered, carries cryptographic provenance weighted by origin, and records versions that roughly map the arrow of time.
A module set is to live in generations. Each install, supersede or rollback is
to record a new generation of the module set, together with the resident binary
it was admitted against. The binary and the module set move independently: rolling
the binary back is to select no generation, and the node is to warn you when the
running binary is not the one the active generation was admitted against. A new
generation is to run at once, tentatively, and to become the set the node boots by default once it has stayed healthy (the fabric
up, the edge ports answering, the active modules alive) for fifteen minutes, or
whatever window you set, or, if you configure your node that way, only when you
bless it. If a generation fails to stand up three times running,
the node is to fall back to the most recent permanent generation on its own and
tell you loudly. If no retained generation qualifies, it is to stand up only what
is compiled into the binary, keep :53 serving, and tell you why every installed
module was refused. A running node found degraded is to report it and leave the
repair to you, a restart into the most recent permanent generation; you can
configure it to make that restart itself. Retention is to keep five generations
or sixty days, whichever keeps more, and never to prune the current permanent
generation, the running one, or the last permanent one besides them. A module's
source bytes are to be kept once, under its identity, and never overwritten, so an
older generation always has the exact bytes it ran.
Rolling back needs no fresh signature, but it is judged now. Returning to a generation the node has run before is to reuse the publisher's and your original signatures, provided they verify now, nothing in them is revoked now, and each was unexpired when the module was first admitted. The node is to refuse a generation it never ran. The fallback is never to select a version its publisher has withdrawn; your own rollback may, if you override it explicitly, and the override is to be logged. A module is to declare which data formats it can read, so the node is to refuse to return to a version that cannot read what a newer one wrote, unless you override that too. Rolling a module back is to happen only by restarting into a generation: I do not believe code can be removed from a running Lisp image reliably without changing SBCL itself, so there is no in-image step backward.
Disabling is to be the only removal while the node runs. Disabling a module is to retire its ports, handlers and registrations through its delete hooks, while its code and classes stay resident; you are to be shown it as disabled and still resident, and the image is to be marked as diverged until the next restart removes the module for real. Superseding a module forward in a running image is to be checked twice before anything compiles in the resident: a comparison of the two versions' forms is to refuse changes the image cannot take in place (structure definitions, constants, declared function types, inline functions, macros already expanded into callers) and say a restart is required, and a dry run in a separate SBCL process is to load the old version and then the new one. A supersede that leaves residue behind is to stay tentative until a restart proves the image clean.
An instance's identity is its namespace manifest plus its module manifest
A valis instance is defined by two content-addressed manifests, not one. The
namespace manifest records what is mounted: the data side, which migration
already reassembles. A module manifest records what the instance is composed
of: the protocol and plugin set it answers with. Both are durable,
content-addressed, and reassembled on recondensation: a fleeing instance does not
carry its code, it records the capability-vouched hashes of the modules it ran,
and the destination pulls, verifies, and compiles them to stand up functionally
equal to the instance that evaporated. Because the substrate already
content-addresses data, it content-addresses the modules too, and iroh becomes the
distribution layer for protocols between nodes, not only a carrier for bytes. modularize's
in-image list-modules plus per-module storage is the live projection of this
manifest; the durable module manifest is its persisted, migratable form. This
closes the migration story for code the way the namespace manifest closed it for
data: a valis instance is its mounts and its modules, both reassembled from
hashes the owner vouched for.
The module-manifest record shape
The module manifest is a durable, content-addressed record distinct from the namespace manifest, and the distinction is the whole point. The namespace manifest records mount roots: what an instance has mounted, the data side. The module manifest records the module set as code: each entry is a system name, the module's ordered source files as paths relative to the module root, and the module's identity: its self-describing multihash (module identity), carried with its own length so a wider hash needs no new record shape. A resident record, one of the HTTP and Gopher pair compiled into valis, carries no identity at all. No network's name for a module is stored. Names and relative paths and an identity: never a key, never a secret, never the vouch itself.
Every manifest is written at version 2. A version-1 manifest, whose records ended in a fixed 32-byte score, still reads: an all-zero score, which is how genesis wrote the resident pair, reads as a resident record with no identity, and any other score is refused as corrupt. I refuse it rather than carry it forward because its derivation no longer names a module, and no live node holds one: every version-1 manifest ever written holds only the resident pair. The record is built once, its entries sorted by system name, so the same module set serializes to the same bytes regardless of discovery order; a content-addressed record demands that determinism. Decode is fail-closed: a bad width, an unknown version, a malformed entry, or a missing referenced byte stream aborts the read rather than yielding a partial set.
The two manifests are siblings (a manifest child and a modules child)
under one durable head tree. At genesis they are written by a single combined
publish that obtains both envelopes without advancing, builds one root tree naming
both children, and advances the head exactly once. This is not a convenience: it
is the only correct sequence. Two separate publishes cannot work, because the
first advance moves the head off the genesis generation and the second's
generation-fenced compare-and-set fails closed. So the manifests are atomic
siblings by construction, and the evacuative reassembly that reads the modules
child is correct precisely because the genesis publish guaranteed that child
exists under the same root.
The record stores no vouch and no key, and that absence is deliberate. A credential frozen into a durable record would carry the authority lifetime of the moment it was written into a record that outlives it: a revoked owner key still vouching for code months later. Instead trust is decided fresh at reassembly: each module's publisher vouch and owner signature are verified against live custody and live revocation/fence state at the moment of condensation. The record's job is to name which modules; the live authority decides whether to admit them. Authority lifetime is decoupled from record lifetime.
The reassembly flow
A successor instance stands its module set up from the durable record the same way the namespace assembler stands a namespace up from its manifest: reassembly, not transfer. The source instance does not hand its running code to the destination: it records the vouched hashes of what it ran, evaporates, and the destination re-derives equivalent running code from the record. This is the code-side symmetry of the data-side migration: an instance's identity is its mounts and its modules, both reassembled from hashes the owner vouched for.
The reassembly is two-pass and all-or-nothing. The first pass reads the manifest fail-closed. The second pass, over every record before any module loads, resolves the recorded relative paths under the module's own identity directory beneath the destination's source root, pulls the bytes, re-derives the module's identity, and asserts it equals the recorded one: a wrong-hash record aborts the whole reassembly here, in front of the first load. The pull is transport-agnostic: a rebindable puller seam means the bytes can come from the local store today and from cooperating nodes over iroh, or an IPFS network, tomorrow without reshaping the record: the IPFS gate stays open. Then both signatures are located for each identity, and a missing one aborts the set. Only after the whole set passes does the third pass load, routing every module through the same capability-gated admission chokepoint that gates an interactive load, never calling the loader directly, so the gate is structurally in front of compilation. If any module is refused at the gate, the adapter registry is restored to its pre-reassembly snapshot, so an earlier-admitted module's adapter cannot leak past a later refusal. A partial module set never boots.
Each version of a module lives in its own identity directory beneath the source root, named for its identity. An identity directory is written once, whole, and never overwritten, and a new version lands beside the old one. The loader is pointed at exactly that directory, and ASDF forgets a system it already knows before loading it again. I keep it this way because ASDF decides what to reload by file dates, so bytes restored under a name it has already seen could be silently skipped, and because a retained version must still be there, unchanged, for you to roll back to.
The resident default set (HTTP and Gopher) is handled honestly rather than forced through this path: it is recorded name-only with no identity, is already present as part of valis itself, and is skipped by the reassembly. Only genuine non-resident modules (those carrying an identity) run the pull-verify-vouch-admit path.
The honest scope of resident-set parity
One claim must not be overstated. The resident default adapters are recorded name-only with no identity, because they are part of the valis system and are already loaded: there is no separate bundle to content-address. So the migration guarantee for the resident set is naming parity: a successor's recorded module set names exactly the source's set, and the resident edge registry the successor serves through names that same set. It is not identity parity for those entries, and the documentation does not pretend it is.
The genuine content path (re-derive the identity, match it against the record, locate an owner vouch, route through the admission gate) is exercised and proven by the non-resident scratch-module reassembly, not by the resident default. A reader who wants to see the identity-to-admit guarantee should look at the non-resident reassembly test, where a module with a real identity is pulled, verified, vouched, and admitted, with an unvouched control proving the successor fails closed. The resident set proves the set-naming parity; the non-resident set proves the content gate.
The reassembly security boundary
The trust boundary the reassembly defends is the one between the durable record and the live authority that admits its modules. The record is published data (anyone who can read the store can read which modules an instance named), so the record holds no secret, and the decision to run those modules is made fresh against live state. The threats and the assertions that pin them:
| Threat | STRIDE | Direct assertion |
|---|---|---|
| Hash substitution: a record names another module's bytes | Tampering | the destination re-derives module-bundle-identity over the resolved bytes and aborts unless it equals the recorded identity |
| The same bytes presented under a weaker hash algorithm | Tampering | a /module/ designation naming an algorithm valis does not mint is refused by name and never matches a module |
| A network's name for a module leaking into the durable record | Information Disclosure | the record carries only the identity; a byte-absence regression checks it for the content identifier and the DHT key |
| Unvouched code execution: recondensing as an open code endpoint | Elevation of Privilege | every module is routed through admit-module, structurally in front of the loader; a refusal never reaches compile or load |
| Revoked or fenced module re-admitted from a stale record | Elevation of Privilege | trust is decided at condensation against live revocation and fence state (the re-mint posture), not a stored credential |
| A partial or unverified module set on a mid-reassembly failure | Tampering / DoS | two-pass reachability and vouch precheck before any load; the adapter registry is snapshot and restored on any abort |
| A key or secret leaking into the durable record | Information Disclosure | the record holds identities, names, and relative paths only; a byte-absence regression and the store-cluster import firewall pin it |
| A custodial or single-transport assumption baked into the pull | Tampering (supply chain) | the pull is a rebindable seam; a content-addressed network transport drops in without reshaping the record |
| A two-step genesis publish leaving one manifest unwritten | Tampering | both manifests are written under one root advanced once; a second advance would fail the generation fence closed |
Superseding a module in a running image
I want a signed change to a module the node carries to take effect without a restart, at the ports the module already serves. A supersede does that in place. Compiling a new version into a running image is itself a change to the image, so every change which a running image cannot make in place is refused before any of it is compiled, in a fixed order:
- The new version's directory must pass the checks an install applies: it sits directly under the module source root, is private to the node, and is named for the bytes it holds.
- On a node that serves real names, a supersede that would write a manifest record is refused before the comparison, just as an install is, until a node can fall back from a module set that fails to stand up.
- The version the image carries must be an admitted module: the directory it was loaded from, its admission record and its manifest record name the same version. A module whose earlier supersede failed part way through its load matches no version, and is refused until a restart.
- The bytes of the two versions are compared as data, and nothing of either is evaluated to do it. A change which a running image cannot make in place is refused with "restart required": a changed structure, constant, macro, compiler macro or type; a changed declared function type or inline declaration of a function the old version defines; a changed generic function lambda list; a class or condition losing a slot or a superclass, or changing its metaclass; a package losing part of its definition; a removed source file; and any change to the module's grouping form or to the guard around it. A top-level form of a kind the comparison does not take apart, such as a let around definitions, a module's own defining macro, or any form of the .asd other than a system definition, is compared whole, and any change inside it is restart required. The comparison refuses on textual difference, so it can refuse a harmless change; a change inside a form it compares whole is never accepted.
- Both versions are then loaded in a child process of the node's own binary, the old one first and the new one on top of it, through the same gate. A new version that fails to load there, or that the child refuses, is refused here with nothing compiled.
- Only then does install-module, which checks both signatures, compile the new version in the resident and record it in the module manifest in place of the old one. The publisher's vouch and the owner's signature are both needed anew over the new identity; the old version's signatures do not carry over.
One lock covers every install, request and supersede, from finding the version the image carries to recording the outcome, so two changes to module code never interleave. A change waits at most a few seconds for another one to finish and is otherwise refused as a change already in progress; the boot, which stands the recorded modules up while the node already serves, waits without a bound.
The grouping form does not run again. Every real module guards it, so a second load leaves the module's packages, options and edge adapter declaration as they were, and a changed port or adapter option in a new version would be ignored without a word. That is why the comparison refuses any change to it. The listener, its port, its counters and the adapter's registration are not touched, and nothing is announced to the host agent.
A supersede replaces every named function and every method with the same specializers, from its next call on, and a class gains the slots its new definition adds. It cannot replace the rest, and these bounds hold:
- Loading is not atomic across files. While the new version loads, a request can reach the new version of one file and the old version of another.
- A request in flight finishes its current frame on the old code, and every named call it makes after that reaches the new code. A connection held open across a supersede gets the new behaviour on its next request.
- Closures and function objects that the old code built and that something still holds, threads parked in an old loop, and finalizers go on running the old code.
- Functions and methods the new version omits stay defined and keep dispatching. The supersede result lists them.
- A new version whose load fails in the resident after passing in the child leaves the image partly superseded, and the image's divergence record holds an entry for the failed load.
After a supersede, a scan of the heap lists the code of the old version still alive in the image. It can over-report, because a stale word on a thread's stack counts as a reference, and it can miss live code, such as the code of a closure something still holds. An empty scan proves nothing. So every supersede is tentative until a restart into the new version, whatever the scan found. Each one is entered in the image's divergence record before anything of the new version is compiled, and the entry is completed with the outcome. A supersede refused before any of the new version was compiled leaves an entry marked as not loaded, which does not count as divergence. For the rest, the entry records the outcome, whether the module manifest now names the new version, and the residue the scan reported, or a note that the scan itself failed. You can list those entries for as long as the process runs.
The new version loads from its own identity directory, and the old version's directory is left byte for byte as it was, so each version stays where its identity names it (the reassembly flow). Side-by-side versions are not offered in place: a supersede replaces the version an image runs and never runs two.
One bound belongs elsewhere. The store's write fence carries only an epoch, so two residents reading one store head compare equal and both pass it. Live supersede makes restarts rare, which lengthens the time an image runs on a store another instance may also hold. Today a second instance on one store arises only during an evacuation hand-over or among peers sharing a store. Closing that window belongs to node clustering, where nodes cooperate over one store, and not to supersede; nothing in a supersede fences it.
Validation coverage: plugin composition
The record shape and the reassembly are anchored to named tests an auditor can run
and watch stand or fail. The record codec and its fail-closed decode are pinned in
tests/module-manifest-test.lisp; the reassembly chokepoint's fail-closed,
all-or-nothing contract is pinned in the same file's condensation suite; the
end-to-end migration parity (a source instance records its module set, evaporates,
and a successor reassembles it through the production path) is pinned in
tests/migration-end-to-end-test.lisp.
| Criterion | Test | File |
|---|---|---|
| Record codec round-trips; fixed-width envelope; order-invariant; pins a generation | record-codec-round-trips, envelope-is-fixed-width, module-order-is-canonical, pinned-generation-is-independent |
tests/module-manifest-test.lisp |
| Record carries no key bytes; the manifest package imports no content symbols | record-carries-no-secret-key-bytes, store-cluster-imports-no-content-symbols |
tests/module-manifest-test.lisp, tests/migration-end-to-end-test.lisp |
| Genesis writes both manifests under one root advanced once | genesis-publishes-both-manifests-one-advance |
tests/module-manifest-test.lisp |
| Reassembly admits a vouched set; the loader fires exactly once | condensation-admits-vouched-set |
tests/module-manifest-test.lisp |
| Unvouched / wrong-hash / revoked module fails closed; no partial set | condensation-refuses-unvouched-set-no-load, condensation-refuses-wrong-hash, condensation-refuses-revoked, condensation-all-or-nothing-multi-record |
tests/module-manifest-test.lisp |
| Reassembled module set names exactly the source's set (functional parity) | resident-module-set-survives-reassembly |
tests/migration-end-to-end-test.lisp |
| Non-resident module pulled, content-verified, vouched, admitted; unvouched control fails closed | vouched-non-resident-module-condenses-unvouched-refused |
tests/migration-end-to-end-test.lisp |
What is framed and what is open
Framed: that a plugin is a combined package-inferred module; that the edge
controller iterates an installed *edge-adapters* registry rather than naming
adapters; that module load is a capability decision over a content-addressed hash;
that the active module set is migration-recorded alongside the namespace; and now
the record shape itself: names, relative source paths, and a module identity,
sibling to the namespace manifest under one genesis root, carrying no stored vouch
because trust is re-decided against live authority at reassembly. The modularize
fit with the package-inferred discipline is settled and cold-verified, the
idempotency trap mapped, and the reassembly proven end to end. Open, and the work
the next arc takes up: the protocol adapters move to their own sibling repository,
c3po (the home for every wire-format module a valis consumes) extracted on the
discipline of real need rather than anticipation, with the existing HTTP and
Gopher adapters relocated so the conversion proves the seam against their
already-green tests; the production capture path that records a real non-resident
module set at publish time (the reassembly and content gate are proven, the
publish-side capture of a live non-resident set is the remaining wiring). What a
supersede in a running image does to connections in flight is framed in
superseding a module in a running image. These are named so the plugin layer is
built against one uniform module
abstraction (discoverable, lifecycled, content-addressed, capability-gated)
rather than a bespoke registry reinvented per plugin kind.
Per-axis sovereignty: one principle, three boundaries
This chapter is the auditor's map of the data-classification model that holds across the three operator-state axes: mail, DNS, and the social graph. It states one principle and then walks the three boundaries that make it concrete, each as a threat (what leak it prevents), an enforcement point (where in the code it is guaranteed), and an audit proof (the test that keeps it honest).
The principle is reach-by-classification, never reach-by-content. Operator state that lives in PostgreSQL holds pointers and metadata (a content-hash pointer to a body, an envelope field, a DID, a lifecycle disposition) and never a message body or a header. An infrastructure or operations path reaches a datum by what class of thing it is, not by reading its content; the content of a message stays opaque on the content-addressed store and is reached only through an authorized, owner-facing view, never through an operations seam.
That invariant is enforced structurally, not by review discipline, because a
future migration is exactly where a body column could otherwise slip in. The
confused-deputy schema audit (audit-migration-schema,
src/operator-state/audit.lisp) introspects the live committed information_schema
after a migration's DDL has applied and before COMMIT: forbidden-schema-columns
rejects any column across the operator-state tables that is not in the explicit
+permitted-operator-state-columns+ allowlist or that matches a body/header shape.
The migration runner (run-pending-migrations, src/operator-state/migrate.lisp)
calls the audit in-transaction on every schema migration and rolls back
fail-closed on any finding, so a migration that would introduce a body or header
column cannot land: the runner cannot be bypassed, and asserting over the live
schema rather than the dao-class definitions catches hand-written DDL too. The
proof that the gate is live and non-vacuous is
run-pending-migrations-aborts-a-body-bearing-migration and
operator-state-schema-has-no-body-or-header-column
(tests/operator-state-sovereignty-audit-test.lisp).
The content boundary: mail bodies never reach PostgreSQL
A message body never enters PostgreSQL. It stays an opaque content-addressed block
on the store, and the mail-transport-state row names it only by a content-hash
pointer, body_ptr: an address, never invertible to content (the opaque-block
view of the mail spine is detailed at the content boundary).
Threat. A confused deputy (an operations or migration path that could name, decode, or copy a body byte through the queue or state seam) would turn a substrate-management capability into a content read. That is the leak this boundary forecloses: operations moves and meters the spine and reaches no body.
Enforcement point. mail-transport-state (src/operator-state/mail-state.lisp)
carries only body_ptr (a 64-char content-hash pointer) plus envelope and
lifecycle metadata (no body or header column), and its package imports no
body-octet reader (read-block / decode-file). The schema audit above bans any
body/header column across the operator-state tables on every migration, so the row
shape cannot regress.
Audit proof. The import-firewall sweeps
ops-queue-projection-imports-no-body-read-symbol and
ops-queue-grant-resolves-no-message-body-name
(tests/mail-sovereignty-audit-test.lisp) prove the operations view reaches no
body both by construction and over a really-queued message; the in-runner gate
a-body-bearing-mail-migration-aborts and
mail-state-migration-unit-passes-the-in-runner-audit
(tests/operator-state-sovereignty-audit-test.lisp) prove the schema audit accepts
the real mail unit and rejects a body-bearing one.
Write-authority and public serving: the DNS zone axis
DNS zone data is published by design: the records are meant to be world-readable. The protected asset is therefore not secrecy but write-authority: who may mutate a zone. valis is the authoritative ndb data owner over PostgreSQL; the nameserver reads through a narrow read seam and composes answers, holding no write authority of its own.
Threat. Unauthorised zone mutation (a session or a confused deputy committing a zone change it does not own, or smuggling its claimed authority in through the submitted bytes) is the leak this boundary forecloses.
Enforcement point. The :names write gate stamps the owning principal from the
session, never from the message bytes: node-create reads the session principal
at submission (%current-submission-did, src/namespace/names.lisp) and
node-close commits it as the zone's owner_did at the 9P clunk. owner_did is an
allowlisted operator-state column carrying the zone's write-authority owner
principal, and serving is a read-only projection over the committed data.
Audit proof. dns-schema-has-no-forbidden-column,
ndb-facet-columns-pass-confused-deputy-audit, and
existing-zone-backfilled-as-primary-on-ndb-anchor
(tests/operator-state-zone-audit-test.lisp) prove the zone schema passes the
confused-deputy audit and that the write-authority facet is carried as metadata,
not content.
Validation coverage: per-axis sovereignty
Each boundary is anchored to its enforcement point and the test that proves it. The common principle (pointers and metadata, never bodies or headers) is the schema audit the migration runner runs before every COMMIT.
| Boundary | Threat | Enforcement point (file:fn) |
Audit proof |
|---|---|---|---|
| The principle: reach by classification, not content | a future migration adds a body/header column | src/operator-state/audit.lisp → audit-migration-schema / forbidden-schema-columns, run in-transaction by src/operator-state/migrate.lisp → run-pending-migrations |
run-pending-migrations-aborts-a-body-bearing-migration (tests/operator-state-sovereignty-audit-test.lisp) |
| Mail bodies → content boundary | a confused deputy decodes or copies a body byte | src/operator-state/mail-state.lisp (body_ptr only, no body-octet import); the schema audit bans body/header columns |
ops-queue-projection-imports-no-body-read-symbol (tests/mail-sovereignty-audit-test.lisp); a-body-bearing-mail-migration-aborts (tests/operator-state-sovereignty-audit-test.lisp) |
| DNS zones → write-authority + public serving | unauthorised zone mutation | src/namespace/names.lisp → node-create / node-close stamp owner_did from %current-submission-did (the session principal, not the bytes) |
dns-schema-has-no-forbidden-column, ndb-facet-columns-pass-confused-deputy-audit (tests/operator-state-zone-audit-test.lisp) |
| Social graph → capability-scoped access | ambient / confused read of the graph | src/plugin/social-admission.lisp → query-reachable gates the raw traverse-reachable behind verify-social-vouch (/social designation + :read) |
query-reachable-is-the-only-caller-facing-entry-to-the-raw-traversal and the four bad-vouch refusals (tests/social-capability-scope-test.lisp) |
Sovereignty as an architectural invariant
Operations manages the substrate and lifecycle; it never touches the content and meaning of a user's data. The namespace fabric is where this line is drawn: operability is expressed as moving and mounting viewers, never as reading content. The capability discipline above is the enforcement: a module (or an operator's tool) holds only the authority designated into its namespace, never an ambient authority over all of a user's data. Where code could still violate the line (observability, backup, multi-tenant isolation) it must be confined the same way, by what is mounted, not left to operational good behaviour.
What this invariant does not claim
The paragraph above is a statement about operations, and a reader is owed its limits in the same place, because the gap between them is where a wrong decision gets made.
Ordering. valis is first a way for a person to hold their own data, against the capture of a network architecture that did not have to end up this way. Resistance to a well resourced adversary is a real secondary capability and it is not the headline. Where the two conflict, holding the data wins. Anyone weighing a feature that trades availability of a user's own data for resistance to seizure should know that trade has already been decided in that direction.
The ceiling, stated rather than implied. A machine that is seized and carried off yields its processor, memory and storage state to an adversary who can then work on it without time pressure. There is no defence at that scale and none is claimed. What is achievable is narrower: notice physical interference, and make the relevant data unreadable afterwards.
That last part is harder than it sounds, and the reason is structural. Overwriting does not reliably erase modern persistent storage, because wear levelling leaves the controller deciding which physical cells a write lands on. Destroying a key is the only fast and checkable way to make data unreadable. So the arrangement has to put a small secret in a position where destroying it is sufficient, with no plaintext having reached swap, hibernation, or any store that outlives the key. That is a property of how the system is laid out. It cannot be added later as a panic button.
One tension that does not dissolve. A node that stays online without a person attending it must be able to restart on its own, so its operating key has to be reachable by the machine with nobody present. Putting that key behind a person's presence defeats what the system is for. So a node that can restart unattended will surrender its operating key to whoever takes the machine. That is a consequence of the goal rather than a defect awaiting a fix, and it is why authority to reconstitute a node elsewhere is worth separating from the key the node runs on.
⚠ Sealed backup is built; splitting the secret is not. What exists
today is passphrase-sealed off-host backup (src/backup.lisp). Splitting a
secret across several holders, so that no single seized machine holds
enough to reconstitute anything, is design intent and is not implemented:
src/ contains no secret-sharing of any kind. Read any description of
reconstitution elsewhere in this document with that distinction in hand.
The public surface: what the internet reaches, and who opens it
A sovereign node answers the public on a routable address, and the question this chapter settles is the one a reader most needs before placing new behaviour: which ports the internet can reach, and who decides. Two mechanisms are involved, not one, and neither is the namespace by itself. This is worth stating plainly because an earlier framing ("the designated-ports boundary is the namespace itself") was true only while the node lived on loopback and stops being true the moment it carries a public IP.
The steer fans every port; the firewall selects them
The privileged host agent's sk_lookup program is a catchall
(fulcrum/src/steering.lisp): it stashes valis's one socket in a single-slot map
and redirects every TCP and UDP port on the unit's IP to it, transport- and
port-agnostic by construction. On a loopback-only island that was harmless:
nothing off-host could reach the namespace, so the ports the node answered were
exactly the ports anything bothered to connect to. Once the namespace carries a
routable address, the same catchall would fan every port on the public IP into
valis.
What narrows the public surface back to the ports the agent admits is a second
mechanism: a default-deny firewall the agent installs inside the namespace
(fulcrum/src/firewall.lisp). Its input chain accepts loopback, established and
related return traffic, and :53 on both transports. Three more accepts are
elected by the agent's own config: the owner port and the public TLS edge port,
each present only when configured, and :80, which ships elected because the
manual has to answer before any credential exists. The last accept names a
kernel set rather than a number, and the members of that set are what this node
declares: one static rule, membership mutated in one transaction, so a
declaration reaches set elements and never rule text. Everything else drops.
The steer says "all ports to valis"; the firewall says "only these enter." They are complementary and both required, and the firewall (not the namespace) is the public-surface boundary. The distinction is not pedantic: a namespace boundary is structural and cannot be misconfigured open, whereas a firewall is a maintained rule that must be installed correctly, before any traffic, and that can in principle fail open. On a public node the surface control is therefore a security-critical, order-sensitive invariant of the same rank as the write fence, documented here rather than treated as an afterthought.
The firewall ruleset is the host agent's, never a module's
Because the boundary is a policy and not a topology, who writes the policy is the
question that decides how far a hostile module can reach, and the answer has two
halves a reader has to keep apart. The ruleset is the privileged host agent's alone:
it is a pure function over the agent's own configuration, fed to a privileged nft
inside the namespace, and it takes no valis-supplied input at all. What that ruleset
accepts from is a named kernel set, and the members of that set are what this node
declares. A module's ports can reach the set; nothing a module does can add, remove
or reshape a rule.
That half is structural rather than checked, which is the property worth keeping,
because a check is something a later change can be argued past. The renderer has no
parameter through which a port set could reach it, so there is no argument to make
for passing one. Widening the ruleset still takes a privileged act at the agent, and
there are two of them rather than one. A change to the agent's source is the obvious
one. The other is a change to the agent's config store: :open-owner-port and
:edge-port (fulcrum/src/launcher.lisp, read through ubiquitous) each add an
accept line with no source change at all. An operator auditing what a live node
exposes has to read both, and the config is the one that is easy to miss.
Set membership is the half that comes from here. Loading a protocol module ends with valis declaring the whole set of ports it serves, and the agent admits what that declaration names without verifying it. A valis compromised deeply enough to issue a declaration can therefore open a port on the public wire that no owner ever vouched for. The privilege split holds everywhere else, since valis still cannot bind a privileged port, create a namespace or load BPF, but it does not hold for what reaches the public wire.
I put the cost there deliberately, because module trust is not the socket layer's
problem and the socket layer is the worst place to re-decide it. admit-module
verifies the publisher's vouch and the owner's signature against a module's identity
before the loader runs at all, so a module missing either never compiles, never loads
and never registers an adapter. A second verification at the agent would need it to hold keys,
revocation and a clock in order to answer a question the party holding identity has
already answered. What remains is a memory-safety compromise of a vouched image, and
whether the firewall should survive that is answered no, on purpose rather than by
omission.
The order the declaration runs in is part of what the agent is trusting. valis binds and serves a port locally first, declares afterwards, and the agent admits after that, so every port the agent opens is one this node is already answering on. Reverse it and the firewall admits a port with nothing behind it.
⚠ The set the firewall admits and the set the steer carries are still different
objects, and one word doing duty for both is how a reader gets this backwards. The
steer is a catchall and carries every port on the unit's IP inward, declared or not;
the admitted set carries what this node declared. designated-port is a serve-side
key and is still precisely the key the ruleset does not read: grep
fulcrum/src/firewall.lisp for the word and it is not there. Read "the steer carries
it" as "the firewall admits it" and you will believe a port is reachable that is
being dropped at the public IP.
The served set: how an owner reads it, and changes it without a restart
The members of that kernel set are what this node declares, so the next question a reader has is where the declaration lives and who may move it. I made the served set a value held in the image rather than a property of the command line, because a node that can only change what it serves by restarting cannot be administered by its owner: the boot flags are one caller of that value and never its definition.
Read it at /edge/served/list, which gives you one line per port: the number, a
tab, and the port's disposition. serving says the controller has the port bound
and answers traffic on it. declared says the node has been told to serve the port
and nothing is on it yet, which is the ordinary state between an owner declaring a
port and a source coming up on it. ⚠ declared is a success, not a fault: a false
entry in a list of faults makes the real ones easy to miss.
Change the set by writing a verb to /edge/served/ctl,
which takes declare-port or undeclare-port on its first line and the port on its
own line after it. Both doors sit behind the owner control plane
(The owner control plane: a keyed terminus, shipped closed), so reaching them already means having proven the owner's key.
The reply names what the set decided rather than whether the write arrived. Line one
is the outcome and line two names the port. You get declared or undeclared when
the set moved, and already-declared or not-declared when it did not, so repeating
a verb gets you a definite answer rather than a refusal.
⚠ A token carrying -unannounced is a success and reads as one: the set changed in
the image, and the host agent could not be told. A third line then names why, either
no-control-session or declaration-failed. The change is real, the wire has not
caught up, and a reply that called that a failure would invite an owner to retry a
verb that already took.
The verb is per member and the announcement is whole set, and that layering is deliberate rather than incidental. Per-member verbs are what keep two owners editing at once from losing each other's change; a whole-set write is what the host agent parses. Making the wire incremental would undo the first without helping the second.
⛔ Declaring a port does not open it. This surface moves the declaration; the agent's firewall is what admits, and the two are separate acts by design. A port newly declared here reaches the kernel set the firewall accepts from, which is why a declaration needs no privileged config change and no re-render.
⚠ Two gaps can leave a declared port answering nothing, and they are the only two.
The first is the -unannounced reply above: the set moved and the agent was never
told, so nothing reached the kernel set at all. The second is transport, and it is
the one with no reply token to warn you. The set carries no transport dimension, so
declaring 53 says nothing about UDP or TCP, while the accept the agent renders from
that set reads tcp dport @<set> accept. UDP is carried by the fixed :53 rule and
by nothing else. ⇒ Declare a UDP service port and you get a member in the set and no
UDP accept behind it. The subsection below on the two DNS arrival paths is the same
asymmetry seen from the other side.
valis holds nothing durable of its own for the set. The host agent's configuration is where a served set outlives a process, and the node re-declares on every boot, so the value you read here is the running node's answer and not a record of what it will serve next time.
DNS arrives by two paths: UDP inherited, TCP steered
The carve-out is UDP only, and it is physical rather than a preference.
sk_lookup chooses which socket receives an inbound packet but does not govern
the reply: a UDP answer is a fresh datagram that must source from the address and
port the client sent to, or the client rejects it. A TCP connection escapes that,
because once the steer selects the listener the kernel tracks the four-tuple and
sources replies correctly, whereas valis answering UDP from its generic steered
socket would source from the wrong address. The argument covers UDP replies and
nothing else. I have ruled that it must not be stretched to cover TCP: DNS over TCP
is required in a nameserver, and opening a second special case beside the UDP one
buys nothing, so the steer carrying :53 TCP is the correct path.
The kernel program matches that ruling because it never had a port or protocol test
in it. fulcrum/bpf/sk-lookup-catchall.lisp is a lookup of slot zero in a
single-entry sockmap, an sk-assign, and SK_PASS: one map, one program, no port
table, no protocol constant, and the compiled object is byte-identical everywhere it
ships. There is no build variant in which it declines a port, so "transport- and
port-agnostic by construction" above is the whole truth about it.
The privileged agent still binds the unit-IP :53 sockets on both transports and
hands the descriptors down, for the reason that covers every inherited descriptor:
binding a privileged port is the agent's to do, since valis runs unprivileged in the
namespace. What differs by transport is what becomes of those descriptors on a node
whose catchall is live. The inherited UDP socket serves. The inherited TCP socket is
bound, handed down, adopted, and then never receives a connection, because the steer
takes TCP on that port at the unit's address away from it.
What the transport split costs: one service, two independent failure modes
Read this before placing anything else on the DNS path, because it is the price the ruled design charges and nothing in the running system reports it. DNS over TCP rides the steered socket: its accept backlog, its per-port budget, its lifetime. DNS over UDP rides the inherited one. The two transports of one service share a name and share nothing else. A steered socket that saturates, that the edge budget deregisters, or that is torn down takes DNS over TCP with it while UDP keeps answering, and no counter, log line or check anywhere signals the split.
A nameserver answering UDP but not TCP is broken while presenting as healthy. A client that receives a truncated answer retries over TCP and gets nothing, so truncation fallback has no fallback. Zone transfers to a secondary fail, so a secondary serves its last good copy until the zone expires and then goes lame. Both failures surface at a resolver or at a secondary's operator, never at the node.
⚠ A green DNS answer is not evidence about which socket served it, in either
direction. The resident recovers the destination port per accepted client rather
than from the listening socket's own bind, so a hijacked connection's child carries
local port 53 and reaches the DNS handler exactly as an inherited one would. A dig
+tcp returns a perfect authoritative answer under both hypotheses. What separates
them is which listening socket's accept queue the connections enter, the one
measurement a DNS answer cannot give. That is what
proving-ground/driver/scenarios/steered-port-delivery.lisp is built to read: it
aims a burst at the inherited listener's own endpoint and attributes the resulting
queue depth to one listening socket or the other.
⚠ The UDP fall-through is REASONED and has not been measured directly. With a
live catchall the program attempts an assign for UDP too, and inherited UDP :53
keeps working because the kernel does not complete a cross-protocol assign and falls
through to its own socket lookup. That mechanism is inferred from the program's
shape and from UDP continuing to answer on a steered node. No gate observes the
kernel declining the assign, so read it as the reasoning it is rather than as a
measurement.
The owner control plane: a keyed terminus, shipped closed
The owner manages the node (mints and revokes capabilities, edits zones, enrols
keys, drives the mail spine) through the 9P namespace, as a client that mounts the
owner-proven management view. That control plane is deliberately not on the
public surface: run-daemon binds the fabric on the namespace's loopback, the
firewall drops the fabric port at the public IP and drops the owner port too unless
the agent's config deliberately opens it, and an
attach is admitted only after a NoiseXX handshake proves the owner's key
(fabric.lisp's node-auth responder, seven's Tauth=/=Tattach path, mercer's
sealed transport). Reachability and authorization are separate axes: the fabric is
never open, it is authenticated, and loopback-binding is what makes "reachable"
require first arriving on that namespace's loopback. The firewall reserves the
loopback path ("the owner's loopback fabric path") for exactly this.
Reaching that loopback from off the host is a second keyed listener, not a hole in
the first. When the resident inherits an owner-port descriptor it adopts it as the
routable owner terminus (--owner-fd, threaded into start-fabric), and that
adoption asserts the auth seam is engaged before it binds: a routable terminus that
admitted an attach without the Noise handshake would be an open owner surface, so it
fails closed exactly as the non-loopback bind assert does. *fabric-auth-enabled-p*
is what lets the fabric bind a non-loopback host at all, and it does so because the
handshake guards the attach rather than in spite of it.
The port is shipped closed. The host agent's launcher carries :open-owner-port,
defaulting to nil, so the firewall drops the owner port and no owner socket is bound
until an operator deliberately opens it. Opening it is the highest-value application
of the host-agent allowlist above, though not the first: the public TLS edge port
already carries its own accept line under the same mechanism.
⚠ A same-host proof and a cross-host proof are separate propositions, and neither stands in for the other. Driving the fence, the session listing, and a publication from a client over a real sealed socket proves that the behaviour meets a client as opposed to an in-image caller. It proves nothing about reach through the namespace and firewall boundary, which is a different path. Read any "verified live" against which of the two it actually crossed. TODO.org carries where that stands.
⚠ One detail of that path is easy to get backwards. A session admitted over the wire registers under the transport key's DID, because a Noise handshake proves the transport key and derives the principal from it, not from the signing key custody holds for the owner. The owner check therefore accepts either DID. A test that constructs principals in the image never crosses a handshake and so cannot observe this at all; only a test driving a real client does.
Remote module install goes through this terminus. The owner client will gain 9P verbs to install a module and to read the node's module manifest, so you reach a node's module set the way you reach everything else you manage on it (where modules come from).
Node configuration: one store, every key declared
A node whose settings can be changed only by editing the environment its unit starts it with cannot be administered while it runs, so I made a node's configuration durable node state. It lives under the node's data root, and you change it through the same owner control plane (The owner control plane: a keyed terminus, shipped closed) as everything else you manage on the node.
The environment seeds each key once; after that the store is the configuration. At
the first boot that considers a key, the node takes the key's value from its
environment (on a deployed host, /etc/valis/valis.env) and records the key as
settled, whether the environment named a value or not, with two exceptions: a value the
key's type refuses is neither stored nor settled, and is tried again at each boot, and a
database address carrying a password is left unsettled. From then on the store is the
configuration. Edit the environment after that and the edit has no effect on a settled
key: the boot logs the key, by name and never by value, as superseded by the store, so
the journal tells you the edit did nothing. A key that a newer binary declares has
never been settled, so it is seeded once, at that binary's first boot.
Every key is declared, and only what was set is written. Each key the node takes is declared once, with its type, its shipped default, and when a change to it takes effect: at once, through a rebinding of the running code, or at the next restart. A value the key's type refuses is never stored. The store keeps only what was set: a key nobody set, or one reset, is absent from the file and reads as its shipped default, so a default that changes in a later release reaches every node that never chose otherwise. The development listener port is a restart key. A listener that runs whatever code it is sent is opened only when a node starts, never in a running one.
Where it lives. The configuration is <data-root>/config/: node.conf.lisp holds the
node's own settings and modules/<system>.conf.lisp holds each module's. On a deployed
host the data root is <StateDirectory>/valis/, so the directory is
/var/lib/valis/valis/config/. ubiquitous supplies the file format, so you can read
the files with any editor. valis writes them itself, privately (each file 0600 in a 0700
directory) and durably, and reads them as plain data, so nothing a file holds can run.
A running node holds the store's lock, and an offline edit takes the same lock, so the
two never overwrite each other.
Three surfaces, one interface. You reach the configuration from the image through the
Lisp API, from the host through valis config, and from wherever you are through the
owner-only /config directory over the sealed owner session. They are one interface
with one meaning for every setting. valis config talks to /config whenever the node
is running or you name a node, and edits the store directly only when the node is
stopped, as the account owning the data root and no other.
/config/listgives one line per node key: its path, where its value came from, its type, when a change takes effect, and its value./config/modules/<system>/listgives the same for a module's keys, followed by what a version of that module could not carry forward./config/modules/<system>/interviewgives a loaded module's questions in order./config/ctltakesset,resetandpurge.
A change answers with when it takes effect. committed-applied says the value is
stored and in force. committed-on-restart says the value is stored and the node takes
it at the next restart. committed-rebind-failed says the value is stored and the
running code refused it, names the kind of condition, and the next restart applies it.
⚠ All three are successes: the value is durable before anything running is touched.
A refused change is an error naming what was wrong, and nothing is stored. A secret's
value is only ever shown as set or unset.
/config is the owner's alone, as /acme and /backup are. The owner's frame mounts
all three directly and the node's root names none of them, so no grant can designate
them and no other session's view reaches them.
Module settings. A module declares its settings in its define-module form, with a
:settings option giving each key its type, its default, what it is for, and whether a
change applies at once (the default) or at restart, with the reason it must wait. The
settings are filed by the module's ASDF system name, so they outlive any one version of
it. Across an upgrade the module's own migration carries the older version's values
forward; a value nothing carries is kept aside in the file and reported by key, never
dropped. Removing a module keeps its settings, and only your purge deletes them. A
module's settings change only through the running node, since only a node that has
loaded the module can check a value for it. valis config menu walks a module's
settings and valis config interview asks its questions, both when you ask for them;
nothing is asked at install.
Backup and restore. Every backup carries the configuration directory, and a restore lands it before anything else. The restore then places the ACME store where the restored configuration says the node keeps it, so a node comes back with the settings it had and its certificates where those settings look for them. A backup made before backups carried configuration still restores, with a warning that the node takes its settings from its environment at its first boot.
A module reads only its own settings, and that is the shape of the API, not a boundary. A module is offered its own settings and no way to name another's. Nothing stops module code that sets out to reach further, for the reason given under why the owner's reach into a live image cannot be a canonical axis: evaluation cannot be confined from the inside, and one image cannot confine its own code. Read the module API as a convention, never as a wall between modules.
The store is per host. A coincident instance (consistency across coincident instances) or an evacuated one does not see this node's configuration except through a backup and a restore. Moving configuration into the content-addressed store under the shared head belongs with cooperating instances, and this store does not attempt it.
Configuration holds no passwords. The store refuses an operator-state database address that carries a password, and the refusal says why. The node reaches PostgreSQL over its local socket under peer authentication, so the configuration, and every backup that carries it, holds no secret. The files are private all the same, and no log line or refusal carries a value. An address that does carry a password can live only in the unit's environment: the node reads it from there, and every boot logs that the key is held in the environment because it carries a secret.
The mail local-domains key is carried in the store and listed with the rest, but no part of the running node reads it yet.
valis apply is not the node and does not read its store. It checks the site facts a
boot needs against the unit's environment, and its --confirm-zone default is
edge.domain in the configuration under the invoking account's own data root, or
VALIS_EDGE_DOMAIN in the invoking shell.
Where to follow it in code. The declarations, types and node keys are under
src/config/ (declare.lisp, node.lisp); seeding is src/config/seed.lisp, the
store src/config/store.lisp, the readers' one lookup src/config/lookup.lisp, and
change outcomes src/config/apply.lisp. Module settings are
src/config/module.lisp and src/config/module-admin.lisp, and valis config is
src/config/client.lisp with its entry in src/main.lisp. The door is
src/namespace/config.lisp, and the backup, which carries the directory as its
configuration member, is src/backup.lisp.
What :80 is for: the manual a node carries about itself
The design is that every valis serves its own documentation on :80, in the
clear, with that port enabled on every node. I rank it with the owner control
plane rather than with the protocol adapters because it is the surface a person
meets first: before a certificate exists, before a name resolves anywhere, and
before anything else on the node is working. A node whose manual you cannot read
is not one you own, so the manual travels with the node and is answered by the
node.
Plain HTTP is the reason rather than a shortcut. Point a phone, a handheld, a decade-old browser, or a text-mode client on a borrowed machine at the address you were given, and you get the documentation, because nothing in that path asks for a modern cipher suite, a trusted root, or a credential the node must first go and obtain from somebody else. A surface that comes up only after the credential machinery has succeeded cannot be the surface that tells you how to make the credential machinery succeed. That is a sovereignty property before it is a convenience: the one door that is open on a node you have just stood up is the one that explains the rest of it.
The content is the published documentation site the tree already carries
(site/, landing/), served as an ordinary publication. It is also the port an
ACME http-01 responder would answer a challenge on: that use is real and it is
secondary, never the reason the port is open.
How :80 reaches the node, because this is the part the document got wrong and
the error propagated. No descriptor is inherited for it and no flag carries it.
Two decisions meet on that number and they are the same two that govern every
port, held in the two places arrival paths and the caveat on the plugin model
already put them:
- Reachability is the agent's. Its firewall carries one accept line for
:80, elected by its own config and named by no port the node supplies. That election ships ON, because the whole point is a surface that answers before any credential exists. - Dispatch is the node's.
:80is in the set this node declares, the steer delivers it to the one socket like any other declared port, and the port recovered from the accepted client selects the adapter that serves the manual.
⚠ Read those as complementary, never as competing. They are the same shape as
the steer saying "every port to valis" while the firewall says "only these
enter": both are required and neither substitutes for the other. An agent that
admits :80 while the node declares nothing for it opens a port that answers
nothing, and a node that declares :80 while the agent admits nothing reaches
no one. ⛔ Treating the two as rival claims on one number is the error this
section used to license: it said the descriptor was inherited, which made :80
the agent's port, which makes any declaration by the node look like a second
claim on it. Nothing about :80 is privileged to the node. It never binds it.
Direction: unifying the boundary in the kernel
The two-mechanism split (a catchall steer plus a separate firewall) is the
current shape, not the settled end state, and this section records the intended
direction without asserting it as built. The unification is to make the sk_lookup
program itself the allowlist: a map of admitted ports, written by the agent under
privilege, that the program consults, redirecting only admitted ports to valis and
letting the rest fall through to closed. The word matters: the map holds what the
agent admits, which is not everything the steer carries. The boundary then lives in
one place, in the kernel, and the separate firewall becomes defence-in-depth rather
than the primary control. Two constraints survive the unification and are worth
stating so the direction is not mistaken for a cure-all: writing the map must remain
a privileged act of the agent's even though this node's declaration is what its
members name (the invariant above is independent of mechanism), and UDP keeps its
socket-activation carve-out regardless, since the steer cannot source a UDP reply.
That second constraint
carries a consequence worth naming here rather than rediscovering later: with UDP
inherited and TCP admitted through the map, a service offering both transports
still has them arriving on separate sockets with separate fates, so unifying the
boundary does not close the transport split. Any move here changes the
privileged kernel path that decides what the internet reaches, so it is gated on a
steering test harness before the program is touched.
Deployment: condense-from-genesis on a real host
Standing up a valis node on a fresh host and condensing an evaporated unit back into existence are the same operation. Deployment is the genesis case of the reassembly path above: there is no separate installer to maintain, and deploy, evacuate (condense at a new location), and restore (condense from a durable head) unify onto one mechanism. The durable thing is the namespace, so a deploy is a reassembly whose predecessor happens to be empty: the only genesis-specific step is minting the owner Ed25519 seed rather than acquiring it from a predecessor.
The target is a stock Debian host; the resident runtime is a bare
save-lisp-and-die executable supervised by a thin Type=notify systemd unit,
isolated with ip netns, receiving its privileged wire descriptors by inheritance
or SCM_RIGHTS. No container runtime runs on the resident. The contract splits at
the point where condense can first run: adopt below it (the run account, a writable
durable-state directory the account owns, the staged serving binary, a reachable
operator-state database: ordinary host bootstrap), and reuse above it (the netns,
the descriptor handoff, the fail-closed serve loop, the generation fence: valis's own
machinery, driven, never re-implemented by a YAML control plane that would drift from
the live image).
Condense stands a node up; installing a module into a node that already stands is runtime insertion (where modules come from). I keep the development listener's port out of the node template on purpose (the listener's destination).
The bring-up order and the fail-closed invariants
The resident's production entry is run-daemon (src/main.lisp), which shares
run-foreground's fail-closed bring-up: the fabric comes up and the anonymous
public view resolves before the edge binds. For the public :53 service, fulcrum
(privileged) creates the netns and moves a dedicated public NIC wholesale into it (the
launcher's mode :dedicated, with the netns default route via that interface's own
gateway), binds :53 UDP and TCP in the netns,
clears CLOEXEC, and execs the unprivileged valis (ip netns exec → setpriv → env →
valis) with the inherited descriptors; run-daemon adopts them, constructs a
pg-zone-source over the operator-state rows, and drives the sibling serve loop over
the inherited fds, binding no privileged port itself. (Both :53 descriptors are
inherited because binding a privileged port is the agent's to do, but on a node whose
catchall is live only the UDP one goes on to serve: the steer takes :53 TCP away
from the inherited listener, so DNS over TCP arrives by the same route as the steered
push-boot handoff and carries the split set out at what the transport split
costs.) The invariants a
deploy must preserve are the substrate's own: unprivileged valis dies rather than
privileged-bind; a binary without the serving codec fails closed at boot; only the
exposed ports are reachable behind fulcrum's default-deny (the
surface control, the boundary that admits them); and two instances sharing
one database are held to a single writer by the generation fence
(the fence and the sweep). The written contract is the host-deployment
contract.
Public TLS: the credential a name is answered under
A certificate is obtained through a dns-01 challenge, and the authority that answers
that challenge is the node itself. I made the :443 edge cert-gated rather than
required because of that circularity: a node must already be serving :53 before it
can ever be issued a certificate, so a certless genesis boot has to be a healthy
state and not a failure. With no usable certificate in custody for the configured
name, the public edge stays dark and the resident still comes up, and the boot output
says which case held. A silently dark port is indistinguishable from a descriptor
that never arrived, which is the confusion that costs an operator an evening.
The edge holds one credential per name it serves and resolves it per connection, so a request for one name is never answered under another name's certificate. A renewal swaps the live leaf on the same cell the connection path reads, and the listening socket is never touched.
Two constraints carry into any operation on this path. Both are irreversible.
- One certificate per registrable domain. Never SAN-pack. Certificate Transparency records the set of names on a certificate permanently and publicly. Pack several of the owner's domains into one credential and you have published, for good, the fact that one party holds them all. Nothing retracts it afterwards.
- The ACME account key shares the certificate store path. Rename or move that store and the next order registers a fresh account, spending production rate limits that do not come back.
The proving ground: the migration thesis under test
Because deployment is the genesis case of condense, the harness that exercises a
deploy is the same harness that exercises evacuate and restore: the migration-thesis
regression harness. It lives under proving-ground/ as committed, declarative
tooling: a Nix devshell pins the verification fabric (the guests it boots stay stock
Debian, and the core is built impurely from the working tree, never as a Nix
derivation);
Terraform-libvirt boots Debian guests on two distinct VLAN-tagged network locations
so a successor and predecessor sit at genuinely different locations: the topology a
single-VM harness cannot express, and the precondition for the fence-and-reconnect
path; and cloud-init provisions each guest to the adopt-below seam, staging the serving
binary rather than rebuilding it. An in-image Lisp assertion layer (valis-proving-ground,
its own ASDF system, never entering the resident binary) drives three scenarios over
one mechanism (deploy-from-genesis, evacuate-cross-location, restore-from-backup)
through three real interfaces: SSH for unit/netns/fd state, off-host dig for
authoritative :53, and a fabric-liveness probe on the owner-proven loopback path.
Each assertion is exit-coded and fail-closed; a scenario reduces to one go/no-go
verdict. The scenarios assert the effects of the shipped condense and fence path,
they do not re-implement it. See proving-ground/README.org.
The restore scenario exercises a passphrase-escrowed backup of the irreplaceable durable state, and a deployed resident reports substrate-only health so a silent failure surfaces before it becomes an outage. Both surfaces (the backup/restore and passphrase-escrow flow, and the logging/liveness/certificate-expiry observability surface) are documented for the operator in Backup and observability.
System shape (Lisp)
valis is an umbrella of cooperating Common Lisp systems, not a monolith.
The core valis system stays focused (listener, registry, protocol
contract, substrate interfaces) and depends on separate repositories for
the heavy, independently-useful pieces:
- seven: the 9P library carrying the namespace fabric and the bus.
- mercer: the factotum. Transport authentication, key custody, and the ACME certificate lifecycle.
- runciter: the authoritative-DNS answer logic the
:53serving path binds to. - c3po: the wire engines, adopted here as content-addressed edge modules.
- fulcrum: the privileged host agent. eBPF steering and the descriptor handoff.
- Whistler: the eBPF compiler and pure-CL loader fulcrum drives.
- iroh: the transport between cooperating nodes; like libp2p, it has no Common Lisp implementation yet.
- libp2p/IPFS bindings: speaking IPFS to a network running the reference implementation, a dependency the Common Lisp ecosystem does not yet answer. CONSTELLATION.org is the map of which repo owns what.
Within the core:
package-inferred-system: each file'sdefpackagedeclares its own dependencies; there is no central component list. Adding a file undersrc/and importing it is how you extend the system.- A new protocol plugin is, minimally: a new
src/<proto>.lispthat defines aprotocolsubclass (or instance), specialiseshandle-connection, and registers itself on its ports. - Cold-verify every change (fresh image, cleared fasls): a warm REPL
hides missing
:import-fromdeclarations.
Status
This document says what valis is and what it is meant to be. It carries no status, deliberately. A design record that also tracks what has landed goes stale every time something ships, and when a reader catches one wrong paragraph they discount the rest of the document with it.
TODO.org is the status board: where valis stands, what has landed, what is built but
not yet deployed, what remains, and the order it assembles in. It also separates what
has run on the live node from what is merely on main, which is the distinction this
section used to blur.
Bibliography
External resources this design draws on, grouped by the part of the architecture they ground.
eBPF steering
- BPF sklookup program type - the kernel mechanism that lets one socket answer every port (the steering layer).
Listener and event multiplexing
- iolib - the Common Lisp I/O library whose epoll multiplexer the listener is built on.
- Efficient IO with iouring (Jens Axboe) - the eventual multiplexer backend behind valis's own interface.
Authoritative DNS
- RFC 1035 - domain names: the message format and master-file presentation the substrate serves and the wire engine encodes.
- RFC 2308 - negative caching: the apex SOA on a negative answer and the
min(SOA.MINIMUM, SOA.TTL)negative-cache lifetime the seam contract requires. - RFC 4592 - the role of wildcards: the synthesis-and-existence rules the nameserver's lookup tree follows.
- RFC 3597 - handling of unknown RR types: the generic rdata form the record boundary round-trips through.
Plan 9 and 9P
- The Use of Name Spaces in Plan 9 - per-process namespaces and 9P, the model the substrate reifies.
- 9P specification and papers - the protocol that is both substrate and bus.
- v9fs (Linux 9P client) - lets the host kernel mount a valis namespace.
- factotum(4) - Plan 9's auth agent (a 9P file system holding keys); the model for valis's auth agent.
- c9 - a small 9P client and server; reference for a Common Lisp implementation.
- Interim - a Lisp environment where "everything is a file is a symbol"; closest prior art.
- awesome-plan9 - curated index of Plan 9 / 9P software.
Storage, consistency, and migration
- Venti: a new approach to archival storage - content-addressed, write-once block storage (the immutable-bulk model).
- Fossil, an Archival File Server - the mutable file server over Venti (the single-writer mutable head).
- CephFS Dynamic Subtree Partitioning - the per-subtree-authority write-scaling growth path.
- Conflict-free replicated data types - leaderless merge model, considered and rejected for the authoritative namespace.
- Git internals: objects - the canonical immutable-objects + mutable-refs pattern.
- CRIU live migration - checkpoint/restore including established TCP connections (TCPREPAIR); the optional in-flight-continuity enhancement.
- Tahoe-LAFS architecture - a least-authority store where the capability is the name (read/write/verify caps).
Identity, capabilities, and cryptography
- UCAN specification - delegable, attenuatable, public-key/DID-rooted capabilities; the primary token model.
- Macaroons - bearer capabilities with contextual caveats (the attenuation/confinement idea).
- Good Practices for Capability URLs (W3C) - the unguessable-URL-as-capability pattern for anonymous publishing.
- Capability Myths Demolished - ambient authority and the confused deputy; the no-ambient-authority discipline.
- The Noise Protocol Framework - connection-time mutual key authentication (also one of libp2p's secure channels).
- SPKI/SDSI (RFC 2693) - the certificate-capability ancestry of UCAN.
- Decentralized Identifiers (DIDs) (W3C) - the principal identifiers UCAN uses.
Content addressing
- iroh - the transport between cooperating valis nodes.
- IPFS - the content-addressed network valis speaks, as its reference implementation runs it.
- libp2p - IPFS's networking stack, which a future Common Lisp CFFI binding would expose.
- cl-ipfs-api2 - an existing CL HTTP-API client to an external IPFS daemon (the interim bridge).
🄯 Brian O'Reilly <fade@deepsky.com>, 2026