Developing Affordance
Contract authority: public contracts, language and harness.
The framework uses Rust 2024 and needs Rust 1.89 or later (cargo +1.89 check --all-targets --all-features). Building and using it does not require Python. Development
conformance checks additionally use Python 3.14+ reference tooling and Node 22+ as independent
oracles. Their dependencies are isolated in the local development environment.
#Validate
./tools/check-engine.sh runs the engine regression gate on a fresh checkout, without model
downloads or reference outputs. These tests use a tiny generated byte tokenizer. They cover
Node differential semantics, Promise completion ownership, crash isolation on worker stacks,
retained memory with live-owner controls, repeated root/child recovery, suspended reference identities, and overlapping log/MCP
lifecycles. Node 22+ and Python 3 (for the inspection interoperability fixture) are required. The gate also rejects incomplete conformance inventories
using a small offline CLI fixture. Model-binding fixtures also inspect actual HTTP requests,
authentication, sessions, retry/cancellation and strict CLI recovery.
The gate runs all library unit tests, including store append rollback/isolation and verified
checkpoint images, plus memo and inspection regressions. These use the same small tokenizer
fixture as the other engine tests. Real-vocabulary checks still need their optional local
tokenizers; the complete release gate supplies the cached reference inventory below.
The Promise suite runs the same small programs in Node, the live Agent, and transcript replay,
then checks journal restoration. It includes FIFO await/reaction/adoption ordering, expression
evaluation across awaits, loops, destructuring and exception/finally behavior. Lifecycle checks
also exercise promises shared by handlers, cancellation and suspended markup.
Run this gate before every commit. The complete gate below also generates independent
model/template oracles, installs the published MCP SDK oracle, and audits the lockfile.
After reference setup, ./tools/check.sh runs the complete local gate, including the package
boundary check and the dependency security check. Install cargo-audit
first (cargo install cargo-audit --version 0.22.2 --locked). Individual checks are:
python3 tools/check-security.py
cargo test --release # needs no reference data
cargo test --release --features reference-tests # also the tests against ref/ (below)
./tools/test-parallel.sh # the same suite, its programs run 16 at a time (about a minute)
cargo check --offline --all-features --all-targets
cargo check --offline --no-default-features --all-targets
cargo fmt --all --check
cargo clippy --offline --all-features --all-targets -- -D warnings
./target/release/affordance lint./tools/check-real.sh runs the command against real model servers: vLLM and SGLang in Docker,
serving a small model. Each check is something a user does: the first agent runs a shell command
and answers, on every binding kind (chat completions, OpenAI Responses, Anthropic Messages, and
token ids on both servers); a server without its tool-call parser and an unreachable server are
reported; a run resumes; serve answers an OpenAI client; host answers a prompt. It is a heavy
check that no gate or CI job runs: it needs Docker, three GPUs and the model on disk. Run it by
hand when a change touches model calls or the first-run path (KEEP=1 leaves the servers up for
the next run).
MODEL=... TEMPLATE=ORG/NAME ./tools/check-media.sh serves one model with SGLang and has an
agent name a blue picture, a green video and a spoken word, each sent with the prompt's token
ids. It checks the placeholder token recorded for the model (agent.media in
views/index.json), and that the server processed the tokens that were sent. It is heavy in the
same way and run by hand.
The box-fitting arithmetic (src/fit.rs) is also proved with Verus, outside this gate:
./tools/verify.sh after a change to that file (VERIFICATION.md says what
is proved and what is assumed).
cargo test on a fresh clone needs no downloads: the tests that read the reference data under
ref/ (real model tokenizers, official template outputs, conformance corpora) are reported as
ignored, with the reason, unless --features reference-tests is given. With the feature they
fail loudly when the data is missing; none passes by skipping.
The full conformance suite needs the cached tokenizers and reference outputs. On a new checkout:
uv sync
.venv/bin/python ref/fetch.py
.venv/bin/python ref/cases.py
.venv/bin/python ref/heldout.py heldout1 1 200
.venv/bin/python ref/heldout.py heldout2 2 300
.venv/bin/python ref/heldout.py heldout3 3 300 clean
.venv/bin/python ref/make_agent_loops.py
for set in dev heldout1 heldout2 heldout3 agent_loops; do
.venv/bin/python ref/render_ref.py "$set"
.venv/bin/python ref/render_ref_pyenc.py "$set"
.venv/bin/python ref/render_ref_mistral_common.py "$set"
done
# The per-family held-out references (ref/new_out), from the official templates:
for family in deepseek glm kolibri llama mimo mistral muse openai qwen; do
.venv/bin/python tools/review_references.py "heldout_$family"
done
mkdir -p ref/new_out && mv ref/out/heldout_* ref/new_out/The middle-layer references (ref/middle_out) come from tools/middle_references.py
(middle layer).
The download consists of tokenizer/template files, not model weights. Additional held-out
reference sets can be generated with the scripts in ref/. Tests disclose unavailable optional
oracles; a release gate must use the complete cached reference sets and Node.
affordance check requires a reference for every selected model in views/index.json and every
case in the selected case set, with valid token IDs or an error string. Missing or malformed
references fail before rendering. Use --only to request a deliberate subset; --ref-dir
changes the location, not the expected inventory. Additional family sets use the model groups
in ref/models.txt, independently of the files downloaded. Generation scripts may report
individual download failures; these cannot turn into a passing complete conformance run.
#Adding a model
A model's prompt format is supported when a view renders the same token ids as the model's official chat template on every case of every case set. There are two ways to get that view.
- Convert the official template.
tools/add_model.py ORG/NAMEdoes the whole addition (Adding a model is the guide).tools/jinja2tsx.pyturns the Jinja chat template into a TSX view, statement for statement; what each Jinja and Python operation means is inviews/common/jinja.tsx. The command registers the view inviews/index.json(markedconverted) only if every case of all six sets is exact, measures what an agent needs to read the model's replies, adds the model toref/models.txtand writes the licence notice. A converted view holds the template's own text, so it lives underviews/third_party/<organisation>/. An agent runs on a converted model throughviews/template_apps/chat.tsx, which every registered model shares. - Write a template app by hand (the families under
views/qwen,views/glmand the others). A hand-written app marks turns and reads replies with code written for that family.
tools/check_templates.py compares each registered template with its repository as it is now
(a weekly job runs it), and tools/add_model.py --refresh ORG/NAME registers a changed one
afresh.
Code from a model repository runs only inside ref/sandbox/run.sh, a Docker container with no
network and a read-only file system, and only for a model whose tokenizer is a Python class.
Every other tokenizer is loaded from its files.
For another reproducible differential run:
JS_FUZZ_SEED=20261023 JS_FUZZ_COUNT=2000 cargo test --release --test suite js_differential::Documented differences from JavaScript are listed in the language specification. The generator excludes those operations from exact comparisons; fixed corpus cases test their declared behavior separately.
When fixing a bug, reproduce it first and identify the shared invariant it violates. Add an
adjacent case that would expose another copy of the same mistake: reference kinds plus live
owners for collection, callback holders and computed/optional references for calls, every byte
boundary plus repeated appends for recovery, and reversed completion order for async state.
New reference-bearing value kinds must appear in the exhaustive graph in src/heap.rs;
new callable kinds use Value::is_callable. Passing examples establish those cases, not a
proof of correctness for the whole language. Keep intentional language differences explicit.
Prefer observable behavior specifications to implementation snapshots. A useful case states what an app can observe: the value or error it receives, which actions survive layout, or the state recovered after an input sequence. Reuse the existing corpus and fake-host runners; one scenario can run against an independent oracle and recovery without three separate test implementations. Avoid pinning allocation counts, AST shapes, request keys or incidental traces unless they are the contract being tested. When replacing a mechanism, retire checks of the old mechanism; do not keep adding layers of regression scaffolding.
tests/modules.rs checks lexical file scopes and replay independently of model-template
fixtures. Keep normative behavior in spec/CONTRACTS.md, spec/LANGUAGE.md and
spec/HARNESS.md; guides link to those authorities. Historical proposals must identify their
status and point to the current contracts before presenting older APIs or implementation claims.
File imports name what they use: import { helper } from "./library.tsx";, with helper
exported by that library. A bare import (import "./library.tsx";) loads a dependency and
introduces no names.
#Current platform evidence
The complete runtime is currently exercised on Linux (Ubuntu), with Rust 1.90.0; the minimum supported Rust version is 1.89.
| Surface | Current evidence and constraints |
|---|---|
| Renderer / library | Linux builds pass, including --no-default-features. Other targets have not been validated; the crate also compiles runtime modules. |
| Agent runtime / model clients | Unix: file APIs, process groups and signals, bash and (for SSH machines) ssh. Nothing needs Linux: a command's time limit is the runtime's own, and without /proc the process table is read with ps. |
| Machines | local, ssh, docker and command run on any Unix. bwrap (bubblewrap) is Linux only and says so elsewhere. A guest or container still needs coreutils timeout. |
| macOS | Written to run there and built and tested by CI (.github/workflows/ci.yml, the macos job); less exercised than Linux. Two tests that read /proc are Linux only. |
| Windows / WASM | No support. A renderer-only build for WASM needs a build boundary between the engine core and the host/runtime modules. |
#Releases
Three workflows in .github/workflows/, each with one job to do. Every action is pinned to a
commit SHA, and no registry token is stored.
| Workflow | Starts when | Does |
|---|---|---|
ci.yml | A pull request, or a push to main | The gate (tools/check-engine.sh) on Linux, a check on the minimum Rust version, build and test on macOS. Never tags or publishes. |
release.yml | A version tag is pushed | Builds the binary for Linux and macOS (x86_64 and arm64) and attaches the archives and checksums to a draft GitHub release. |
release.yml, by hand | Actions > release > Run workflow | A dry run: the same builds, kept as workflow artifacts. Nothing is tagged or released. |
publish.yml | The draft release is published | Publishes to the registries, after approval in the release environment: the crate to crates.io, and the prebuilt binary to PyPI (affordance, one wheel a platform) and npm (affordance-rs, one package with every platform's binary). tools/package-registries.mjs builds those from the release's archives; the dry run builds and tries them too. |
Before a merge to main and before a tag, a maintainer runs the full suite with the reference
data (./tools/test-parallel.sh, then affordance check on the case sets): GitHub's runs do
not have that data and skip the tests that need it.
To release: bump version in Cargo.toml and add the entry to CHANGELOG.md in an ordinary
commit; git tag vX.Y.Z && git push origin vX.Y.Z; read the draft release and its archives;
publish it; approve the publish run. There is no schedule: a release is cut when it is ready.
One-time setup (repository settings and registries, not code): an environment named release
with required reviewers; a tag protection rule for v*; and on crates.io, a trusted publisher
for this repository, the workflow publish.yml and the environment release; the same on PyPI
for the project affordance, and on npm for the package affordance-rs.
#Benchmarks and output
cargo run --release --example turn_bench -- --help
cargo run --release --example middle_demo -- --sampleBenchmark examples are development programs, not installed product binaries. bench/ contains
Python reference timing tools. Conformance reports and benchmark output go under target/. Runtime runs, downloaded
tokenizers, and build output are local ignored artifacts. Keep small independent regression fixtures under tests/fixtures/.
Do not add training studies, model weights, emulator drivers, or experiment archives to the framework. Maintain those in the separate research project and consume Affordance through its public library, CLI, and TSX APIs.
#Environment variables
Affordance reads only variables named AFFORDANCE_* (besides the standard MALLOC_ARENA_MAX and
MALLOC_TOP_PAD_). None changes what a view renders
or how a journal replays; they add capture, checking, diagnostics or test targets.
| Variable | Effect |
|---|---|
AFFORDANCE_CAPTURE_DIR | Write every model request and response (and stream event) there, per agent (model bindings). |
AFFORDANCE_ROOT | The source checkout affordance's development commands use. |
AFFORDANCE_RENDER_VERIFY | Check every reused render against a fresh one. |
AFFORDANCE_MEMO_VERIFY | Recompute reused scans and compare them (harness guide). |
AFFORDANCE_INPUT_TIMING, AFFORDANCE_RESTORE_TIMING | Print where replay and restore time goes. |
AFFORDANCE_FLAMEGRAPH | With --features profile: write a flame graph of the run to this path (AFFORDANCE_FLAMEGRAPH_HZ sets the rate in turn_bench). |
AFFORDANCE_NO_FASTTOK | Use the tokenizers crate's pre-tokenizer instead of the fast path (for benchmarks). |
AFFORDANCE_TEST_THREADS | Test threads for tools/check*.sh (default 2). |
AFFORDANCE_SGLANG_URL, AFFORDANCE_STREAM, AFFORDANCE_MCP_SDK_DIR, AFFORDANCE_ANTHROPIC_SMOKE, AFFORDANCE_RESPONSES_SMOKE (with ..._KEY_FILE) | Opt-in live tests against a server, SDK or hosted API. |
AFFORDANCE_RUSTSEC_DB, AFFORDANCE_SECURITY_OFFLINE | tools/check-security.py's advisory database and offline mode. |
Source: docs/DEVELOPING.md, about 10 minutes to read.