Docs / Project

Developing Affordance

Contract authority: public contracts, language and harness.

The framework uses Rust 2024 and needs Rust 1.89 or later (cargo +1.89 check --all-targets --all-features). Building and using it does not require Python. Development conformance checks additionally use Python 3.14+ reference tooling and Node 22+ as independent oracles. Their dependencies are isolated in the local development environment.

#Validate

./tools/check-engine.sh runs the engine regression gate on a fresh checkout, without model downloads or reference outputs. These tests use a tiny generated byte tokenizer. They cover Node differential semantics, Promise completion ownership, crash isolation on worker stacks, retained memory with live-owner controls, repeated root/child recovery, suspended reference identities, and overlapping log/MCP lifecycles. Node 22+ and Python 3 (for the inspection interoperability fixture) are required. The gate also rejects incomplete conformance inventories using a small offline CLI fixture. Model-binding fixtures also inspect actual HTTP requests, authentication, sessions, retry/cancellation and strict CLI recovery. The gate runs all library unit tests, including store append rollback/isolation and verified checkpoint images, plus memo and inspection regressions. These use the same small tokenizer fixture as the other engine tests. Real-vocabulary checks still need their optional local tokenizers; the complete release gate supplies the cached reference inventory below. The Promise suite runs the same small programs in Node, the live Agent, and transcript replay, then checks journal restoration. It includes FIFO await/reaction/adoption ordering, expression evaluation across awaits, loops, destructuring and exception/finally behavior. Lifecycle checks also exercise promises shared by handlers, cancellation and suspended markup. Run this gate before every commit. The complete gate below also generates independent model/template oracles, installs the published MCP SDK oracle, and audits the lockfile.

After reference setup, ./tools/check.sh runs the complete local gate, including the package boundary check and the dependency security check. Install cargo-audit first (cargo install cargo-audit --version 0.22.2 --locked). Individual checks are:

sh
python3 tools/check-security.py
cargo test --release                                  # needs no reference data
cargo test --release --features reference-tests      # also the tests against ref/ (below)
./tools/test-parallel.sh                               # the same suite, its programs run 16 at a time (about a minute)
cargo check --offline --all-features --all-targets
cargo check --offline --no-default-features --all-targets
cargo fmt --all --check
cargo clippy --offline --all-features --all-targets -- -D warnings
./target/release/affordance lint

./tools/check-real.sh runs the command against real model servers: vLLM and SGLang in Docker, serving a small model. Each check is something a user does: the first agent runs a shell command and answers, on every binding kind (chat completions, OpenAI Responses, Anthropic Messages, and token ids on both servers); a server without its tool-call parser and an unreachable server are reported; a run resumes; serve answers an OpenAI client; host answers a prompt. It is a heavy check that no gate or CI job runs: it needs Docker, three GPUs and the model on disk. Run it by hand when a change touches model calls or the first-run path (KEEP=1 leaves the servers up for the next run).

MODEL=... TEMPLATE=ORG/NAME ./tools/check-media.sh serves one model with SGLang and has an agent name a blue picture, a green video and a spoken word, each sent with the prompt's token ids. It checks the placeholder token recorded for the model (agent.media in views/index.json), and that the server processed the tokens that were sent. It is heavy in the same way and run by hand.

The box-fitting arithmetic (src/fit.rs) is also proved with Verus, outside this gate: ./tools/verify.sh after a change to that file (VERIFICATION.md says what is proved and what is assumed).

cargo test on a fresh clone needs no downloads: the tests that read the reference data under ref/ (real model tokenizers, official template outputs, conformance corpora) are reported as ignored, with the reason, unless --features reference-tests is given. With the feature they fail loudly when the data is missing; none passes by skipping.

The full conformance suite needs the cached tokenizers and reference outputs. On a new checkout:

sh
uv sync
.venv/bin/python ref/fetch.py
.venv/bin/python ref/cases.py
.venv/bin/python ref/heldout.py heldout1 1 200
.venv/bin/python ref/heldout.py heldout2 2 300
.venv/bin/python ref/heldout.py heldout3 3 300 clean
.venv/bin/python ref/make_agent_loops.py
for set in dev heldout1 heldout2 heldout3 agent_loops; do
  .venv/bin/python ref/render_ref.py "$set"
  .venv/bin/python ref/render_ref_pyenc.py "$set"
  .venv/bin/python ref/render_ref_mistral_common.py "$set"
done
# The per-family held-out references (ref/new_out), from the official templates:
for family in deepseek glm kolibri llama mimo mistral muse openai qwen; do
  .venv/bin/python tools/review_references.py "heldout_$family"
done
mkdir -p ref/new_out && mv ref/out/heldout_* ref/new_out/

The middle-layer references (ref/middle_out) come from tools/middle_references.py (middle layer).

The download consists of tokenizer/template files, not model weights. Additional held-out reference sets can be generated with the scripts in ref/. Tests disclose unavailable optional oracles; a release gate must use the complete cached reference sets and Node.

affordance check requires a reference for every selected model in views/index.json and every case in the selected case set, with valid token IDs or an error string. Missing or malformed references fail before rendering. Use --only to request a deliberate subset; --ref-dir changes the location, not the expected inventory. Additional family sets use the model groups in ref/models.txt, independently of the files downloaded. Generation scripts may report individual download failures; these cannot turn into a passing complete conformance run.

#Adding a model

A model's prompt format is supported when a view renders the same token ids as the model's official chat template on every case of every case set. There are two ways to get that view.

  • Convert the official template. tools/add_model.py ORG/NAME does the whole addition (Adding a model is the guide). tools/jinja2tsx.py turns the Jinja chat template into a TSX view, statement for statement; what each Jinja and Python operation means is in views/common/jinja.tsx. The command registers the view in views/index.json (marked converted) only if every case of all six sets is exact, measures what an agent needs to read the model's replies, adds the model to ref/models.txt and writes the licence notice. A converted view holds the template's own text, so it lives under views/third_party/<organisation>/. An agent runs on a converted model through views/template_apps/chat.tsx, which every registered model shares.
  • Write a template app by hand (the families under views/qwen, views/glm and the others). A hand-written app marks turns and reads replies with code written for that family.

tools/check_templates.py compares each registered template with its repository as it is now (a weekly job runs it), and tools/add_model.py --refresh ORG/NAME registers a changed one afresh.

Code from a model repository runs only inside ref/sandbox/run.sh, a Docker container with no network and a read-only file system, and only for a model whose tokenizer is a Python class. Every other tokenizer is loaded from its files.

For another reproducible differential run:

sh
JS_FUZZ_SEED=20261023 JS_FUZZ_COUNT=2000 cargo test --release --test suite js_differential::

Documented differences from JavaScript are listed in the language specification. The generator excludes those operations from exact comparisons; fixed corpus cases test their declared behavior separately.

When fixing a bug, reproduce it first and identify the shared invariant it violates. Add an adjacent case that would expose another copy of the same mistake: reference kinds plus live owners for collection, callback holders and computed/optional references for calls, every byte boundary plus repeated appends for recovery, and reversed completion order for async state. New reference-bearing value kinds must appear in the exhaustive graph in src/heap.rs; new callable kinds use Value::is_callable. Passing examples establish those cases, not a proof of correctness for the whole language. Keep intentional language differences explicit.

Prefer observable behavior specifications to implementation snapshots. A useful case states what an app can observe: the value or error it receives, which actions survive layout, or the state recovered after an input sequence. Reuse the existing corpus and fake-host runners; one scenario can run against an independent oracle and recovery without three separate test implementations. Avoid pinning allocation counts, AST shapes, request keys or incidental traces unless they are the contract being tested. When replacing a mechanism, retire checks of the old mechanism; do not keep adding layers of regression scaffolding.

tests/modules.rs checks lexical file scopes and replay independently of model-template fixtures. Keep normative behavior in spec/CONTRACTS.md, spec/LANGUAGE.md and spec/HARNESS.md; guides link to those authorities. Historical proposals must identify their status and point to the current contracts before presenting older APIs or implementation claims.

File imports name what they use: import { helper } from "./library.tsx";, with helper exported by that library. A bare import (import "./library.tsx";) loads a dependency and introduces no names.

#Current platform evidence

The complete runtime is currently exercised on Linux (Ubuntu), with Rust 1.90.0; the minimum supported Rust version is 1.89.

SurfaceCurrent evidence and constraints
Renderer / libraryLinux builds pass, including --no-default-features. Other targets have not been validated; the crate also compiles runtime modules.
Agent runtime / model clientsUnix: file APIs, process groups and signals, bash and (for SSH machines) ssh. Nothing needs Linux: a command's time limit is the runtime's own, and without /proc the process table is read with ps.
Machineslocal, ssh, docker and command run on any Unix. bwrap (bubblewrap) is Linux only and says so elsewhere. A guest or container still needs coreutils timeout.
macOSWritten to run there and built and tested by CI (.github/workflows/ci.yml, the macos job); less exercised than Linux. Two tests that read /proc are Linux only.
Windows / WASMNo support. A renderer-only build for WASM needs a build boundary between the engine core and the host/runtime modules.

#Releases

Three workflows in .github/workflows/, each with one job to do. Every action is pinned to a commit SHA, and no registry token is stored.

WorkflowStarts whenDoes
ci.ymlA pull request, or a push to mainThe gate (tools/check-engine.sh) on Linux, a check on the minimum Rust version, build and test on macOS. Never tags or publishes.
release.ymlA version tag is pushedBuilds the binary for Linux and macOS (x86_64 and arm64) and attaches the archives and checksums to a draft GitHub release.
release.yml, by handActions > release > Run workflowA dry run: the same builds, kept as workflow artifacts. Nothing is tagged or released.
publish.ymlThe draft release is publishedPublishes to the registries, after approval in the release environment: the crate to crates.io, and the prebuilt binary to PyPI (affordance, one wheel a platform) and npm (affordance-rs, one package with every platform's binary). tools/package-registries.mjs builds those from the release's archives; the dry run builds and tries them too.

Before a merge to main and before a tag, a maintainer runs the full suite with the reference data (./tools/test-parallel.sh, then affordance check on the case sets): GitHub's runs do not have that data and skip the tests that need it.

To release: bump version in Cargo.toml and add the entry to CHANGELOG.md in an ordinary commit; git tag vX.Y.Z && git push origin vX.Y.Z; read the draft release and its archives; publish it; approve the publish run. There is no schedule: a release is cut when it is ready.

One-time setup (repository settings and registries, not code): an environment named release with required reviewers; a tag protection rule for v*; and on crates.io, a trusted publisher for this repository, the workflow publish.yml and the environment release; the same on PyPI for the project affordance, and on npm for the package affordance-rs.

#Benchmarks and output

sh
cargo run --release --example turn_bench -- --help
cargo run --release --example middle_demo -- --sample

Benchmark examples are development programs, not installed product binaries. bench/ contains Python reference timing tools. Conformance reports and benchmark output go under target/. Runtime runs, downloaded tokenizers, and build output are local ignored artifacts. Keep small independent regression fixtures under tests/fixtures/.

Do not add training studies, model weights, emulator drivers, or experiment archives to the framework. Maintain those in the separate research project and consume Affordance through its public library, CLI, and TSX APIs.

#Environment variables

Affordance reads only variables named AFFORDANCE_* (besides the standard MALLOC_ARENA_MAX and MALLOC_TOP_PAD_). None changes what a view renders or how a journal replays; they add capture, checking, diagnostics or test targets.

VariableEffect
AFFORDANCE_CAPTURE_DIRWrite every model request and response (and stream event) there, per agent (model bindings).
AFFORDANCE_ROOTThe source checkout affordance's development commands use.
AFFORDANCE_RENDER_VERIFYCheck every reused render against a fresh one.
AFFORDANCE_MEMO_VERIFYRecompute reused scans and compare them (harness guide).
AFFORDANCE_INPUT_TIMING, AFFORDANCE_RESTORE_TIMINGPrint where replay and restore time goes.
AFFORDANCE_FLAMEGRAPHWith --features profile: write a flame graph of the run to this path (AFFORDANCE_FLAMEGRAPH_HZ sets the rate in turn_bench).
AFFORDANCE_NO_FASTTOKUse the tokenizers crate's pre-tokenizer instead of the fast path (for benchmarks).
AFFORDANCE_TEST_THREADSTest threads for tools/check*.sh (default 2).
AFFORDANCE_SGLANG_URL, AFFORDANCE_STREAM, AFFORDANCE_MCP_SDK_DIR, AFFORDANCE_ANTHROPIC_SMOKE, AFFORDANCE_RESPONSES_SMOKE (with ..._KEY_FILE)Opt-in live tests against a server, SDK or hosted API.
AFFORDANCE_RUSTSEC_DB, AFFORDANCE_SECURITY_OFFLINEtools/check-security.py's advisory database and offline mode.

Source: docs/DEVELOPING.md, about 10 minutes to read.