Showcase / Built with Affordance
We trained Qwen3-4B to work in a plain document instead of a chat, from 11% to 78% right
We gave Qwen3-4B, a chat model, a task written as a plain document instead of a chat. At first it answered 10.9% of attempts correctly. After 70 steps of training it answered 77.7%, the same as the model trained in its usual chat format.
Model
Qwen3-4BRight answers, at first
10.9%After 70 steps
77.7%A language model reads and writes tokens, which are numbers that each stand for a short piece of text. A chat model is trained on conversations laid out one exact way, with tokens that mark who is speaking. The program that lays them out is called a chat template. Our document has no speakers and does not use the chat template.
We trained with reinforcement learning. The model tries the task many times, and the attempts that got the right answer are made more likely. Affordance is an open-source program that builds the exact tokens a model reads from an app, a file written in TSX, the language of React web pages. We wrote the document layout as a 24-line Affordance app.
The task and the document
The model sees a notebook of 16 to 48 four-digit numbers and has to submit the largest. It can only see about 40 tokens of the notebook at a time, a few rows. It has three tools: scroll the notebook, write itself a note, and submit an answer.
The document starts with the task and the list of tools. Then each numbered step shows the notebook, then Action: and the model's reply, then Result:. This is the start of one attempt before training, with the model's own tokens highlighted:
Task: The notebook below has 21 numbers, one per row, but you can only see a few rows at a time. Find the largest number in the notebook and submit it. Use scroll to move through the notebook and note to keep track of what you have seen.
Tools. Take one step at a time by calling one tool, written as
<tool_call>
{"name": <tool name>, "arguments": {<arguments>}}
</tool_call><|im_end|>
- scroll: Scroll the notebook by a number of rows (negative scrolls up). Arguments: {"by":{"type":"number"}}
- note: Replace your notes with new text. Arguments: {"text":{"type":"string"}}
- submit: Submit your final answer: the largest number in the notebook. This ends the task. Arguments: {"answer":{"type":"number"}}
Step 1
Notebook (21 rows), starting at row 0:
0: 8857
1: 7298
2: 8991
3:...
Your notes: (empty)
Action:
<tool_call>
{"name": "scroll", "arguments": {"by": 1}}
</tool_call><|im_end|>
Result: The notebook now starts at row 1.
Step 2
Notebook (21 rows), starting at row 1:
1: 7298
2: 8991
3: 3598
4:...
Your notes: (empty)
Action:
<tool_call>
{"name": "scroll", "arguments": {"by": 1}}
</tool_call><|im_end|>
Result: The notebook now starts at row 2.
Step 3
Notebook (21 rows), starting at row 2:
2: 8991
3: 3598
4: 6564
5:...
Your notes: (empty)
Action:
<tool_call>
{"name": "scroll", "arguments": {"by": 1}}
</tool_call><|im_end|>
Result: The notebook now starts at row 3.
13 more steps: the model scrolls by 1 each time until it runs out of stepsThe model writes its tool calls in the form it learned in chat, and ends each step with <|im_end|>, the special token that ends its turn in a chat. The tool list shows both in its example. Our first version left <|im_end|> out of the example, and the model kept writing past its own step in 82% of replies. With the token in the example, 96% of its replies were well-formed tool calls.
Here is the app that lays out the document:
export function DocApp({ children }) {
const input = templateInput(children, null, true);
const fns = input.tools.map(t => t.function);
let step = 0;
const body = input.turns.map((m, i) => {
const content = typeof m.content === "string" || isFragment(m.content) ? m.content : "";
if (i === 0) return ["Task: ", content, "\n\nTools. Take one step at a time by calling one tool, written as\n", tok.tool_call,
"\n{\"name\": <tool name>, \"arguments\": {<arguments>}}\n", tok.tool_call_end, tok.im_end, "\n",
fns.map(f => `- ${f.name}: ${f.description} Arguments: ${JSON.stringify(f.parameters.properties)}\n`).join("")];
if (m.role === "user") { step += 1; return [`\nStep ${step}\n`, content]; }
if (m.role === "assistant") return ["Action:\n", <turn role="assistant">{tokens(m.ids)}</turn>, "\n"];
if (m.role === "tool") return ["Result: ", content, "\n"];
return "";
});
return [<>{body}{"Action:\n"}</>,
<span label="protocol" syntax="hermes" stop={[tok.im_end.id, tok.endoftext.id]} in_think={false}/>];The model's replies go back into the document as tokens(m.ids), the numbers of the tokens it wrote, unchanged.
Before training
In the document, the base model scrolled one row at a time and rarely answered. It submitted an answer in 24% of attempts, against 95% in its chat format. It answered 10.9% correctly, against 40.2% in chat.
Accuracy during training
Share of held-out attempts answered correctly, by training step
The same 256 notebooks, never trained on, every 10 steps. Same model, task and training settings in both runs.
The document run went from 10.9% to 62.1% in the first ten steps, and caught up with the chat run by step 50.
| Held-out attempts in the document | Before training | After 70 steps |
|---|---|---|
| Attempts that ended with an answer | 23.8% | 100.0% |
| Attempts that scrolled to the last row | 26.2% | 87.1% |
| Attempts answered correctly | 10.9% | 77.7% |
| Tokens the model wrote per attempt | 489 | 404 |
An attempt after training
The trained model scrolls 3 rows at a time, writes every number it has seen into its notes, and submits. This attempt is from the final test:
the task, the tools, and steps 1 to 4, each scrolling by 3
Step 5
Notebook (20 rows), starting at row 12:
12: 4978
13: 6864
14: 231...
Your notes: (empty)
Action:
<tool_call>
{"name": "scroll", "arguments": {"by": 3}}
</tool_call><|im_end|>
Result: The notebook now starts at row 15.
Step 6
Notebook (20 rows), starting at row 15:
15: 4442
16: 9334
17: 708...
Your notes: (empty)
Action:
<tool_call>
{"name": "scroll", "arguments": {"by": 3}}
</tool_call><|im_end|>
Result: The notebook now starts at row 18.
Step 7
Notebook (20 rows), starting at row 18:
18: 6706
19: 5961
Your notes: (empty)
Action:
<tool_call>
{"name": "note", "arguments": {"text": "Collected numbers: 7014, 6124, 8299, 8775, 8796, 7926, 8054, 3523, 3292, 2168, 7515, 2886, 4978, 6864, 231, 4442, 9334, 708"}}
</tool_call><|im_end|>
Result: Noted.
Step 8
Notebook (20 rows), starting at row 18:
18: 6706
19: 5961
Your notes: Collected numbers: 7014, 6124, 8299, 8775, 8796, 7926, 8054, 3523, 3292, 2168, 7515, 2886, 4978, 6864, 231, 4442, 9334, 708
Action:
<tool_call>
{"name": "submit", "arguments": {"answer": 9334}}
</tool_call><|im_end|>Its notes include 231 and 708. Those are rows 14 and 17, which the notebook window cut off after three digits, and the model copied the digits it could see.
Why the trained model still misses some
The 40-token window shows three full rows at the top of the notebook. From row 10 on, the row numbers have two digits and take more tokens, so only two rows fit and the third is cut off. The model learned to scroll by 3 in the first rows and kept doing it, so after row 10 it never sees every third number in full.
Of the 57 attempts it got wrong after training, 54 were notebooks whose largest number it never saw in full. The model trained in chat misses for the same reason: 58 of its 58 wrong answers.
Comparison with the server's records
We recorded every request to SGLang, the model server, and every reply. Then we compared the tokens Affordance exported for training with those records.
| Training and testing | Count |
|---|---|
| Attempts | 19,968 |
| Model calls | 236,207 |
| Prompt tokens | 180,963,409 |
| Tokens the model wrote | 8,426,505 |
| Tokens that differed from the server's records | 0 |
One training run on one task, with a 4-billion-parameter model.