Showcase / Benchmark
Affordance inside SGLang made replies start 24% sooner on 120,000-token prompts
We put Affordance inside SGLang, an open-source model server, to build the model's prompts. On Qwen3.6-35B-A3B, with prompts over 120,000 tokens, the first token of each reply came after 552 to 562 ms with stock SGLang and 421 to 422 ms with Affordance, over three runs each.
Model, on SGLang
Qwen3.6-35B-A3BPrompt
over 120,000 tokensStock SGLang
552 to 562 msWith Affordance
421 to 422 msChange
24% soonerA language model reads and writes tokens, which are numbers that each stand for a short piece of text. A model server is the program that runs a model and answers requests. Affordance is an open-source program that builds the exact tokens a model reads from an app, a file written in TSX, the language of React web pages.
The work a server does before the model starts
A request arrives as a list of messages. The server joins them into one long string with the model's chat template, a small program written in a template language called Jinja. Then a tokenizer splits that string into tokens.
Both steps start from the first character on every request. An agent sends its whole conversation again at every step, with one new turn at the end. So at step 100, the server joins and tokenizes all 99 earlier turns again.
The change we made
Affordance goes from the messages straight to tokens. It keeps the tokens of turns it has already built, so each request costs only its new turn. We put it into SGLang and into vLLM, another open-source server, in place of their template and tokenizer step. On 2,703 test requests for SGLang and 2,880 for vLLM, including ones with tools, each server produced the same tokens with Affordance as without it.
Three models on two servers
Each run is one conversation that grows by about 1,600 tokens a turn, to 164,000 tokens (117,000 for gpt-oss-20b). We timed how long each reply took to produce its first token. Each server ran three conversations it had never seen.
Then each server ran the same conversations a second time. A server stores the work it did on a prompt it has seen, so the model's own work is small the second time, and building the prompt is most of the wait.
| Model and server, prompts over 120,000 tokens (60,000 to 120,000 for gpt-oss-20b) | Stock server, three runs | With Affordance | Change | Second time: stock | Second time: with Affordance |
|---|---|---|---|---|---|
| Qwen3.6-35B-A3B, SGLang | 552 to 562 ms | 421 to 422 ms | 24% sooner | 165 to 192 ms | 42 to 45 ms |
| Qwen3.6-27B, SGLang | 1,103 to 1,143 ms | 967 to 969 ms | 14% sooner | 187 to 222 ms | 66 to 67 ms |
| gpt-oss-20b, SGLang | 440 to 486 ms | 447 to 466 ms | 2% sooner | 157 to 172 ms | 152 to 166 ms |
| Qwen3.6-35B-A3B, vLLM | 477 to 509 ms | 472 to 476 ms | 1% sooner | 218 to 266 ms | 209 to 213 ms |
| Qwen3.6-27B, vLLM | 1,170 to 1,175 ms | 1,131 to 1,143 ms | 3% sooner | 354 to 359 ms | 330 to 331 ms |
| gpt-oss-20b, vLLM | 394 to 395 ms | 368 to 369 ms | 7% sooner | 88 to 89 ms | 57 to 59 ms |
On SGLang the time saved on the longest prompts was about the same for both Qwen models, 135 to 165 ms. Qwen3.6-35B-A3B runs faster than the 27B model, so the same saving is a larger share of its wait. When that model saw a prompt the second time, its first token came after 165 to 192 ms with stock SGLang and 42 to 45 ms with Affordance.
gpt-oss-20b on SGLang did not change. We think that is because SGLang builds gpt-oss prompts with its own code instead of Jinja. On vLLM, whose tokenizer is already fast, the first token came 7% sooner on gpt-oss-20b and 3% sooner on Qwen3.6-27B.
Qwen3.6-35B-A3B on SGLang
Wait for the first token of a reply, as the conversation grows
Median of three runs, grouped by prompt size in tokens. Shorter is better.
The same wait, the second time the server sees each prompt
Median of three runs. Shorter is better.
Qwen3.6-27B on SGLang
Wait for the first token of a reply, as the conversation grows
Median of three runs. Shorter is better.
Timing the prompt step alone
We also timed the prompt step with no model running, on 200 prompts from one growing conversation. SGLang's template and tokenizer took 17.8 seconds for all 200, and Affordance took 0.47 seconds.
All runs on one machine with four RTX PRO 6000 graphics cards, SGLang v0.5.23 and vLLM v0.33.0.