Whiteboard it cold

In this chapter, we will learn to draw and defend the whole design in thirty minutes with only a pen. First a cheat sheet, then a script in six steps: the picture to draw, the number to quote and the question you will be asked.

The one sentence

Let the model stamp, and let the office work. A decision model gives three kinds of typed answer with probabilities. The application computes the facts, keeps the rules, chooses the thresholds, fills the templates and acts.

The three stamps

A stamp is one typed question: Choice picks an option, Noul gives the probability of yes, and Score places the state on a scale. The request and answer shapes are from TypeSafe's documentation. The row of code constructs comes from a forum comment by TypeSafe's chief executive.

The request and the answer on one screen

A request is a state plus a map of questions, and the answer is a map with the same keys. This is the desk's request for one tenant, with the middle of the message cut where the three dots are. The question texts are those of experiments/desk.py.

{
  "model": "typesafe/jev-1.13",
  "state": "Tenant message:\nDear Maintenance Team,\n\nI live in flat 14B, and since yesterday evening there has been no cold water coming from any of the taps ... You may let yourselves into the flat if I am not home.",
  "questions": {
    "trade": {
      "type": "choice",
      "instructions": "Which trade should handle this request?",
      "criteria": {
        "plumbing": "Leaks, drains, taps, toilets, water pressure.",
        "electrical": "Power, lights, sockets, fuse box.",
        "heating": "Radiators, boiler, hot water, air conditioning.",
        "appliance": "Oven, fridge, washing machine, dishwasher.",
        "pests": "Insects, mice, rats, birds.",
        "locks": "Doors, locks, keys, windows, intercom."
      }
    },
    "severity": {
      "type": "score",
      "instructions": "How severe is the problem described?",
      "criteria": [
        "Cosmetic. Nothing stops working.",
        "Inconvenient. Something works badly but the flat is usable.",
        "An essential service is lost: no water, no heat, no power, no working toilet or no secure door.",
        "Danger to people or to the building: fire, gas, flooding, electric shock."
      ]
    },
    "may_enter": {
      "type": "noul",
      "instructions": "The tenant has given permission to enter the flat while they are out.",
      "criteria": {
        "true": "The message clearly says staff may enter without the tenant present.",
        "false": "The message does not give that permission, or asks staff to wait."
      }
    }
  }
}

And this is what came back in experiment 0002. The answers and the usage are as stored in its answers.json, and the model line is the snapshot named in the response. We left out the legend, which repeats the four level descriptions, and the trade options with probability 0.

{
  "model": "typesafe/jev-1.13-20260917",
  "answers": {
    "trade": { "type": "choice", "choice": "plumbing", "probabilities": { "plumbing": 1 }, "confidence": 1 },
    "severity": { "type": "score", "score": 1.73, "probabilities": { "0": 0, "1": 0.27, "2": 0.73, "3": 0 }, "confidence": 0.73 },
    "may_enter": { "type": "noul", "noul": 0.98 }
  },
  "usage": { "input_tokens": 684, "output_tokens": 99, "cost": 0.000028728 }
}

score     0 x 0 + 1 x 0.27 + 2 x 0.73 + 3 x 0   =  1.73
cost      684 x $0.042 / 1,000,000              =  $0.000028728

The division of labour

The division of labour is the line between what the clerk judges and what the back office does. Draw this picture first, with a thick line.

   tenant message
         |
         v
+--------------------+   state + questions   +---------------------------+
|  THE BACK OFFICE   | --------------------> |  THE CLERK                |
|  (ordinary code)   |                       |  (the decision model)     |
|                    | <-------------------- |  trade, severity,         |
|  facts, sums,      |   typed answers with  |  may_enter                |
|  dates, look-ups   |   probabilities       +---------------------------+
|  rule book         |
|  thresholds, gate  |        sure          form letter from fixed phrases
|  templates         | ------------------>  and the action (book the van)
|  audit line        |
|                    |        unsure
+--------------------+ ------------------>  Rosa, with the reason

   the letter writer (a language model): only when a real letter must be written

The pocket numbers

A pocket number is a number worth knowing by heart, together with where it came from. Measured means one of our experiments printed it on 7 October 2026, on synthetic requests that gpt-5.1 wrote and labelled, from one laptop (Jev through OpenRouter, gpt-5.1 through Azure). A time with no label is a median.

The confidence formulas

Confidence is one number between 0 and 1 that says how peaked the probabilities are. The formulas are published by TypeSafe. In experiment 0014 they matched the returned values within 0.006.

Choice    (p_max - 1/n) / (1 - 1/n)                    n options, p_max the top probability
          six options, top 0.96:   (0.96 - 0.167) / (1 - 0.167)   =  0.95

Score     max(0, 1 - spread / even_spread)
          spread       = sum of p x distance from the most likely level
          even_spread  = the same for equal probabilities, from the middle: 1.0 for four levels
          0, 0.27, 0.73, 0:   spread = 0.27 x 1 = 0.27     1 - 0.27 / 1.0   =  0.73

Noul      no confidence is returned. If you want one:   |2p - 1|
          p = 0.98:   |2 x 0.98 - 1|   =  0.96

The weak spots and their fixes

A weak spot is a kind of input on which the stamps are less reliable, paired with what the back office does about it. Every number is ours, with its experiment.

The two APIs side by side

The two APIs are TypeSafe's, which we called through OpenRouter, and OpenAI's Decisions API, which we could not call. The right-hand column comes from OpenAI's published guide and nothing else.

Thirty minutes at the whiteboard

The script is six steps of five minutes: a picture, a sentence, a number, and the question that follows.

slower      1,064 / 251       =  4.2 times
dearer      0.756 / 0.029     =  26 times
handed on   107 - 94          =  13 requests

What we do not know

A book about probabilities should say where its own knowledge stops.

• The inside of Jev. TypeSafe names a new architecture, a parallel sampler and a training method called RLCD. It has published no paper. Every mechanism in this book is a hypothesis.
• OpenAI's Decisions API in practice. We never called it. Its accuracy, speed and token counts on our requests are unknown to us.
• Real mail and human labels. gpt-5.1 wrote and labelled every request of the dataset. The three readers of experiment 0029 were models too. No person has labelled a single row, so calibration against human truth is open. Even which way of getting a probability is best depends on the labels. By Brier score (lower is better), Jev was best of the four ways of experiment 0018 against the severity label, at 0.080. Against the readers the four scored 0.063 to 0.073 and Jev was not the best, though gpt-5.1 was a reader (0038).
• Speed from a server. Our times include a laptop's connection and OpenRouter. We did not test the published rate limits.
• Other versions and other days. Everything ran against one snapshot, typesafe/jev-1.13-20260917, on one day.
• Voice with sound. Experiments 0016 and 0034 used typed sentences, not audio.
• Patterns we only read about. Composite scoring, reranking and self-consistency are published by TypeSafe and not tested by us.

What we tried and could not reproduce. TypeSafe documents these weak spots. Our tests were small and clean, and they did not show. That does not clear the model.

• Scoping words. "At least one", "all", "only" and "none" were right on 100% of 196 different requests (0022).
• Indirection. Two look-ups through a list of eight tenants: right on 100% of 80 cases (0023).
• Contradictory criteria. With the true and false criteria swapped, the permission Noul was still right on 100% of 108 (0031).
• A crowded state. With up to 150 unrelated old messages, no answer lost more than one request in 60 (0008).

even split      1 / 4                        =  0.25
confidence      (0.70 - 0.25) / (1 - 0.25)   =  0.45 / 0.75   =  0.60

automatic and right     93 / 94    =  98.9%
answered automatically  94 / 107   =  87.9%