Privatemode turns GLM-5.3-Flash into a Jev-like decision model

Privatemode AI has shown how an off-the-shelf language model can be turned into a decision model. In a blog post published on 24 September, the German confidential-computing provider reported that GLM-5.3-Flash returns typed decisions, with a probability for every option, in a single forward pass and without fine-tuning.

Models built for this task, such as TypeSafe's Jev and Convai's Laya, return the chosen option together with a confidence value for each. Privatemode's method gives a stock LLM the same behaviour.

How the method works

A language model never writes text directly: at every step it produces a probability distribution over its whole vocabulary. The method exploits exactly that. The state, the question and the numbered options enter the prompt as JSON, and the prompt already ends with the prefix choice_index:, so the model's next token must be an option index. The library does not read that token; it reads the probabilities assigned to all option indexes at that position and normalizes them over the options.

The implementation runs GLM-5.3-Flash on vLLM and uses /chat/completions with continue_final_message and add_generation_prompt: false. vLLM's allowed_token_ids masks the vocabulary down to the option indexes, while logprob_token_ids returns the log probabilities of exactly the requested tokens.

Accuracy on par with Jev

The authors benchmarked three systems on 29 public, labeled datasets with between 2 and 151 options, from intent routing to scanned documents, in English and German. On the 28 text datasets, GLM-5.3-Flash and Jev finish on par: each is more accurate on 10 datasets, eight are within one percentage point, and the median gap of 0.7 points in Jev's favour is not statistically significant (p = 0.64). Locally run Laya, a 421-million-parameter model, trails both by a median of 13 to 15 points.

Latency, cost and images

Timing depends on geography: measured one request at a time, Privatemode answered in 180 ms from Germany against 264 ms for Jev, while from the United States the order reverses, 164 ms for Jev against 299 ms for Privatemode. On price, Jev is cheaper: a million decisions cost about EUR 62 with GLM-5.3-Flash and about EUR 16 with Jev. Jev adds roughly 270 fixed tokens per question plus 10 per option, the GLM-5.3-Flash prompt about 55 plus 20.

Because GLM-5.3-Flash is vision-capable, a question can include images; Jev and Laya are text-only. On RVL-CDIP, 1,600 scanned business documents in 16 classes, the model reaches 70.2% accuracy, with an image adding about 1,350 input tokens. The authors also note a limit: switching on reasoning is more accurate but costs about EUR 350 per million decisions.