What is RLCD? Inside the reward model that became Jev

A technical post on the di-zhang-llm blog calls RLCD the secret behind Jev, TypeSafe's decision model: multiway preference modeling plus probability calibration, or a schema-conditioned Plackett-Luce objective. Jev turns it into a product with typed outputs and parallel inference, and the reward model no longer hides behind a generator: it becomes the model.

From a scalar reward to a preference probability

Classic reward models return one number for a context and a candidate answer, but the post argues it is not absolute: 0.8 has no stable meaning across problems. LLaMA-Berry's Pairwise Preference Reward Model, or PPRM, made the real signal explicit with a Bradley-Terry probability, trained on almost 7.8 million mathematical-solution pairs with DPO.

Plackett-Luce is the multiway PPRM

Decision interfaces usually see more than two candidates. The Luce choice model, or multinomial logit, normalizes a utility per candidate; with two candidates it reduces exactly to Bradley-Terry. Given a full ranking, the Plackett-Luce likelihood repeatedly picks the next-best remaining candidate, the mathematical center of RLCD.

Calibration turns a distribution into a decision

Normalization is not calibration: a softmax always sums to one, but that does not mean a prediction reported as 0.8 is right 80% of the time. The contract: among predictions at 0.8, roughly 80% should be correct, which TypeSafe also promises for Jev.

The Brier score gives that goal a price: a decision reported at p=0.8 scores 0.04 when the event happens and 0.64 when it does not, so a confidently wrong forecast costs sixteen times as much as a confidently correct one. Calibrated confidence is what decides whether software executes, defers or escalates.

Jev makes the evaluator the interface

In the usual RLHF stack the reward model stays backstage, behind a generator. Jev reverses that: state plus a candidate schema goes in, a calibrated distribution over decisions comes out. Noul, Choice and Score are three schemas over one object, the probability of a typed outcome given a state, a question and a candidate set.

The post also takes apart the parallel inference claim: without the branding, the sampler is sequence packing plus an attention mask followed by typed decision heads, so one forward pass scores every candidate with no token-by-token loop. TypeSafe announces a new architecture but publishes no new attention operator or ablation.

RLCD, the post concludes, is defined by its output contract: a typed probability distribution. Jev is what happens when the reward model stops grading the product and becomes the product.