Ollaya runs open-source Jev-style decision models locally, in milliseconds

Ollaya is a locally installable runtime for what its authors call decision models, positioned as a local alternative to hosted TypeSafe Jev. One forward pass returns a typed answer for every question in a request: a choice from a list of criteria, a boolean, a number on a scale or a short string. The runtime is Apache-2.0 and the code is on GitHub.

The first example on the site runs ollaya run laya --preset triage on "I was charged twice this month and want a refund" and gets probabilities instead of prose: intent refund at 1.00, is_urgent no at 0.87 and refund_requested yes at 0.88. Every option carries a probability, so a caller can set a threshold rather than parse generated text.

Latency and calibration

On an NVIDIA RTX 4090, Ollaya measures about 8-10 ms end to end for a five-question Laya request through its HTTP API. For comparison the page cites 236-276 ms as the median latency of the hosted TypeSafe Jev API, taken from third-party benchmarks that include network time; Ollaya notes the setups differ and calls it an order-of-magnitude comparison.

Calibration is the second claim: after temperature fitting, Laya's expected calibration error is 0.081 against 0.246 for Jev. Lower means a probability of 0.9 is closer to a real 90% hit rate, which is what makes fixed thresholds practical.

Compatible with TypeSafe's API

Ollaya serves /v1/systemone and /v1/models with TypeSafe's request and response shapes, and the site says the official TypeSafe Python SDK 0.7.1 works unchanged against a local server. Its invoice example returns intent invoice at confidence 0.9547 with probabilities of 0.9698, 0.0172 and 0.013.

Open weights and platforms

Weights come from their authors' Hugging Face repositories, pinned to a commit and verified against sha256; Ollaya does not re-host them. The starting family is Laya from Convai Innovations: an English model, a 100+ language model and a variant fine-tuned for typed decisions. The server runs on ONNX Runtime, listens only on the local interface by default and uses the CPU or an NVIDIA GPU.

Builds exist for macOS on Apple silicon, Windows 10 and 11 x64, Linux, WSL 2 and Docker on amd64 and arm64. Every model runs on the CPU; an NVIDIA GPU on Linux, WSL 2 or Docker, with driver R580 or newer and CUDA 13, brings a request into the millisecond range.

Why it matters

Decision models are a small-model category for typed classification and scoring that chat APIs handle slowly and at per-token cost: triage, intent, sentiment, routing, moderation flags. Running them locally keeps sensitive messages on the machine that already holds them, removes metering and returns a probability instead of a sentence.