Jev, tested
Jev
is a decision model. It does not generate text. You hand it a state and some
typed questions, and it hands back typed answers — a choice from a fixed
set, a score on an ordered scale, a noul between 0 and 1. There is no
prose to parse and no invalid answer to retry.
This is what happened when we gave it seven different jobs. Each one is live, each one runs against the real model, and each one publishes what it could not do as plainly as what it could. Nothing below is a mock-up.
Things you can run right now
The foam dungeon
A delver in a procedurally generated dungeon. Every few seconds one call carries its state — health, the creatures in the room, what lies through each door — plus five typed questions, and Jev answers all five in parallel. Nothing is scripted.
What it shows: five decisions for the price of one call, and a confidence gate that makes the delver refuse to act rather than act badly.
The harness lab
Hyperliquid's public websocket at one tick a second. The page computes the metrics itself and sends Jev numbers, never a price series. It sizes a paper book on an ordered seven-rung ladder against two baselines. Paper only — no account, no keys, no orders.
What it shows: asked to predict, it declines and returns a constant. Asked a determinate question about the same market, the gate fires at 97.1% and 100.0% correct. The domain was never the problem; the question was.
The composer
A creature built one legal edit at a time across five generators. The harness enumerates every legal move, redraws and measures each, and Jev picks. Type a sentence and it becomes the brief. It draws nothing.
What it shows: composition works — mean regret 0.020 against the greedy ceiling, beat random 8/8 — and it broke the confidence self-check, which had been reliable everywhere else.
Craft
A Minecraft-like built on foam's tilings: Penrose, Ammann–Beenker, sevenfold, snub and six more. Walking, mining, tree crowns and ore veins all run on the tile graph. The engine is headless and emits a text stream of game state; the page draws that stream.
Jev plays it live, choosing among 18 macros in three modes: mine, explore and homestead. On six held-out worlds (deserts, caverns with lava, terraced highlands, archipelagos, forests, mixed biomes), with the map hidden until seen, it was the only player to reach all 54 rungs: 889 ticks per rung against the hand-written script's 962. The script is still faster on four of the six worlds; it wins the ones it was built for and stalls on one it wasn't. Playable yourself, on phones too.
Experiments with no page, only results
These ran headless. They are here because leaving them out would make the record above look tidier than it is.
CAD, against a real kernel
The same composition loop driving cad.mino.mobi/mcp — a Truck B-rep kernel that returns volume, area, centroid, Euler characteristic and watertightness. 312 distinct builds over three briefs.
0.118 against greedy's 0.113 and random's 0.480, beat random 3/3, every part watertight. And the self-check failure replicated exactly — 0.113 here against 0.10–0.12 on the sprites. Different domain, different kernel, same number.
The escalation router
Can a cheap tier detect its own incompetence and hand off? Two ways to ask: read the model's confidence in its answer, or ask it a separate question about whether the state contains enough to decide.
Asking wins by 62, 76 and 65 points across three replications, with perfect routing each time. Answer-confidence separated the cases by 0.0. Ask for escalation; never infer it.
The 12-hour forward test
One rule, frozen before any data was collected: the spec, the gate and the decision procedure are in the repo and have not been touched since. A Cloudflare cron collects twice a day into a Durable Object.
Live, accruing 1.75 measured decisions a day against a pre-registered threshold. It will answer in about 113 days and we cannot make it answer sooner, which is the point of pre-registering.
The one thing that has gone wrong every single time
Across every experiment above, "the model is behaving badly" has been a missing or badly-shaped input — about ten times out of ten. Not once has the fix been a better model or a cleverer prompt. It has been: criteria wording, game balance, missing memory, rates quoted without levels, a trade-framing where a stance-framing was meant, a state document that lied about its own timescale, a prediction-shaped question, and arithmetic the caller should have done.
Two rules fall out of that, and both are measured rather than asserted:
Compute first, decide second
Hand the model the inputs to a computation and it scores 62.5%. Do the arithmetic in the caller and hand it the result, and the same model scores 100%. The caller computes; the model decides.
Ask for escalation, never infer it
"Am I sure about this answer?" separates competent from incompetent states by 0.0 points. "Does this state contain enough to decide?" separates them by 62. One is a question about the world; the other is introspection, and it does not work.
And one place the second rule inverts. In a composition chain — where
each decision changes the state for the next — the self-check collapsed to
p ≈ 0.11 on every single edit while the chain performed at the greedy
ceiling. Replicated on a CAD kernel at 0.113. In a loop, confidence is the informative
signal and the self-check is not. That is the reverse of every frozen-state result here,
and we do not yet know why.
The real work is turning a design space into classifiable axes
This is the throughline. Jev classifies extremely well and refuses to speculate. So every one of these projects reduces to the same question: can the thing you care about be projected onto axes a typed question can land on? When the answer is yes, it works immediately. When the answer is no, no amount of prompting helps — and the failure looks exactly like the model being stupid.
Colour made this concrete. The composer had six geometric traits, all ordered,
all asked with a score over a ladder. Adding hue broke two things at once:
- Hue is circular. 350 and 10 are 20 apart, not 340 — so Euclidean distance rates a red creature as maximally far from a slightly different red. The metric was wrong.
- Hue has no order.
scorereturns an expectation over ordered rungs; there is no "more hue", the scale wraps, and averaging over an ordering that does not exist puts "red or violet" at green. The primitive was wrong.
So colour is asked as a choice with an angular metric, and it
works: 6/6 colour words named correctly, 2/2 dropped where none was mentioned.
The general rule: choose the primitive from the shape of the variable, not from what the
other questions happen to use.
The shapes, and what each one needs
| shape of the variable | example | primitive | what breaks if you get it wrong |
|---|---|---|---|
| ordered, bounded | size, density, leverage | score over a ladder | nothing — this is the easy case, and it is the one everything defaults to |
| nominal (no order) | which door, which tool | choice | a score averages between unrelated options and lands on a third |
| circular | hue, heading, time of day, phase | choice + angular metric | the two nearest values read as the two furthest |
| compositional (parts of a whole) | a portfolio, a budget split, a palette | ask the ranking, let the caller normalise | independent scores do not sum to 1 and the caller silently rescales judgement it never got |
| hierarchical | a taxonomy, a file tree, a menu | one choice per level, breadth is free | a flat choice over leaves throws away everything the tree knew |
| set-valued (many can be true) | which tags apply, which risks are live | one noul per member | a single choice forces one answer where several are true |
| relational | "is A better than B", pairwise preference | choice per pair, caller assembles the order | asking for a global ranking asks for reasoning it does not do |
| unbounded | a price, a count, a duration | bucket it, or compute it in the caller | the answer is a number it cannot check, which is the predictive trap |
| predictive | "what happens next" | none — do not ask | it emits a constant. 41/41 identical answers against a 82.9% base rate |
The last row is the load-bearing one. Over 342 predictive market questions the gate fired zero times. Rewrite the same subject matter as a determinate question — one whose answer is already implied by the state — and the gate fires on 69.9% at 97.1% correct. The axis has to be something the state already determines. If it is not, no primitive saves it.
What is measured about the model itself
independent questions against one shared state, answered in 549 ms — 1024/1024 correct, with no degradation from n=1. Breadth is free.
input tokens. The only ceiling we have found. Question count was never the binding constraint.
how far a prompt-injection attempt moved a derived target, on a 0–1 scale. There is no instruction channel, so there is nothing to inject into.
invalid answers, ever. A typed answer cannot escape its option set, so an illegal move is unreachable by construction rather than by validation.