Bring your own engine

A decision engine looks at what happened in calls and returns a move for every state:

state  →  move          e.g.   "2:4:budget"  →  "handle_objection"

A state is stage:trust:objection, which turn of the call (0–3), how much the customer trusts us in tenths (0–10), and what they are pushing back on. A move is one of the Arena’s eight: greet, give_details, confirm, handle_objection, reassure_family, propose_viewing, offer_slot, close. That mapping is the engine. Everything else, how it was learned, what it needed, how sure it is, is declared beside it or reported with it. You write three things. The Arena already has fourteen engines built in; this page is how to add your own, in your own repo, with your own dependencies, without touching ours.

The one idea worth holding onto: fit is the contract; where it runs is a deployment detail. The same function fits on your laptop for iteration and as a levers9 job with a GPU for scale. The Arena does not know the difference, and neither does your code.

This is the whole path from the calls a world has already stored to a verdict on your engine: every step of it either a file you can open or one HTTP call: Both refusals happen inside run, which is why they arrive as a sentence rather than an index error halfway through an algorithm: the evidence is checked before your function is called, and the policy is checked before anything is measured. Nothing on the way back up writes to the Arena, a measurement is not more evidence, so the only thing that improves the top of this picture is more, and more varied, calls.

In ten minutes

Get the sample evidence

The demo folder ships 300 episodes from the Arena’s real simulator: the Saudi real-estate pack, its personas and objections, its judge.

cd demo/decision_engine
ls
#  app.py  evidence.sample.json  README.md

Or pull a real world’s evidence from your Arena, to a file you can open (after levers9 login):

levers9 engine worlds                     # acme · saudi_real_estate · 544 calls · simulator yes
levers9 engine evidence acme -o acme.json

from levers9 import Evidence
evidence = Evidence.load("acme.json")      # or Evidence.fetch("acme") to skip the file

Run the example

python app.py

evidence: {'pack': 'saudi_real_estate', 'episodes': 300, 'transitions': 1200, 'states': 47, 'moves': 8,
           'pairs': 829, 'propensity': True, 'simulator': False}

fitting optimistic-greedy here …
  47 states · 376 (state, move) pairs seen · 14 still exploring

  turn 0 · trust 0.3 · skeptic   →  offer_slot
  turn 0 · trust 0.4 · budget    →  reassure_family
  ...

That is a policy. Fourteen states “still exploring” means the engine picked an untried move on optimism alone: 300 episodes across 8 moves and 47 states is thin, and it says so rather than hiding it.

Read the three things

Open app.py. The whole contract is there and it is short.

from levers9 import Blueprint, Evidence, Param, decision_engine

@decision_engine(                                       # 1. declare it
    name="optimistic-greedy", family="Bandit",
    learns_from="episodes",
    params=[Param("optimism", "Optimism", "number", 80.0, 0.0, 200.0,
                  help="What an untried move is assumed to be worth.")],
    image=Blueprint(python_packages=["numpy"]),         # for .remote()
)
def fit(evidence: Evidence, params: dict) -> dict:
    for tr in evidence.transitions():                   # 2. learn from the evidence
        ...                                             #    tr.state, tr.move, tr.ret, tr.propensity
    return {"policy": {state: move, ...},               # 3. return a policy
            "diagnostics": {...}}

Replace the body with yours

Anything that reads the evidence and returns {"policy": {state: move}}. Thompson sampling, a contextual bandit, a gradient-boosted value model, whatever you have. Put its packages in image=.

image=Blueprint(python_packages=["numpy", "scikit-learn", "torch"]),
gpu="L4",

Run it as a job

python app.py --remote

Same function, same result shape. It ran in a container with your image and your GPU, in your workspace, on your budget, not on your laptop. That is .remote() instead of .run(), and nothing else changed.On the dashboard (Apps → the engine → Logs) the container’s log is everything you see of the fit, and it is your fit’s stdout: print() from inside it. The SDK adds one line going in (the evidence summary and the cleaned params) and one coming out (time taken, policy size, diagnostics), so a fit that prints nothing still leaves a readable trace. Anything you print in __main__ stays on your laptop.

Measure it: no humans needed

A table is only as good as the calls it produces. Play it against the world’s simulator:

python app.py --simulate acme

playing optimistic-greedy against acme for 60 simulated calls …
  yours    mean return   359.7   booked 98%   violations 0   unknown states 0 (0% of turns)
  uniform  mean return   355.8   booked 92%   violations 0
  lift     +3.9 return per call over chance, 60 calls, judge: heuristic, replies by v2 · trained

Your engine chose every move, the pack’s personas answered, the judge scored each reply, and a uniform policy ran the same seeds beside it. lift is against chance on the same customers. This is the whole RL loop with the judge as the only critic, no RLHF, nobody picking between replies, which is how you iterate on an engine in minutes. Nothing is stored; a measurement is not more evidence.

What crosses over when it runs as a job

Only plain data crosses, and that is deliberate: the SDK inside the container is the one baked into the worker image, not the one on your machine, so an Evidence object sent from here might be a class that does not exist there: a dict of strings and numbers unpickles against any version. The container builds its own Evidence from the document and runs the same checks, which is why .run() and .remote() cannot drift apart.

Without writing a driver: `levers9 engine`

Everything above is also four commands, which is how to try it on your own machine before writing anything around it. They read and write plain JSON files you can open, so a fit can run on evidence you pulled an hour ago, cut down by hand, or built from your own CRM export.

levers9 engine worlds                                  # what worlds exist, and how much each has to learn from
levers9 engine evidence acme -o acme.json                # that world's calls, saved here
levers9 engine ls app.py                               # what engines your file declares, without running them
levers9 engine fit app.py --evidence acme.json          # fit here, or --remote for the image and GPU it declared
levers9 engine test acme --policy optimistic-greedy.policy.json --tell 2

fit prints what the engine decided and writes the policy to a file; test plays that policy against the world’s simulated customers and prints the verdict. --param k=v sets a declared parameter (coerced and clamped as the declaration says), and --engine NAME picks one when a file declares several. Nothing here needs the Arena to be reachable except worlds, evidence and test: a fit on a local file is entirely local, so an engine can be developed on a plane.

Reading an engine before you trust it

Every engine on the Engines page carries three things beyond its description, because a description is not enough to decide whether an algorithm suits your data:

its equation, set as mathematics rather than described in words. It is the rule that engine’s code actually applies, read from the function itself, so where an implementation differs from the textbook (a clipped ratio, an exact solve instead of sampling, a tolerance as the stopping rule) the equation says so.
a sentence naming the symbols, in the same plain language as the rest of the page.
the Python that runs, from GET /api/rl/engines/{name}/source, with its file and line, so you can find it in the repository. A registered engine has no source here: its code lives in the workspace that deployed it, and the page says that rather than showing nothing.

Registering an engine from the browser

POST /api/rl/engines takes a declaration and nothing else: name, family, what it learns from, the knobs, and which deployment it belongs to. On the Engines page there is a paste box for your @decision_engine(...) block, read with Python’s ast on the server, never eval, which fills the form in for you. Registering makes the Arena able to list and describe your engine. It does not make the Arena able to run it, and that is deliberate: the dashboard holds every workspace’s token, so it executes nothing it was handed. Running stays where it already works:

levers9 engine fit your_engine.py --evidence acme.json --remote

DELETE /api/rl/engines/{name} removes a registration. A built-in cannot be removed.

The evidence

A plain JSON document. Exported from the Arena, saved to a file, put on a volume, or hand-made: your engine sees the same thing every time. The Engines step of a world has the downloads at the top, under This world’s data: the evidence (JSON), and the world’s real calls themselves, as JSONL (one object per call, every turn with the judge’s score) or CSV (one row per turn), phones masked as on arrival, mock calls left out. Copy to the datasets volume writes a dated pair under datasets/rl/<tenant>/, where Storage lists them and levers9 cp levers9://datasets/rl/acme/calls-<date>.jsonl . fetches them. From a script:

curl -H "Authorization: Bearer $TOKEN" "$API/api/rl/calls/export?pack=saudi_real_estate&tenant=acme&format=jsonl" > acme.jsonl
curl -H "Authorization: Bearer $TOKEN" "$API/api/rl/calls/export?pack=saudi_rent_split&format=csv" > rent-split.csv

Field	What
`pack`, `mdp`, `moves`, `states`	The world. `states` are the keys actually observed, not the full cross product, so tables built from them are the size of the evidence.
`transitions[]`	One row per agent turn: `state`, `move`, `reward`, `penalty`, `next_state`, `terminal`, `done`, `ret`, `propensity`
`pairs[]`	`[state_idx, better_move_idx, worse_move_idx]`: preferences by judged return, for engines that learn from those
`simulator`	Whether a live simulator exists for this pack, which is what an engine that explores needs

Two fields do most of the work: ret is what the rest of the call was worth from that turn: G = Σ γᵗ·rₜ + γᴴ·terminal − c·violations. It is the number a value method predicts and a preference method compares. propensity is the probability the policy gave the move it took. It is what every off-policy estimator needs and what most logs forget, and it cannot be reconstructed afterwards: nobody knows later what the distribution was at the time. The sample has it on every turn (the moves were chosen uniformly, so it is 1/8). A real pack has it only for turns a move policy drove; when the LLM chose the move itself there is no distribution to report, and the field is null. Treat null as unknown, never as 1.0.

A real pack has turns nobody labelled. Calls ingested from a platform carry no move on the agent’s turn, and those rows arrive as move: "unknown", a ninth value that is not in moves. transitions() leaves them out by default so an engine indexing by move never meets an arm it was not told about; summary()["unlabelled"] says how many there were, and transitions(include_unlabelled=True) returns them. On the acme world that is 82 of 2,174 rows.

The sample’s moves were chosen uniformly at random, on purpose. A sample produced by a good policy would contain almost no comparison, the same move from the same state, every time, and you cannot learn from a decision you never varied. Random choice is what makes every move tried from every state reached.

Regenerate it larger from the dashboard repo: python -m app.decision.sample --episodes 1000 --out evidence.sample.json.

Simulation without RLHF

The Arena has two loops and they are separate, which is what lets an engine be fitted and measured with no human in it at all. They share exactly one step, the language model writing the reply, and that is where the picture is worth more than the sentence. Solid is what runs today; the dashed box is designed and being built. Read the one join carefully: the engine decides what to do and the language model decides how to say it, and the rendering step is all they share: no arrow runs from one loop into the other. Nothing in a stored human preference records which move produced the winning reply, so joining them would mean inventing that mapping rather than measuring it. “The version the Arena plays” is also narrower than it sounds today: a promoted move policy drives the moves of the Arena’s simulated episodes, while the production reply endpoint still consults none, which is exactly what the today arm below measures.

from levers9 import Evidence, simulate

result = fit.run(Evidence.fetch("acme"))
report = simulate("acme", result, episodes=200, seed=1, name="thompson")
report["lift"]                      # mean return over a uniform policy on the same seeds
report["yours"]["unknown_states"]   # turns your table had no entry for (a uniform move was used)

Field	What
`yours`, `today`, `uniform`	`mean_return`, `booked_rate`, `violations`, `min_return`, `max_return` for your table, for the live version choosing its own moves, and for chance: all on the same seeds
`lift_over_today`	the number a business asks for: is the engine better than what customers get now?
`lift`	against chance. Only a floor: an engine that cannot beat it has learned nothing.
`yours.unknown_states`, `unknown_share`	turns where your table had no entry. A policy asked about a state it never saw is a different thing from a policy that answered badly.
`judge`, `rendered_by`	`heuristic` or `model`; which version’s prompt rendered the replies
`by_objective`	per objective: how often your table chose it and the mean return from those turns, beside the same for chance
`calls` (with `narrate=N`)	the first N calls told turn by turn: what the customer said, the state the engine saw, what it chose and whether that came from its table or from chance, what the agent said, the judge’s score, how trust moved, the outcome

To read a run rather than a number:

python lead_to_order.py --simulate acme --tell 2

WHAT IT DECIDED, in the situations it saw most
  at the opening, lukewarm, has to check with the family     → greet them warmly       seen  72× · expected 260
  mid-call, lukewarm, has to check with the family           → propose a viewing       seen  65× · expected 162

HOW SURE IT IS
  97 of 376 situation × objective combinations have never been tried: the engine is
  guessing there, and says so. The next round of calls should try them.

THE TEST: 40 simulated customers, played twice on the same dice
             value per call  viewings booked  rule breaks
  engine                362             95%            0
  random                349             88%            0

  The engine is worth about +13 points per call more than random choice: better than chance.

WHAT IT DOES DIFFERENTLY FROM RANDOM
  Greet them warmly       more often (35% of turns vs 13%) · worth 337 when it does, 233 when random does
  Go for the close        less often (2% of turns vs 16%) · worth 136 when it does, 237 when random does

CALL 1 · أبو محمد · opens lukewarm (has to check with the family)
  Customer: “Peace be upon you, I saw the villa listing in Al Yasmin and want to ask about it”
  Situation: at the opening, lukewarm, has to check with the family.
  Engine:   greet them warmly (learned)
  Agent:    “And peace and mercy be upon you, welcome Abu Mohammed…”
  Judge:    76/100 · trust rose (0.50 → 0.66)
  …
  Outcome: a viewing was booked. This call was worth 338.

Two functions produce that: explain(evidence, result) for the fit and tell(report) for the test, both in levers9.decision. A state is 1:4:budget to an engine and “early in the call, lukewarm, worried about the price” to a person; an objective is handle_objection or “address their concern”; Arabic replies come with their English gloss. The words belong to the world, not to the SDK. A real-estate customer is worried about the price; a collections customer says they cannot pay right now, and propose_viewing there means proposing a payment plan. So each pack authors its own words block, stages, trust bands, moves, objections, and the Arena ships it with the evidence and with the report. A world without one reads its ids, tidied (handle_objection → “handle objection”), which is still followable: the ids are the contract, the words are only the words. levers9.decision.Vocabulary is what reads them, and Evidence.vocabulary is the one for the world you fetched. Nothing is summarised away, the numbers are the same ones, but the page can be read by someone who has never seen the state encoding. It is the page to look at when a lift surprises you. In the example above the engine greets on 35% of turns against 13% by chance, including a turn where the customer is asking about price, and the built-in judge still scores it well. That is a finding about the judge as much as about the engine, and it is visible only here. Behind it is POST /api/rl/environments/{id}/simulate with {policy, episodes, seed, name, narrate}, capped at 500 episodes per arm. today is the live version with no move policy at all: it writes a reply and the move is whatever that reply turns out to be, so it is the honest picture of what customers get now. A move that is not one of the Arena’s eight is refused before anything runs. In mock mode the judge is cue-based, so lifts are small and honest; with a model judge the same call measures the same thing.

A worked engine for the lead-to-order pilot

demo/decision_engine/lead_to_order.py is the engine for the decision the strategy memo picks first, which conversational objective next, measured on viewing booked. It is a contextual bandit (Thompson sampling over state × objective on the judged return) with the three things the memo asks for as data: an exploration floor in the collection policy it hands back (so exploration never decays to zero, and every move carries its propensity for the next fit), a per-decision off-policy estimate of the new policy on the logged turns that carry propensity (or null with a reason), and min_visits so a single call is never generalised from. On the sample the estimate says 178 vs 148 logged; on the real world by simulation the lift is +13. Its README has a parameter sweep that shows what each knob does and why the defaults are the neutral choice.

The container imports your whole file, not just fit, because that is how it finds it. Anything at the top of the file is imported again in there, so a module-level import matplotlib, or anything else only your driver needs: fails the job with ImportError before fit runs. Keep the top of the file to what fit needs and what is in your image=; import the rest inside if __name__ == "__main__":. The lead_to_order.py example does exactly that with its reporting helpers.

`Evidence`, in your `fit`

evidence.transitions()      # list[Transition] : .state .move .reward .penalty .r .ret .propensity .next_state
                            #   only rows whose move is one of `moves`; transitions(include_unlabelled=True) for all
evidence.by_episode()       # list[list[Transition]], ordered: for SARSA and anything else that needs "the move after"
evidence.pairs()            # list[(state_idx, better, worse)]
evidence.states             # list[str]      evidence.state_index   # str → int
evidence.moves              # list[str]      evidence.move_index    # str → int
evidence.mdp, evidence.gamma
evidence.has_propensity()   # every row carries one
evidence.summary()          # what you saw printed above

What you can return

{
  "policy": {state: move, ...},          # required. one move per state.
  "value": {state: float, ...},          # optional. shown beside the policy.
  "diagnostics": {...},                  # optional. anything you want displayed. shown, not trusted.
  "artifact": {...},                     # optional. your model, if you want it kept with the run.
}

The result is checked before it is accepted. A move that is not one of the Arena’s eight is refused: that is a bug. A state the evidence never reached is allowed: an engine that plans over the whole space, or ran against the simulator, legitimately knows more states than the log does.

`learns_from`

Which kind of evidence your engine needs. It decides whether the engine can run on a pack at all, and the wrong kind is refused with a sentence rather than an index error.

Value	Your engine reads	Costs	Built-in examples
`episodes`	`transitions()`: stored calls	Nothing. Offline.	Q-learning, SARSA, Monte Carlo, value iteration
`preferences`	`pairs()`: one move beat another from a state	Nothing. Offline.	DPO, Bradley-Terry reward model
`simulator`	A live environment to sample from	One real episode per simulated call. The only kind that explores.	REINFORCE, PPO, R-max

`Param`

Param(name, label, kind, default, min=None, max=None, choices=(), help="")
#            kind: "int" | "number" | "choice"

Declared as data rather than as a function signature, on purpose: it is what lets the Arena list an engine it has never seen and draw a form for it. A submitted value is coerced and clamped to the range rather than refused: someone who typed 999 into a 0–100 box meant “as far as this goes.” Write help for the person about to turn the knob: what changes, not what the symbol is called in the paper.

Running one of the built-in engines from the Arena

Before writing your own it is worth seeing what fourteen standard algorithms make of the same world: that is the number yours has to beat. The Arena runs its own over its API, and one run is the three steps this page has been doing by hand:

# what can run on this world, and why not
curl -H "Authorization: Bearer $TOKEN" "$API/api/rl/engines?env=acme"
# assemble the evidence, fit, and measure what it produced
curl -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  "$API/api/rl/engines/q-learning/run" \
  -d '{"env": "acme", "params": {"passes": 20}, "episodes": 100, "seed": 3, "narrate": 2}'

The measurement is the same /simulate code, not a second copy of the arithmetic, so a run’s lift_over_today is the number you would get by posting the table yourself. It comes back as a job (GET /api/rl/jobs/{id}) whose progress says what it is doing: “fitting Q-learning”, “playing 100 simulated customers”, because a fit over every stored call and three hundred simulated calls are two slow things, and a bar that reads “running” for a minute is indistinguishable from a hang. GET /api/rl/engines?env=<world> is also the honest answer to which of these can I try here. Each engine carries learns_from and, for that world, can_run with a why_not sentence: a world with no logged calls cannot run Q-learning, a world where no state ever saw two different moves has no preference pairs, and a culture pack with no personas has no simulator to explore. Naming which of the three is missing turns a disabled button into something to act on. What a run produces is kept. GET /api/rl/engines/runs?env=acme lists them newest first with what each was worth; GET /api/rl/engines/runs/{id} returns the table, the value it put on each state, the engine’s own diagnostics, and the whole report including the narrated calls. The artifact is deliberately not stored: a Q table or a matrix of weights is megabytes of numbers nothing reads back, and a run that has to be reproduced exactly is repeated from its params and seed. Only the newest fifty runs of a world are kept. POST /api/rl/engines/runs/{id}/promote is the last step: the table becomes a new version of the pack, which is the live version’s prompt plus that table as the engine choosing each turn’s move. What changes is what the agent does, not a word of how it says it. Promoting the same run twice is refused: one table is one step forward, and a second copy of it would make the pack’s history a claim about work that never happened.

This is how your engine will be run too: the same declaration, the same measurement, the same promotion. It is why a lift you report from your terminal and a lift the Arena reports are comparable numbers.

On real calls, and against every other engine

The simulator is one test. The other is the calls your agent actually made. An evidence set is built from a world’s real calls (the calls it holds, or the full export on the call-logs volume): phone numbers are removed, a tagger reads each agent turn’s move and the customer’s state before it, each call is priced by its platform outcome, and the calls are split 80% to fit and 20% held out. The engine benchmark then fits every engine on the fit calls and scores each on the held-out calls with off-policy evaluation, beside today’s agent and a random choice of move. Your engine joins it with one flag:

levers9 engine evidence-build acme --source volume --tagger judge -n 20000
levers9 engine benchmark --evidence SET_ID --file app.py            # yours, fitted on the fit calls only

In Python, Evidence.from_set(SET_ID).part("fit") is the side to fit on, and levers9.decision.benchmark(SET_ID, tables={"mine": result}) scores the table. The scores rank candidates; they do not prove a lift. How the sets are built, what the three estimators mean and what they cannot tell you is on Benchmark engines on real calls. The contract above is the stable part: an engine written against it is fitted on simulated episodes, on a real evidence set, or on a file you made yourself, without a line changing.

Get started

Build with the SDK

Dashboard

Data Lab

RL Arena

Operate

Bring your own engine

In ten minutes

What crosses over when it runs as a job

Without writing a driver: `levers9 engine`

Reading an engine before you trust it

Registering an engine from the browser

The evidence

Simulation without RLHF

A worked engine for the lead-to-order pilot

`Evidence`, in your `fit`

What you can return

`learns_from`

`Param`

Running one of the built-in engines from the Arena

On real calls, and against every other engine

​In ten minutes

​What crosses over when it runs as a job

​Without writing a driver: levers9 engine

​Reading an engine before you trust it

​Registering an engine from the browser

​The evidence

​Simulation without RLHF

​A worked engine for the lead-to-order pilot

​Evidence, in your fit

​What you can return

​learns_from

​Param

​Running one of the built-in engines from the Arena

​On real calls, and against every other engine

In ten minutes

What crosses over when it runs as a job

Without writing a driver: `levers9 engine`

Reading an engine before you trust it

Registering an engine from the browser

The evidence

Simulation without RLHF

A worked engine for the lead-to-order pilot

`Evidence`, in your `fit`

What you can return

`learns_from`

`Param`

Running one of the built-in engines from the Arena

On real calls, and against every other engine