Skip to main content
Most decision models run on your own cards: the weights are loaded inside a job in your workspace and the calls never leave it. Jev is different. It is TypeSafe’s hosted model, reached over its System One API (POST https://api.typesafe.ai/v1/systemone), a third-party service hosted outside the Kingdom. A model like that is external, and it is badged external, sends data out wherever it appears.

​
The rule

A run or a comparison that includes an external model on a dataset of real calls is refused (HTTP 422) unless the tenant’s external models setting is on. Real calls are:
  • an Arena world’s calls (arena:<tenant>);
  • a call-log export on a volume (volume:call-logs/<tenant>/...), unless it is marked synthetic.
Made-up calls need no setting:
  • the scripted dataset, scripted:decide: the Decide page’s scenarios, in English and Saudi Arabic;
  • a file on a volume marked synthetic: a folder or a name part called synthetic (acme/synthetic/calls.jsonl.gz, acme_synthetic_calls.jsonl.gz).
On the page, Jev is greyed out while a real dataset is picked and the tenant has not allowed it.

​
Turning it on

Only do this when the client has agreed in writing that their calls may be processed by a service outside the Kingdom. On the Models section, open Compare models, pick one of the tenant’s real datasets and press Allow external models. The confirmation asks for a reference to the written permission (a document id, or an email’s subject and date) and a tick that the client agreed. The setting is stored per tenant with who turned it on, when, and that reference; every change is kept in its history, and Turn off reverses it.

​
What leaves, even when allowed

  • Only the transcript text, one string per turn: no phone field, no customer block, nothing else from the row.
  • In the job, right before each request, phone numbers, emails and any run of six or more digits are replaced with [phone], [email] and [number]. Names and everything else said on the call are not removed.
  • The report says that the dataset was sent to an external service, on what grounds (the permission reference, or that the calls were made up), which model answered, and how many phone numbers, emails and long numbers were scrubbed.

​
Setting it up

Jev needs one workspace secret, TYPESAFE_API_KEY (Secrets, or levers9 secret create TYPESAFE_API_KEY <key>). Only the Jev job mounts it, and only when a Jev model is in the run. Two optional dashboard settings: LEVERS9_JEV_MODEL (the model id asked for; empty uses the SDK’s default, jev-latest) and LEVERS9_JEV_PRICE_PER_M_INPUT (the price per million input tokens, for the cost column). The Algorithms benchmark never sends real calls to an external model: asking it to tag with Jev is refused.

​
Models the dashboard itself calls

Jev is called from a job. A few models are called from the dashboard API itself: an Arena version or the judge on an outside vLLM, and an outside names model in a personal-data scan. Those addresses go through the dashboard’s outbound check (see endpoints the Arena calls): an outside vLLM has to be a public https host, or a host the operator lists in LEVERS9_PROVIDER_HOSTS.