Parameters

Every abstraction (job, endpoint, queue, cron, Box, Sandbox, VLLM) accepts the same resource, blueprint and storage parameters. The ones that differ are listed at the end.

Resources

parameter	default	what it does
`cpu`	`1.0` (jobs `0.125`)	cores per container; fractions allowed, or a string like `"500m"`
`memory`	`128` MiB	per container; `"8Gi"` style strings work
`gpu`	none	a type (`"T4"`, `"L4"`, `"A10G"`, `"L40S"`, `"A100-40"`, `"A100-80"`, `"H100"`, `"H200"`, `"RTXPro6000"`) or a preference list; the scheduler takes the first with capacity
`gpu_count`	`0`	GPUs per container; more than one for multi-GPU training
`pool`	scheduler’s choice	pin to a worker pool by name
`timeout`	`3600` s (endpoints `180`)	after which a task is killed and marked timed out
`retries`	`3`	retries on failure for jobs, crons and queues; `retry_for=[SomeError]` limits which exceptions retry
`task_policy`		`TaskPolicy(max_retries=, timeout=, ttl=)` as one object

Blueprint and files

parameter	what it does
`image`	a `Blueprint(python_version=, python_packages=[], commands=[])`; see Blueprint. `Blueprint.from_dockerfile(path)` and `Blueprint.from_registry("...")` also work, and `.with_docker()` enables Docker-in-Docker for sandboxes
`volumes`	a list of `Volume(name=, mount_path=)` or `CloudBucket(name=, mount_path=, config=CloudBucketConfig(...))`; see Volumes
`disks`	`DurableDisk` block storage on local machines
`secrets`	names of workspace secrets to inject as environment variables; see Secrets
`env`	extra environment variables, in plain text
`name` / `app`	the app the function belongs to in the dashboard; defaults to the script name
`description`	what the function is for, shown under its name in the dashboard; defaults to the first paragraph of the handler’s docstring (280 characters), `""` shows nothing
`show_args`	jobs show each call with the arguments it was called with, whole, in the calls table, on the call’s own panel and in `levers9 task list --format json`. A config dict shows every key. A keyword argument or a dict key named like a credential (`token`, `api_key`, `password`, …) is redacted at any depth, and so is a string that looks like one (`hf_...`, `sk-...`, a JWT). Only a payload is shortened: a string past 2,000 characters, a container past 200 items, bytes past their first twelve, and the whole line past 8,000 characters, where it says how much is missing. `False` sends nothing. Default `True`

The current directory is synced into every container. Put a .levers9ignore file next to your script to leave things out (same syntax as .gitignore). Anyone in the workspace can read the synced folder in the dashboard, so keep credentials out of it. Files that look like secrets (.env, .env.*, *.pem, *.key, id_rsa*, id_dsa*, id_ecdsa*, id_ed25519*, .aws/, .netrc, .npmrc, .pypirc) are still uploaded today, and every run, serve and deploy names them in a warning. A later release leaves them out. Put the values in a secret instead, or, if a file really has to ship, keep it with a line such as !.env in .levers9ignore. LEVERS9_SYNC_SECRET_FILES=skip leaves them out now. The dashboard’s code view lists such files but doesn’t show what’s in them.

Serving

Endpoints, queues, boxes and VLLM keep containers alive between calls.

parameter	default	what it does
`keep_warm_seconds`	endpoints `180`, queues `10`, boxes `-1` never	how long an idle container stays up; `0` stops it as soon as it is idle, `-1` never stops it. Whatever is left pays a cold start on the next call
`workers`	`1`	worker processes per container
`max_pending_tasks`	`100`	how many calls may be waiting or running before a new one is answered `429` (`exceeded max pending tasks`). An endpoint call stops counting as soon as it ends, when its caller hangs up while it waits, and when its container turns it away without starting it
`autoscaler`	`QueueDepthAutoscaler()`	`QueueDepthAutoscaler(max_containers=, tasks_per_container=)`; `LLMTokenPressureAutoscaler` for VLLM
`authorized`	`True`	require the workspace token on the URL; `False` for public endpoints
`on_start`		a function that runs once per container before the first request (load a model, open a connection); its return value is passed to the handler as `context.on_start_value`
`on_deploy`		a function that runs at deploy time (warm a cache, download weights)
`checkpoint_enabled`	`False`	snapshot a warm container so later cold starts restore instead of booting; on a GPU the container must be sized to hold a copy of the card, see Cold starts
`callback_url`		a URL the controlplane posts each task’s result to. A caller can also pass its own `callback_url` with a call; that one is checked (see `callback_hosts`)
`callback_hosts`		hosts a caller may name in a per-call `callback_url`, besides the host of `callback_url`; `"*.example.com"` covers subdomains. Any other per-call URL has to be https to a public address. This release only logs, in the container’s output, a per-call URL that would be refused; `env={"LEVERS9_CALLBACK_ENFORCE": "true"}` refuses it now, and the result then goes to `callback_url` alone
`inputs` / `outputs`		a `Schema` for validated, typed endpoints

What a request brings with it

An endpoint, box or sandbox port gets the caller’s headers and cookies as they were sent, with one exception: the dashboard’s own cookies (levers9_session, levers9_oauth and levers9_cli) are taken out on the way in, so your code never sees someone’s dashboard sign-in. Your app’s own cookies get through untouched. On a box reached through an authorized URL (/box/<name>/... or /box/id/...), the Authorization header is the caller’s levers9 token that let the request in. Don’t build on reading it: a controlplane can be set to drop it before it reaches the box (LEVERS9_BOX_STRIP_AUTHORIZATION), and that’s where it’s heading. On a public box the header is your app’s own and always gets through.

Outputs

Output saves a file produced inside a container and gives you a URL for it:

from levers9 import Output, job

@job()
def render():
    out = Output(path="chart.png")      # or Output.from_pil_image(img), Output.from_file(fh)
    out.save()
    return out.public_url(expires=3600)

Where each one is described

Jobs

.remote(), .map(), deploy as HTTP

Endpoints

serve with hot reload, keep-warm, autoscaling

Queues

.put(), retries, workers

Crons

when="0 2 * * *"

Boxes

ports, keep_warm_seconds, box.deploy()

Sandboxes

processes, files, network rules, snapshots

Get started

Build with the SDK

Dashboard

Data Lab

RL Arena

Operate

Resources

Blueprint and files

Serving

What a request brings with it

Outputs

Where each one is described

Jobs

Endpoints

Queues

Crons

Boxes

Sandboxes

​Resources

​Blueprint and files

​Serving

​What a request brings with it

​Outputs

​Where each one is described

Jobs

Endpoints

Queues

Crons

Boxes

Sandboxes

Resources

Blueprint and files

Serving

What a request brings with it

Outputs

Where each one is described