Skip to main content
Your function does not run on your machine. It runs in a container on a worker, so something has to decide what is installed there: which Python, which wheels, which system tools. That something is a Blueprint, and what it builds is the environment your code wakes up in.

​
You may not need to write one

A function that names no blueprint gets one worked out from the project. levers9 reads the dependency manifest the project already has, requirements.txt or pyproject.toml, and if there is none it reads the imports in the code that is about to be synced and pins them to the versions installed on your machine, so the remote run matches the one you have been developing against. Your own modules and the standard library are never proposed for installation.
What it found is printed before the build, so an inferred environment is never a mystery:
Inference is a starting point, not a policy. The moment you pass any argument to Blueprint, even an empty Blueprint(), your answer is used exactly as written and nothing is inferred. LEVERS9_INFER=0 turns it off everywhere. For anything long-lived, write the blueprint down: a manifest that drifts gives you a different environment next month, and the printed list is there to be pasted.

​
The whole path, once

Three things in that picture surprise people:
  1. The build runs on a worker, not on the controlplane. The same kind of machine that will run your code assembles the blueprint with buildah, and its output is streamed back to the process that asked for the build, which is where a failure reads as Error: building at STEP "RUN …".
  2. The result is not pushed to a container registry. It is stored as a content-addressed clip archive in an S3 bucket. A worker needs read access to that bucket and nothing else, which is what lets a cluster you own run your blueprints without holding credentials to anyone’s registry.
  3. The container does not wait for a download. The worker mounts the archive over FUSE and starts, fetching the parts of the blueprint that are actually opened. That is why a cold start is seconds rather than the minutes a multi-gigabyte pull would take, and why the clip_version on a blueprint matters: it is the version of that archive format.
The platform’s own images (the agent, the worker) are ordinary container images pulled from a registry. Only blueprints take the S3 path.

​
Why the build is its own container, not part of your job

The build gets a container of its own, named build-<id>, with its own small CPU and memory request and its own pool. Your job’s container never runs a build. Four reasons, and they all point the same way:
  • A container cannot build the thing it is already running in. Your job’s filesystem is the blueprint, mounted from the archive at start. It has to exist before the container does.
  • One build serves everything afterwards. A blueprint is content-addressed, so the build happens once and every later run, every other function, every replica and every machine mounts that same result. A four-GPU job installs its packages zero times, not four.
  • Building needs tools and privileges your code should not have. buildah, an overlay filesystem and a writable root live in the build container and stop there.
  • The build does not need the machine your job needs. It asks for about one CPU and a gigabyte on the build pool, so a GPU node is never spent running pip. The exception is a Blueprint that declares a GPU of its own, which is the case for packages that compile against CUDA at install time; then the build takes one GPU.
The visible consequence is where the output goes. The build is a controlplane call your process makes before the task exists, so its log streams to the terminal you launched from, while the task itself sits in PENDING with nothing in its own log. A build that fails looks like a task that never started.

​
You do not build it; the first call does

There is no build step to run. The first call that needs a blueprint builds it, prints the build log where you started the process, and then runs. Later calls with the same Blueprint start in seconds, because blueprints are content-addressed: the same Python version, the same package list and the same commands produce the same blueprint id, and that id is reused across functions, runs and machines. While a build runs the task sits in PENDING. A build that fails shows up as a task that never starts, and the reason is in the output of the process that launched it, not in the task’s own log.

​
Pin versions, and install at build time

An unpinned package means the blueprint you build today and the one you build next week are different programs. Worse is installing at run time: a job that pip-installs its own dependencies picks up whatever released since the last run, inside a container that already has a working set. We lost a night of training runs to exactly that. The job installed datasets on start, pip resolved a newer transformers alongside it, and every rank died on cannot import name 'PreTrainedModel' before the first step. The fix was a pin, and the lesson is that the pin belongs in the Blueprint. If you must install something at run time, name the versions of everything the base image already depends on, so pip cannot upgrade them behind you.

​
Installs that fail, and what they need

Commands run in the order you give them, after python_packages, so anything pip needs from apt has to come from a Blueprint whose commands already installed it, not from a later line in the same list.

​
Secrets a build needs

.with_secrets(["PIP_INDEX_URL"]) hands workspace secrets to the build, for a private package index or a gated model download. Each one is set as $PIP_INDEX_URL in the build’s commands, the way it always was. If you build from your own Dockerfile, you can also read it as a file, which keeps it out of the image’s history:
The build runs with only what the builder itself needs from the machine, never the worker’s own settings or credentials.

​
Seeing what was built, and why

Every build records the spec that produced it, what it cost, and how it differs from the workspace’s previous blueprint:
The same rows are on the dashboard under Blueprints, where each one opens to its full package and command list, its build log (the build container’s output, kept with the workspace’s other logs) and Runs on it: the functions registered on that blueprint, each linking to its app, with the GPU it asks for. A blueprint is only ever built because a function asked for it, and that list is the answer to “which one?”. The link runs the other way too: a function’s card on Apps carries a bp <id> chip and its Details tab links the blueprint id straight to that row. The rows also come from the API at GET /api/v1/blueprint/{workspaceId}, with build_container_id naming the container whose log it is. The changed since column is the one worth reading: a blueprint that rebuilt for a reason you did not intend (a floating version, a command that differs only in whitespace) shows up there immediately.

​
Reusing a blueprint someone else built

A blueprint id is stable, so one that took fifteen minutes to build can be attached directly:
That skips the build entirely and pins the environment exactly, which is what you want for a long experiment series where the environment must not drift between arms. The dashboard shows the id under a function’s Details, and levers9 deploy prints it when it publishes.