> ## Documentation Index
> Fetch the complete documentation index at: https://levers9.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Identity and credentials

> Who can sign in, what a token is, what holds one, and how access is taken away

Sign-in and credentials are two different things. [Sign-in](https://levers9.mintlify.site/guides/sign-in.md) is how a person proves who they
are; a credential is what a machine carries afterwards. This page is the second half: what exists, what holds
it, and how it is taken away.

## The shape of it

```mermaid theme={null}
flowchart TB
  P[Person] -->|Google OIDC| API[Dashboard API]
  API -->|session cookie, signed| B[Browser]
  API -->|token| C[CLI config]
  C --> CP[Controlplane]
  B --> API
  API --> CP
  CP --> W[Worker container]
  W -->|runtime token, one run| CP
  I[Integration] -->|bearer token| E[Deployed endpoint]
  E --> CP
```

Everything reaching the controlplane carries a token. The browser is the exception: it carries a session
cookie to the dashboard API, and the API uses the workspace's token on its behalf.

## Signing in

Google is the only identity provider. There are no local passwords and no password reset, because there is no
password: the dashboard redirects to Google, verifies the returned id token, and creates a session.

* `LEVERS9_AUTH_ALLOWED_DOMAINS` decides who may sign in at all. An email outside those domains lands on a
  waitlist that a platform admin approves from **Members**, rather than being rejected.
* `LEVERS9_AUTH_WORKSPACE_MODE` decides what shape of workspace they get — one shared, one of their own, or
  both. See [What a person gets](#what-a-person-gets).
* The session is a signed cookie, `LEVERS9_AUTH_SESSION_DAYS` long. It is signed with `LEVERS9_AUTH_SECRET`,
  so it cannot be edited: changing one byte invalidates the whole session rather than altering it.
* The cookie names a session the dashboard keeps a row for. Logging out ends that session for good, so a copy
  of the cookie stops working too, and **sign out everywhere** (`POST /auth/logout-everywhere`) ends every
  session the person has. Cookies from before sessions had ids keep working until they expire, unless the
  person signs out everywhere or `LEVERS9_AUTH_SESSION_ID_REQUIRED` is on.

Deleting a person's user row ends their sessions, because every request resolves the session against the
database. Suspending them in Google Workspace doesn't, on its own: the cookie stays valid until it expires.
Do both, or end their sessions too. How the cookies are scoped, and the check on where a change comes from, are
in [Sessions and cookies](https://levers9.mintlify.site/guides/sign-in.md#sessions-and-cookies).

## What a person gets

A workspace is a controlplane workspace of its own — its own token, its own volumes, its own apps, its own
secrets. Nothing is a filter over shared storage, so the boundary between two workspaces is real, and
membership is the whole of access. `LEVERS9_AUTH_WORKSPACE_MODE` picks how those workspaces are handed out:

| Mode | What exists | Where a sign-in lands |
| - | - | - |
| `domain` | One workspace per email domain | That workspace |
| `user` | One per person, nothing shared | Their own |
| `both` | An organisation workspace **and** a personal one per person | The organisation's |

`both` is the two-level model: colleagues have somewhere to work together, and each person also has somewhere
nobody else is a member of, reached from the workspace switcher. A sign-in lands in the organisation's, because
work meant to be shared should be shared by default rather than by remembering to move it.

## Joining a workspace

Being allowed to sign in and being admitted to a team are different questions. Allowed domains decide who may
reach the door; a workspace's **join policy** decides who may come through it.

| Policy | Who gets in |
| - | - |
| **Invitation** (default) | The person who founded the organisation, anyone an admin invites by email, and anyone whose waiting request an admin approves |
| **Open to the domain** | Anyone who can sign in with an email at the organisation's domain, with nothing to approve |

An admin sets this in **Members**, with *Anyone with an @domain address can join*. It is a property of the
workspace rather than an environment variable, because who belongs in a team is something only that team knows.

An invitation writes the membership before the first sign-in, which is what makes an invited colleague land
straight in their team rather than waiting. Someone turned away isn't rejected: their request is recorded with
the workspace it's waiting on, that workspace's admins see it under **Members → Waiting to join**, and the
person's in the next time they sign in. Admins of other workspaces don't see it. A request from outside the
allowed domains waits on no workspace, so only a platform admin sees and decides it. Turning the policy back to
invitation-only decides who arrives next. It never removes anyone already working there.

The first person into a new organisation founds it and is its admin. Somebody has to be able to admit the
second person, and an organisation with no admin could never admit anyone.

> [!NOTE]
>
> Workspaces that existed before join policies arrived keep the open door they had, written explicitly when the
> column was added. Applying the new default to an organisation already in use would have started turning
> colleagues away the moment it deployed, with nobody told why. Check yours in **Members** and tick the box off
> if you want invitations; existing members are unaffected either way.

## What holds a credential

| Holder | What it carries | Lives in |
| - | - | - |
| A browser | Session cookie | The browser, `LEVERS9_AUTH_SESSION_DAYS` |
| The CLI and Python client | Workspace token | `~/.levers9/config` |
| An integration calling an endpoint | The same token, as a bearer | Wherever the integration keeps secrets |
| A worker container on a shared pool | The workspace's primary token, as `LEVERS9_TOKEN` | The container, for its run |
| A worker container on a private (agent) pool | A restricted runtime token | The container, for one run |
| The dashboard API | Every workspace's token, to act on their behalf | `~/.levers9/dashboard/auth.db` on its volume |

> [!WARNING]
>
> Unless `LEVERS9_PER_USER_TOKENS` is on, the CLI receives the **workspace's own token**, the same for every member,
> and the **Access** page reveals it to any member who asks. It does not expire and cannot be traced to a
> person, so revoking one holder means rotating for everyone. Treat it as a shared password until per-person
> tokens land — see [Where this is going](#where-this-is-going).

## Tokens, by kind

The controlplane distinguishes several token types. Three matter day to day:

* **Workspace** — what the CLI, the Python client and endpoint callers carry. It acts as the whole workspace.
* **Restricted workspace** — the same, with a narrower set of permissions. It uses secrets through the
  containers it runs and isn't meant to read a secret's value back out: the v1 API refuses that, and the older
  one does too once `restrictedSecretRead` (below) is set to `enforce`.
* **Runtime** — minted for one container, for one run, and worthless afterwards.

```bash theme={null}
levers9 token list                 # every token this workspace has, and whether it is active (a key_hint, never the key)
levers9 token create               # mint another one
levers9 token toggle ID            # disable it without deleting it
levers9 token delete ID            # revoke it permanently
```

You see a token's value once, when you create it. After that, lists and toggles only show a hint of it (its
first six and last four characters), which is enough to tell tokens apart and not enough to use one. Send a
token in the `Authorization: Bearer` header. The controlplane still takes one in a `?auth_token=` query
parameter for now, but it logs every such call, and it'll stop taking them once nothing sends one, since a URL
ends up in proxy and load balancer logs.

The other token types aren't carried by people. Each one is for one job, and the controlplane checks that it's
only used for that job:

| Type | Held by | What it can do |
| - | - | - |
| Worker | Every worker pod | The worker's own calls: its queue, its containers' state, checkpoints, its tunnel |
| Machine | The agent on a machine a provider pool started | Register that machine and fetch its configuration |
| Cluster | The agent of a cluster you attached | Hold its connection to the controlplane open |

None of them can list tokens, read secrets, deploy or manage anything else. The same rule runs the other way:
the services the workers use only take a worker token. Only an operator can create a machine token.

Operators switch each of these checks from logging to refusing in the controlplane config, under
`controlplane.auth`: `tokenScope` for the token types above, `restrictedSecretRead` for restricted tokens
reading secret values, and `queryToken` for tokens in the URL. Each one starts as `observe`, which logs what it
would refuse (the token's id and type, never its value) and lets the call through. Set it to `enforce` once
that log has been quiet for a while.

The primitives for issuing and revoking are already there; what the dashboard does not yet do is tie a token
to a person, which is why revocation today is all-or-nothing. The **Access** page shows the current token
with a reveal and a copy button, and the exact `levers9 config create` and `curl` lines to use it. The page loads
without the token and asks the API for it only when you reveal or copy it.

### Who holds what

The controlplane issues and revokes tokens but does not model an owner — `CreateToken` takes only a type. So
the record of who holds which credential is the dashboard's, keyed by the token's external id, the one stable
non-secret handle both sides agree on and the same handle revocation takes.

The unit is the **person**, not the workspace. A controlplane token is workspace-scoped, so someone in two
workspaces holds two credentials — but they are held by them, listed under their name, and taken away
together. That is what makes removing a leaver one action rather than a hunt.

## What a workspace keeps to itself

A workspace is a controlplane workspace: its own token, its own volumes, its own apps, its own secrets. That
boundary is enforced at the controlplane, which refuses a workspace id that does not match the token on the
request. Nothing there is a filter over shared storage.

The dashboard is a second layer, and it works differently. Its own registries — datasets, notebooks, RL packs
— are files it owns rather than controlplane objects, so the dashboard can see all of them and the question
is whether each route asks who is calling. Every route that takes a record's id now does, through one shared
check, and the answers are deliberate:

* **Refusals are 404, not 403.** A 403 confirms the id exists and belongs to somebody. Ids are guessable, so
  that would turn the check into a way of enumerating what other teams have.
* **A request with no session is refused, never resolved.** The dashboard holds a cluster-admin credential and
  falls back to it when no workspace token is set, so "which workspace is this request?" had an answer that
  was wrong in the most dangerous direction.
* **Listings do not carry credentials.** A dataset record holds every share link's token and the webhook key
  that lets anyone post rows into it. Those live on the detail routes, which check who is asking; the list
  carries a count instead.
* **Records written before this existed are resolved, not assumed.** A dataset is claimed by whoever can list
  its files with their own token, which only its real owner can do.

> [!WARNING]
>
> The RL Arena's registry has no workspace dimension at all rather than an unfilled one, so it is not yet
> scoped: any signed-in person can read any pack's calls, and the webhook key that authenticates posts into a
> pack. It is one customer's data today. Scoping it changes the shape of the file rather than adding a check at
> a route, which is why it is not done in the same pass.

Share links are the deliberate exception, and worth understanding. A share link carries no session, so the
dashboard binds **the dataset owner's** token for that request — otherwise every file would 404. That is what
makes a link work for an outside reviewer, and it is why creating one is an owner-only action.

## Platform administrators

`LEVERS9_ADMIN_EMAILS` names the people who administer the deployment rather than a single workspace. It is
configuration, not a database row or a membership flag, and that is deliberate: an entitlement that cannot be
granted from inside the running application has no escalation path through a stolen session or a write to the
dashboard's database, and it survives that database being recreated.

A workspace `admin` role is a different, smaller thing. It administers one team (inviting members, approving
the requests waiting on that workspace) and reaches nothing outside it.

Platform admins get `/admin`: every workspace, what is running in each, who has signed in, and the audit
trail. From **People → View as** they can act as another person, which is how you support someone whose
workspace you are not a member of.

> [!NOTE]
>
> Impersonation is deliberately loud. An amber banner naming the person is drawn on every page while it lasts,
> the session records who started it inside the signed cookie, and entering, leaving and stopping anyone's work
> are each written to an audit table before the action happens, so a failed attempt still leaves a trace. The
> audit tab shows the trail and nothing in the application can edit it. It also ends by itself: after
> `LEVERS9_AUTH_IMPERSONATION_MINUTES` (two hours by default) the admin's back in their own workspace.

## Taking access away

| Situation | Do this | Effect |
| - | - | - |
| Someone leaves | Suspend in Google Workspace **and** delete their user row | No new sign-in; existing sessions end at the next request |
| A laptop is lost or a cookie may have leaked | The person signs out everywhere (`POST /auth/logout-everywhere`) | Every session of theirs ends at the next request, in every browser |
| A token is believed leaked | Rotate the workspace token | Every holder must reconfigure — see the warning above |
| An admin should no longer be one | Remove them from `LEVERS9_ADMIN_EMAILS` and redeploy | Takes effect when the pod restarts |
| A workspace should be closed | Delete its workspace row | Its token stops resolving |

## Where this is going

The shared workspace token is the known gap, and it is being replaced in phases that keep every existing
config working:

1. ~~**A register** of every credential with its last use~~ — **done**. `/admin` → **Tokens** lists every
   credential on the deployment with its type, last use and the client that used it. Values are masked on
   the server; the full key is never sent to the browser.
2. ~~**A token per person** at `levers9 login`~~ — **done, and on in the hosted deployment.**
   `LEVERS9_PER_USER_TOKENS=true` gives each person their own credential instead of the workspace's, in every
   workspace they belong to, listed under their name in the register. The workspace token keeps working, so
   turning it on is additive and turning it off is a rollback. Tokens issued before it was on are still shared
   and show in the register with no owner, which is how you find them.
3. **Revocation**: ~~sessions that can be ended~~ **done**: logout ends a session for good and sign out
   everywhere ends all of a person's. Still to come: disabling a person from `/admin` ends every session of
   theirs at once, and sessions shorten to seven days.
4. **Service tokens** for systems — named, owned, no expiry by design, revocable — and pipeline runs stop
   carrying the requester's personal credential.
5. **Expiry** on user tokens only, ninety days, once the rest is proven.
6. **Retirement** of the shared path: the Access page shows *your* token, with reissue, and admins keep a
   break-glass rotation of the workspace primary.

The order matters. Expiry is a backstop for a credential that leaks unnoticed; attribution and revocation are
the actual control, which is why they come first.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.