In the game this is mission 2, Find the hot spots: where the agent is weakest, so review time goes there.
How they are found
Every agent turn is a situation: what the customer just said (weighted double) plus the agent’s previous reply. Situations are turned into vectors (by meaning when the embedding credentials are configured, by character n-grams otherwise; the map says which withgrouped_by), clustered, and each cluster becomes a
scenario with a stable id, a size, a mean score, a violation count and a priority that grows with size,
weakness and violations. Regroup reclusters after new calls arrive; it runs as a background job with a
progress bar. Every job running on a world (a regroup, a rescore, a seed, a training run) shows at the top of
that world’s page whoever started it, including one started from the API or another tab. Grouping costs more than linearly in situations, so past six thousand of them the board is built
from a fixed sample of the calls and the header says grouped from N of M calls; every call is still scored
and still in Calls and the review queue. On real Arabic calls meaning-based grouping mattered: the same 5,867
situations made 1,305 scenarios by spelling and 349 by meaning, and the 398 spelling-scenarios that were all
the scripted opening became one.
The board and the map
The board lists scenarios by priority with their size, mean score and an auto-generated name you can rename. A name is the customer’s line, or, when that is a word or two (half of real customer turns are), the line and the reply it drew:صحيح → تمام، عندي ثلاث أسئلة سريعة…. With English on, names are translated by the
configured judge and cached; with no model there is no English name and the Arabic shows, rather than a
placeholder that made every row read “opening”.
The map draws them: halos sized by how many situations they hold, points inside for the situations
themselves, the top scenarios labelled.
- pan and zoom with the mouse; wheel zooms about the cursor;
- 2D / 3D: the third axis is another component of the same embedding; drag to rotate, it drifts when idle;
- hover a point for the customer’s words and the score; click for Review this moment;
- Select (or Shift-drag) draws a rectangle: the panel shows the count, mean score and violations of what is inside, with Review these (a Compare queue, weakest first) and Name as a scenario (a manual scenario that survives regrouping).