> ## Documentation Index > Fetch the complete documentation index at: https://levers9.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Sound datasets > Label short clips by ear: annotators listen to each clip and pick one answer from a list you write Some questions about audio can only be answered by listening. Is this sound an interruption, or someone saying "mm-hm"? Is this clip speech or noise? Whose voice is it? The `sound` dataset kind (shown as **Sound classification**) is for those. A row is one clip. An annotator plays it and picks one answer from a list you write, with a single key. Nothing drafts a sound dataset, and nothing a model said about a clip is shown to the annotator: the labels are meant to be made by ear alone. The one thing shown besides the clip is a row's text, when you give it one, such as a transcript of the clip. ## What you write: the question and its answers A sound dataset has one question, asked about every clip, and two to nine answers to choose from. ```json theme={null} { "question": "What is this sound?", "options": [ {"label": "Backchannel", "hint": "The caller shows they are listening and does not want to speak."}, {"label": "Interruption", "hint": "The caller cuts in to take the turn."}, {"label": "Noise", "hint": "Not speech: a cough, a bang, line noise."} ], "listen_s": 1, "tracks": {"left": "agent", "right": "caller", "mark": "right"} } ``` | field | what it is | | - | - | | `question` | what the annotator is asked about each clip | | `options` | the answers, in the order of their keys: the first is `1`, the second `2`. Each has a `label` and, optionally, a `hint` shown under it. A `key` is made from the label (`another_voice` from "Another voice") unless you give one; the key is what is saved | | `listen_s` | how many seconds of the sound have to be played before an answer is taken. `1` when left out, `0` to take an answer at once | | `tracks` | optional, for stereo clips: what the `left` and `right` channels are, and `mark`, which of the two the marked sound is on (`left` or `right`). Each channel is drawn as a track under its name and can be silenced while listening. The marked one is drawn on top with the box on it; without `mark` the box is on both | ## What you bring: clips, and optionally a manifest Put the clips in a folder on the `datasets` volume. A clip is kept as it is: its channels and its sample rate are what the annotator hears. A clip is usually longer than the sound it is about, so that the annotator hears what came before. A **manifest** in the same folder says where in each clip the sound in question is, and the page marks that part of the player. It is a `.jsonl`, `.tsv` or `.csv` file with a row per clip: | column | what it is | | - | - | | `audio` | the clip's file, relative to the manifest. Required | | `id` | the row's id. The file's name without its extension when left out | | `call_id` | what the clip was cut from | | `mark_start_s` | where the sound in question starts in the clip, in seconds | | `mark_end_s` | where it ends | | `split` | `train`, `val` or `test`. Worked out from `call_id` when left out | | `text` | optional: what the annotator reads under the clip, such as its transcript. Line breaks are kept, and up to 20,000 characters | ```json theme={null} {"audio": "clips/s00001.wav", "id": "s00001", "mark_start_s": 1.5, "mark_end_s": 1.93, "split": "train"} {"audio": "clips/s00002.wav", "id": "s00002", "mark_start_s": 1.5, "mark_end_s": 2.84, "split": "test"} ``` **Rows keep the manifest's order, and that is the order annotators meet the clips in.** Shuffle the manifest if the clips should not arrive grouped by call or by kind. Without a manifest every audio file in the folder is a row, in name order, with no mark. > [!NOTE] > > Keep what a model or a rule said about each clip out of the manifest, the ids and the text. A row's columns are > sent to the annotator's page. Keep that table elsewhere and join it to the answers by row id afterwards. ## Build it In the dashboard: **Datasets → New dataset → Sound classification**, pick the folder, then write the question and the answers, one per line as `Label | hint`. From a terminal, with the task in a file: ```bash theme={null} levers9 dataset upload ./clips --to interruptions levers9 dataset create interruptions --kind sound --path uploads/interruptions --task task.json ``` ## The annotation screen The clip is drawn as its tracks, one for each channel: how loud each side is from moment to moment, with a white box on the sound in question and a ruler that counts seconds from where that sound starts. On a call recording that is the caller on one track and the agent on the other, so the annotator sees what the other side was doing when the sound was made. A click on the picture plays from there. Under the picture is the row's text, if it has one. Each line is shown in its own direction, so an Arabic line reads right to left next to a line such as `PAUSE at 3.84 s`, and a long text scrolls rather than being cut. The clip plays as soon as a row opens (autoplay is on for this kind, and each annotator can turn it off). The next three clips are fetched while the open one is listened to, so moving on does not wait on the network, and a clip gone back to is not fetched again. **Play** plays it again from its start, **Only the sound** plays the marked part with a third of a second either side, and under **Hear** each track can be silenced: with one track on, it plays in both ears, which is how a quiet sound under a loud voice is heard. An answer is taken only once the sound has been played from its start for `listen_s` seconds, or to its end when it is shorter. A play that starts inside the sound or after it does not count. An answer already given can be changed without listening again. Picking an answer saves it, sets the row to **validated** and moves to the next clip, so a clip costs one key. Above the picture is how often each answer has been given so far in the whole dataset. | keys | action | | - | - | | `1` to `9` | pick that answer and go to the next clip | | `Space` | play the clip again from its start, or pause it | | `X` | play only the sound | | a track's first letter | silence that track, or bring it back: `C` and `A` for a caller and an agent. A track has a key when its name starts with a Latin letter that the other track and the keys in this table do not use | | `←` `→` | back to the previous clip, skip to the next | | `R` | play from the start | | `Backspace` | take this clip's answer back; the row is to do again | | `⌘ .` (Ctrl on Windows) | flag a clip that is broken or cannot be judged at all, and go on | Every key works on an Arabic keyboard layout too: it is read by its place on the keyboard as well as by the character it types. If "cannot tell" is a real answer for your question, make it one of the options. **Flag** is for a clip that did not play. ## Reword the task later The question, the options' labels and hints, the listening time and the tracks can be changed after the dataset is built, while people are labelling. The options keep their keys and their order, because the answers already given are stored by key and picked by place: ```bash theme={null} curl -X PUT "$LEVERS9_API/api/datasets/$ID/task" -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" --data @task.json # {"task": {...}}, each option with its "key" ``` A task that drops, adds, renames or reorders an option is refused. No row and no answer is touched, and an annotator's open link shows the new wording when they next load it. ## Give the rows a text later A built dataset's rows can be given a text, or a new one, without building it again. Write a file with a JSON line per row: ```json theme={null} {"id": "s00001", "text": "0.00 caller: ...\nPAUSE at 3.84 s"} {"id": "s00002", "text": ""} ``` ```bash theme={null} levers9 dataset set-text interruptions texts.jsonl ``` The API takes the same as `{"texts": [...]}`: ```bash theme={null} curl -X PATCH "$LEVERS9_API/api/datasets/$ID/text" -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" --data '{"texts": [{"id": "s00001", "text": "..."}]}' ``` Only the rows in the file change, and an empty text takes a row's text away. The clips, the ids, their order, the task and every answer stay as they are. Every id must be a row of the dataset and appear once: otherwise nothing is written and the answer names the ids it did not know. It counts the rows given, how many changed and how many rows now have a text. Under the `mask` personal data policy the text is scanned like a transcript, and a link shows it masked. A text set after the last scan is held back from links until you scan again ([Personal data](https://levers9.mintlify.site/lab/personal-data.md)). ## Share it with annotators A sound dataset is shared like any other: **Share** gives each annotator a link that opens this dataset's annotation screen and nothing else, with no account, and behind an agreement if the dataset has one ([Sharing](https://levers9.mintlify.site/lab/sharing.md)). The link plays this dataset's clips and no other file on the volume. Each answer is recorded with the annotator's name. ## Check how steady the answers are Two things you can do with the manifest alone: * **Repeats.** List some clips twice, under different ids and far apart. An annotator who gives the same clip two different answers tells you how far to trust a single answer. * **Overlap.** Put clips you have already labelled yourself among the new ones. Their answers measure how well the annotator agrees with you. In both cases the ids must not give it away, and the table that says which rows are repeats stays with you. ## Bring answers you already have Clips labelled somewhere else, on a page of your own or in a spreadsheet, keep their answers. Build the dataset from the clips, then import a TSV with a row per answered clip: `id`, `choice` (the key of the answer) and `validated` (`1`). **Import TSV** on the dataset page takes it, and so does the API, which also records whose answers they are: ```bash theme={null} curl -X POST "$LEVERS9_API/api/datasets/$ID/import?user=alfian" \ -H "Authorization: Bearer $TOKEN" --data-binary @answers.tsv ``` Every answer in the file is checked against the dataset's options before anything is written. One that is not an option refuses the whole file, with the row it is on. A row the file leaves out, or leaves blank, stays as it is, and the same file sent twice adds nothing. ## Get the answers out **Export TSV** (or `levers9 dataset export NAME`) has a row per clip with its `choice` (the key of the answer picked), and `validated` and `flagged`. Its `text` column holds each row's text on one line, with every line break written as `\n` and a backslash as `\\`. `labels.tsv` on the volume is the full log: every answer with who gave it and when, including answers that were later changed. ## Not yet * **Publishing to a bucket.** The published manifest is built around transcripts. Export the dataset instead. * **Pre-labelling.** Nothing drafts a sound dataset. * **Two annotators on the same clip.** A row holds one current answer: a second annotator's answer replaces the first as the row's answer, though both stay in `labels.tsv`. To compare two people, give each their own dataset built from the same folder. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.