Use Qualia from an agent

An agent uses Qualia with an API key, through the CLI or the SDK. A key acts as the person who made it, inside one organization. It can do what that person’s role allows, and its scopes can only narrow that.

This page walks one job from start to end: curating a dataset before a training run. You upload it, link your tasks, wait for the checks Qualia starts by itself, read the results as one table, rule on the episodes, pin a version and download it. Then it covers the cost, plain HTTP, a new batch every day, robots, finding your way around, admin work, and what a key cannot do. Each step shows the CLI first, then the same work in Python. The command reference and the SDK pages list every option.

Pass --json for output a script can read. A command that fails exits 1 with the reason on stderr. A command used wrongly exits 2.

Install

The CLI is the qualia-cli package, and it installs the SDK (qualia-sdk) with it. Install the version this deployment speaks: the exact command is under Installation, and the one for the SDK alone on the SDK overview. Keep the pin: a client and a deployment negotiate the upload protocol exactly, so they must be of the same generation.

qualia version

It prints the CLI and SDK versions that are installed.

Keys

Make a key in the dashboard: open Settings, then Account › API keys, click New key, then Create key. Copy the key: it is shown once. Keys expire after 90 days by default. No key can make another key.

The key acts in the workspace it was made in. QUALIA_WORKSPACE picks another workspace you can reach. The reference pages call it an organization key. Every step on this page works with it.

A key restricted to some workspaces (a workspace key) reaches less for now, and cannot walk this path: it is refused on tasks, which belong to the whole organization. It lists only its own workspaces, and it cannot change or delete a workspace.

A key carries every scope except three admin scopes, so it can upload and curate as soon as it exists. The three are members:read, members:write and integrations:write. None is on by default. Open More options and tick one under Admin (opt-in) when you make the key. A key with * does not get them. See Admin scopes.

Give the key to the CLI once:

qualia auth login --base-url https://junction-api.qualiastudios.dev
qualia auth status

qualia auth status shows what is stored. It does not call the API. qualia auth whoami does, so it proves the key works: it answers with the account, the workspace, the role and the key’s name and reach. Every command acts in the key’s own account, so none needs an account id (Your account).

An agent usually gets its key from the environment instead. Set QUALIA_API_KEY and QUALIA_BASE_URL together, because the CLI ignores its config file once the key is set. QUALIA_WORKSPACE picks the workspace to act in, if it is not the key’s own. Scripts and CI has the details.

The SDK reads the same variables:

from qualia import Qualia

client = Qualia()

The path

1. Upload

qualia dataset upload ./session-0412 session-0412 --standalone --json > receipt.json
DATASET=$(jq -r .dataset_id receipt.json)
qualia dataset ingest-report "$DATASET"

The later steps reuse $DATASET. The upload’s exit code says what it came to: 0 published, possibly with flagged episodes; 1 it failed or was refused; 3 the bytes are kept but no episode could be derived. qualia dataset upload lists the formats.

An episode keeps its place and its task. Its index is its episode_index in a LeRobot folder, counted from 0, and its task label is the folder’s task string for it: the text training reads.

The upload is not the end of it. Where a deployment prepares uploads, preparation and two checks start by themselves, and one of the checks costs credits. Read step 3 before you start anything.

Exit 1 does not mean nothing is left. A transfer that fails partway stays in the upload feed as failed, and the bytes it stored stay too. Read the feed before you try again: qualia dataset uploads works from any machine. cancel-upload stops an upload in flight. dismiss-upload takes a failed one off the feed and leaves its bytes.

--standalone keeps the upload on its own. Without it, a machine with a saved destination appends the upload to that merged dataset. The upload is then part of that dataset, and the server refuses the steps below on it: run them on the merged dataset (A new batch every day).

receipt = client.data.upload("./session-0412", name="session-0412", standalone=True)
dataset = receipt["dataset_id"]
report = client.datasets.ingest_report(dataset)

See data and datasets.

2. Set up the robot and gripper

Curation needs no robot: skip this step. A dataset needs one only to be played on it, which the reach check and handheld (UMI) recordings do. The commands are under Robots and grippers.

3. Know what starts by itself

Where a deployment prepares uploads, an upload starts three things and asks about none of them:

Preparation reads the takes. Count on 3 to 8 minutes and allow for more: one run of 24 episodes took 17, and the platform’s own deadline for a run is two hours. It sorts the takes into tasks and gives each one a Task: a published Task of the account that matches it, somebody else’s included, or a new draft. It links the episodes to that Task, and it may split a take into pieces.
The free checks (heuristic: code reads each episode’s data, with no model) start when preparation ends.
The paid check (VLM: a model reads each linked episode against its Task) starts then too. It reserves its whole estimate from the account’s credits. No upload option caps it or turns it off.

qualia dataset preparation "$DATASET"    # its state, the tasks found, the Task of each
qualia check runs --dataset "$DATASET"   # the runs on the dataset, newest first
qualia check runs --dataset "$DATASET" --json \
  | jq -r '.checkRuns[] | [.name, .state, .creator.agent] | @tsv'   # the same, with whole names
RUN=$(qualia check runs --dataset "$DATASET" --json | jq -r '.checkRuns[0].name')
qualia check status "$RUN"               # one run, by its whole name

The table cuts a run’s name short, so take names from --json. A run that started by itself has qds-scheduler as its creator.agent.

Whether this deployment prepares uploads shows in qualia dataset preparation, not in the upload’s receipt. A state of QUEUED, RUNNING, SUCCEEDED or FAILED means it does, and the checks follow. No prepare run (state NONE with --json) means no preparation is queued: nothing starts by itself, and you start the checks in step 5. Ask twice, some seconds apart, before you believe NONE. Right after an upload the preparation may not be queued yet, which is also why the receipt’s automatic_check can be missing where uploads are prepared.

The paid check judges each episode against the Task it links when the run starts: yours if you linked it first, else the one preparation gave it. So if you bring your own Tasks, do step 4 now, while preparation runs, and you pay for one reading, not two.

To stop a run you do not want, this one included, cancel it by the last part of its name: qualia status cancel "${RUN##*/}" (client.statuses.cancel(run.name.rsplit("/", 1)[-1])). The run ends FAILED. It pays for the calls it had made, and the rest of its reservation comes back.

See preparation and qualia dataset preparation.

4. Link your tasks

A Task says what the robot should do and how success shows. Make one for each task string in the dataset, publish it, and link the episodes that carry the string:

qualia dataset summary "$DATASET" --facet task    # the dataset's task strings, with counts
TASK=$(qualia task create pour.yaml --name "Pour coffee" --json | jq -r .name)
qualia task publish "$TASK" -m "First revision"
qualia dataset assign-task -d "$DATASET" --task "$TASK" \
  --label "pour coffee into the cup" --filter 'task = "pour coffee into the cup"'

pour.yaml holds your rules. A description and a success are enough to publish:

description: Pour coffee from the pot into the cup
success: The cup is full and no coffee is on the table
failure: Coffee is spilled, or the cup tips over

assign-task links every episode the filter leaves, so a dataset with four tasks takes four calls. Repeat the string the episodes already carry as --label: without it, their task label is replaced by the Task’s name.
A Task’s name is at most 32 characters, so a long instruction cannot be it.
Publishing a Task links, by itself, every unlinked episode of the workspace whose task label is exactly the Task’s name, in other datasets too. qualia task link-episodes "$TASK" --dry-run lists what that would be.
qualia dataset link-task "$DATASET" "$TASK" links the dataset instead. Only an episode with no link of its own reads that link, so it leaves the episodes preparation linked where they are. assign-task moves them.

Preparation leaves an episode you linked alone, so link before it ends and the check that starts by itself judges your Tasks. Link later and the links move all the same, but what that check read against the other Tasks no longer stands: run it again in step 5.

You can keep the Tasks preparation gave instead. qualia dataset preparation "$DATASET" lists each task it found with the id of its Task, and qualia task get ID shows what it says. Against a draft, a check raises notes only: publish it (qualia task publish ID), then run the check again in step 5. Two things to know. Preparation’s link also sets the task label of the episodes it links to the Task’s name: --facet task shows it, and assign-task --label puts a string back. And saving an edit to a Task that was never published starts the free and the paid check on its dataset again, by themselves, about two minutes after the last edit.

A link keeps the revision the Task had when it was made, and checks judge that revision: publishing a newer one starts no check. Each run names what it judged: qualia check status <run> prints Judged: Restock shelf v3. When a newer revision is out, qualia dataset get says so (v4 is published: qualia dataset relink-task), and the dataset page’s Task panel reads “Restock shelf v4 is out; checks ran on v3.” One call moves the dataset’s own link, and the episodes linked to that same Task, to the newest revision and reruns the checks preparation starts, the VLM check among them: qualia dataset relink-task "$DATASET" (client.datasets.relink_task), or the panel’s “Relink to v4 and rerun checks”. Both show the credit bound first: --validate-only (validate_only=True) answers it and changes nothing, and the dashboard asks before it starts. It needs the dataset itself to link the Task (link-task): a dataset linked only through its episodes answers links no Task; link one first. It never moves episodes linked to another Task than the dataset’s own.

task = client.tasks.create(
    "Pour coffee",
    draft={
        "description": "Pour coffee from the pot into the cup",
        "success": "The cup is full and no coffee is on the table",
        "failure": "Coffee is spilled, or the cup tips over",
    },
)
client.tasks.publish(task.name, note="First revision")
client.episodes.assign_task(
    dataset,
    "pour coffee into the cup",
    filter='task = "pour coffee into the cup"',
    linked_task=task.name,
)

See qualia task, tasks and Robots and tasks.

5. Wait, then read the results

A check never keeps, rejects or deletes an episode. It raises issues and notes. The runs that start by themselves come when preparation ends. Wait for it, then for them, before you read a result, and give every wait a limit:

wait_until() {   # wait_until SECONDS COMMAND...: ask every 20 s, give up after SECONDS
  local deadline=$((SECONDS + $1)); shift
  until "$@"; do
    [ "$SECONDS" -lt "$deadline" ] || { echo "gave up waiting for: $*" >&2; return 1; }
    sleep 20
  done
}
prepared() {   # prepared UPLOAD: its preparation is neither queued nor running
  qualia dataset preparation "$1" --json | jq -e '.state != "QUEUED" and .state != "RUNNING"' > /dev/null
}
followed() {   # followed DATASET SINCE KIND...: a run of each KIND started by itself after SINCE, all ended
  qualia check runs --dataset "$1" --json | jq -e --arg since "$2" '
    [.checkRuns[]? | select(.creator.agent == "qds-scheduler" and .createTime > $since)
      | .name | split("/")[3]] as $came    # accounts/A/checks/KIND/runs/R
    | all($ARGS.positional[]; IN($came[]))
      and all(.checkRuns[]?; .state == "SUCCEEDED" or .state == "FAILED")' --args "${@:3}" > /dev/null
}
wait_until 1800 prepared "$DATASET" &&
  PREP=$(qualia dataset preparation "$DATASET" --json) &&
  [ "$(jq -r .state <<< "$PREP")" = SUCCEEDED ] &&
  wait_until 1800 followed "$DATASET" "$(jq -r .createTime <<< "$PREP")" heuristic vlm

import time


def wait_until(done, seconds=1800, every=20):
    deadline = time.monotonic() + seconds
    while not done():
        if time.monotonic() > deadline:
            raise TimeoutError("gave up waiting")
        time.sleep(every)


def followed(dataset, since, kinds=("heuristic", "vlm")):
    """A run of each kind started by itself after `since`, and every run has ended."""
    runs = client.checks.list_runs(dataset=dataset).items
    came = {
        r.name.split("/")[3]  # accounts/A/checks/KIND/runs/R
        for r in runs
        if r.creator and r.creator.agent == "qds-scheduler" and r.create_time > since
    }
    return came >= set(kinds) and all(r.state in ("SUCCEEDED", "FAILED") for r in runs)


wait_until(lambda: client.preparation.get(dataset).state not in ("QUEUED", "RUNNING"))
prep = client.preparation.get(dataset)
if prep.state == "SUCCEEDED":
    wait_until(lambda: followed(dataset, prep.create_time))

An empty list of runs does not mean they ended. Right after preparation the runs may not have started yet, and the list may hold only a run you started or older ones. One finished run does not mean it either: the free run starts first, and the paid one can come minutes after it ended, since it waits for any VLM run already on the dataset, yours too. So the wait names both kinds, heuristic and vlm, the fourth part of a run’s name. It looks for a run of each that started by itself (creator.agent is qds-scheduler) after preparation was queued: none starts while preparation is queued or running. Then it waits for every run on the dataset to end. Name only heuristic where no episode links a Task, published or draft: no paid run comes then. Only a preparation that SUCCEEDED is sure to be followed. After FAILED (its error says why) or NONE, the wait stops there: start the checks yourself, below. At the limit, qualia check runs shows which kind never came.

You need not wait for preparation to start a run of your own. Once your episodes link your published Tasks (step 4), a run you start begins at once, whatever preparation is doing. Know what follows:

When preparation ends, the automatic runs start all the same. The paid one takes every linked episode again and reserves its whole estimate again. A reading it can reuse is not charged twice (What it costs), but nothing says beforehand how many it will reuse. Cancel it (step 3) if the account cannot spare the reservation, or if you have the verdicts you need. Once cancelled it stays away until the dataset is touched again: by a new upload, or by an edit to a Task that was never published.
Two VLM runs never read one dataset at once. While yours runs, the automatic one waits for it to end. A run you ask for while another is on the dataset is queued behind it, and reserves credits only when it starts.
A take that preparation splits after your run is replaced by its pieces, which your run did not read.

Two refusals only mean “wait”. Until preparation has linked the episodes, qualia check estimate answers no selected episode is linked to a published task or has a draft task. While the free checks run, starting them again answers … already running on this dataset in …; wait for it to end.

Start the checks yourself where none starts by itself, when you will not wait for preparation, or after you linked Tasks late:

qualia dataset checks list "$DATASET"    # every check: whether it can run here, its last result
qualia dataset checks run "$DATASET" --kind heuristic --all --wait
qualia check estimate "$DATASET"
qualia check run "$DATASET" --max-cost 200 --wait

The heuristic checks read each episode’s data with no model and cost nothing. The VLM check reads each episode against its linked Task with a model and costs credits: qualia check estimate starts and charges nothing, and --max-cost caps what the run may reserve and spend (What it costs). VLM catalog checks cannot run on this deployment.

heuristic = client.checks.run(dataset, kind="heuristic", all=True)
client.checks.wait(heuristic.check_run)
estimate = client.checks.estimate(dataset)
started = client.checks.run(dataset, max_cost=200)
vlm_run = client.checks.wait(started.check_runs[0])  # a VLM check starts one run

Read the results as one row per episode, from three lists:

qualia dataset episodes "$DATASET" --filter 'child_count = 0' -n 1000 --json > episodes.json
qualia dataset checks findings "$DATASET" --check graph.evaluate -n 1000 --json > verdicts.json
qualia check findings "$DATASET" -n 1000 --json > fails.json

episodes.json, under episodes: every episode, with its name, index, task, state and openIssueCodes (what blocks it, from any check), and a decision once someone has ruled on it. The filter leaves out a take that was split and keeps its pieces. Without it the list holds both, the take with childCount above 0 and each piece naming it in parentEpisode, while the dataset’s own counts and a version hold the pieces only.
verdicts.json, under checkFindings: what the VLM check concluded on each episode it read, passes too: episode, verdict (PASS, FAIL or UNKNOWN), confidence, explanation, taskRevision (the Task it was judged against) and checkRun. An episode with no finding here was not checked, whatever its state says.
fails.json, under issues: only the failures and doubts, each with blocks, its state and, where the model pointed at any, moments in finding.moments[].timeSeconds: seconds into the recording. A pass has none, and a failure can have none either. A finding names its issue in issue, which is the issue’s name. Where a finding says UNKNOWN, its issue says UNCERTAIN and the run counts it in uncertainCount: all three mean the model could not tell.

One check goes by three names. You run it as vlm. Its findings are filed under graph.evaluate, marked legacy: true and checkVersion: 0 even when new. Its issues carry the code graph_outcome, the verdict on the task as a whole. A Task with a step graph adds findings about its steps (graph_walk, graph_count_…), so an episode can have several, and one FAIL fails it.

What a failure does depends on how sure the model was. At 80% confidence or more, against a published Task, it raises an issue that blocks (blocks: true): the episode’s state is UNREVIEWED until someone rules on it. A failure below 80%, a doubt, anything read against a draft Task, and a failure on an episode you had already kept raise a note (blocks: false), which holds nothing. So state = KEPT is not a verdict. An episode nobody read, one read against a draft, and one that failed at 60% all stand kept.

episodes = client.episodes.list(dataset, filter="child_count = 0", limit=1000).items
verdicts = client.checks.list_findings(dataset, "graph.evaluate", limit=1000).items
fails = {i.name: i for i in client.checks.findings(dataset, limit=1000).items}

rows = {
    e.name: {"index": e.index, "task": e.task, "state": e.state, "checked": False}
    for e in episodes
}
for f in verdicts:
    row = rows.get(f.episode)
    if row is None:  # a finding on a take that was split since
        continue
    issue = fails.get(f.issue)
    row["checked"] = True
    row["blocks"] = row.get("blocks", False) or bool(
        issue and issue.blocks and issue.state == "OPEN"
    )
    if f.verdict == "FAIL" or (
        f.finding_key == "graph_outcome" and row.get("verdict") != "FAIL"
    ):
        row.update(
            verdict=f.verdict,
            confidence=f.confidence,
            reason=f.explanation,
            judged=f.task_revision,
            moments=[m.time_seconds for m in issue.finding.moments]
            if issue and issue.finding
            else [],
        )

qualia dataset checks findings lists the findings that stand: per check and episode, the newest run’s at the Task revision the episode links. --all-revisions adds the ones later runs superseded. Each call returns one page of at most 1000: past that, pass --cursor (cursor=page.page_info.end_cursor) for the next. To look at a moment, qualia dataset videos <EPISODE> -d "$DATASET" --download ./evidence/ep-7 saves <camera>.mp4 for each camera. Give each episode its own folder: a file of the same name is replaced.

qualia check status <run> prints where a run stands once and returns. Run it again until the run ends, or wait with --wait on qualia check run or with client.checks.wait(run). A VLM run that could not read some episodes still ends SUCCEEDED, and its status says so: Result: completed, 4 of 30 episodes not checked, then each of those episodes with why. Each also gets an open not_checked issue. Two kinds of run show the count alone, with no episode named and no not_checked issue: a run made before runs named what they could not read, and a run that was cancelled. There the rule above is the one to use: no finding, not checked.

When a VLM run has questions about the episodes it failed, --wait and qualia check status print them last, under Review health:, each with example episodes. Look at those episodes before you reject on the run’s findings.

See qualia dataset checks and qualia check, which also reads check runs, reference ranges and the reach check, and checks.

6. Rule

Read what is there, then rule. Rule after the checks: a failure a later run finds on an episode you kept is only a note. Pick episodes by name, or with the filter the dataset page’s filter bar writes.

qualia dataset summary "$DATASET" --facet state --facet open_issue_codes
xargs -r qualia dataset keep "$DATASET" --reason "passed the VLM check, nothing open" < keep.txt
qualia dataset reject "$DATASET" --filter 'open_issue_codes:"graph_outcome"' --reason "failed the VLM check"
qualia dataset reject "$DATASET" --filter 'metrics.duration_seconds < 2' --reason "too short"
EPISODE=$(jq -r '.episodes[0].name' episodes.json)
qualia dataset reset "$DATASET" "$EPISODE"    # withdraws a ruling

keep.txt holds one episode name per line: the rows of your table that passed and have nothing else open, so their state is KEPT. A keep or reject that names no episode is refused: the CLI exits 2, and the SDK raises selection.names is empty. So xargs -r runs nothing when the file is empty, and the Python below skips an empty list. A filter that selects nothing changes nothing and succeeds.

A keep is yours, and it is what a version takes by default (step 7). It also closes the blocking issues open on the episode, so never keep by a filter you have not read. A reject leaves its issues open, as the record of why. The first filter above rejects what the verdict on the whole task blocks; pass names, as for keep, to reject by your table.

keep and reject never delete anything, and reset withdraws them. A take a person has open in the studio is left as it was: the command lists it and still exits 0. Run it again once they are done.

Issues are settled apart from rulings:

qualia dataset issues summary "$DATASET"    # counts per code and state
qualia dataset issues list "$DATASET" --filter 'state = OPEN AND blocks = true'
qualia dataset issues dismiss "$DATASET" --filter 'code = "idle_tail"' --reason "fine for this task"
qualia dataset issues add "$DATASET" "Gripper slips at the second grasp" -e "$EPISODE" --at 12.5

See qualia dataset issues.

A trim or a split is drawn on the take’s draft. The dataset does not change: only a version published from a preview holds the drawing (step 7). qualia dataset trim --clear draws the full extent again, and qualia dataset unsplit discards a drawn split. --dry-run plans a split and writes nothing. qualia dataset edit changes labels, on the dataset itself.

qualia dataset edit "$EPISODE" -d "$DATASET" --outcome SUCCESS
qualia dataset trim "$EPISODE" -d "$DATASET" --start 1.5 --end 12
qualia dataset split "$EPISODE" -d "$DATASET" --keep 0.5-6 --keep 8-14 --dry-run

See qualia dataset edit, trim and split.

passed = [
    n for n, r in rows.items() if r.get("verdict") == "PASS" and r["state"] == "KEPT"
]
failed = [n for n, r in rows.items() if r.get("verdict") == "FAIL" and r["blocks"]]
if passed:  # an empty list of names is refused
    client.episodes.keep(dataset, passed, reason="passed the VLM check, nothing open")
if failed:
    client.episodes.reject(dataset, failed, reason="failed the VLM check")

episode = episodes[0].name
client.episodes.reset(dataset, [episode])  # withdraws its ruling
client.issues.dismiss(dataset, filter='code = "idle_tail"', reason="fine for this task")
client.episodes.update(episode, dataset=dataset, outcome="SUCCESS")

See episodes and issues.

7. Pin a version

A version freezes a selection. Training and downloads read versions, never the live dataset. By default a version takes the episodes kept by a person, a key’s keeps included, with the trims and splits saved on them. An episode that is only kept automatically is left out: keep it yourself, or pass --include kept.

qualia dataset version-preview "$DATASET" --json > preview.json
jq '.preview | {episodeCount, unprocessable}' preview.json
qualia dataset pin "$DATASET" 1.0 --preview "$(jq -r .preview.name preview.json)" --note "first cut"
qualia dataset version list "$DATASET"

The preview freezes and measures the selection and changes nothing. Read how many episodes it holds and the takes it leaves out, each with why, then pin exactly that preview. While it leaves a take out, the pin is refused: --allow-left-out (allow_left_out=True) accepts that. Each item names its take in take. Its label counts from 1, so episode 8 is the take whose index is 7.

--as-recorded and --digest tag head instead: every kept episode as recorded, the automatic keeps too, with saved trims and splits left out. --digest takes the targetDigest that qualia dataset version diff --json prints, and is refused if head moved since. A mixture always tags head. Tags never move. qualia dataset version create takes the same flags as qualia dataset pin.

preview = client.versions.preview(dataset)
print(preview.episode_count, preview.unprocessable)
tag = client.versions.create(dataset, "1.0", preview=preview, note="first cut")

See versions and Versions.

8. Download

qualia dataset download "$DATASET:1.0" ./session-0412-1.0

The CLI checks every file against its size and sha256, shows progress on stderr, and creates the folder only once the whole tree is there. A run that stops picks up where it left off.

The first download of a version in a format waits while the server renders it. That takes minutes, one to three for 10 to 24 episodes when measured, behind one spinner line, or behind nothing with --json. Starting a rendering needs a key that can write to the dataset (data:write); a rendered version downloads with read access (data:read). A rendering is not kept for good: a new renderer starts over, so a version rendered last week can render again today. client.versions.get_materialization("<dataset>:1.0", layout="lerobot_v3") reads its state (NOT_STARTED, RUNNING, SUCCEEDED or INTERRUPTED) and starts nothing.

--format takes lerobot_v3 (the default), lerobot_v2_1, mcap_ros2, pi_mcap_v1 and dreamlabs_v2_1 (files to send to Physical Intelligence or Dream Labs), or original (every file uploaded to the dataset, byte for byte; no version, and an organization key for now). See qualia dataset download and Download a version.

A LeRobot download holds the version’s episodes renumbered from 0, their task labels, and state and action values equal to the upload’s. A handheld (UMI) recording whose poses were resolved at the TCP is the exception: its observation.state holds the calibrated TCP poses instead, and it has no action columns (What a LeRobot download holds). The download says where each episode came from: robot_type in meta/info.json (the upload’s own, else unknown), and per episode qds_episode_id, qds_source_dataset_id (the upload) and qds_take_episode_id (the take a piece was cut from), in meta/episodes.jsonl (v2.1) or meta/episodes/chunk-000/file-000.parquet (v3). It holds no verdict, reason, confidence, moment, decision or subtask: keep your table beside it. qds_episode_id is the last part of an episode’s name, and that episode’s index is its episode_index in the folder you uploaded; a piece of a split take carries its take’s as parentIndex.

A LeRobot download’s video is AV1. A camera recorded in another codec is encoded again, at the same size and frame rate, so its pixels are close to the upload’s but not equal. For the robot’s own pixels, download original.

result = client.data.download(f"{dataset}:1.0", "./session-0412-1.0")

What it costs

The VLM check spends credits. The heuristic checks spend none.

The estimate is a ceiling, not the price. A run reserves its estimate when it starts and is charged for the calls it made when it ends, never more. The rest comes back. A run that fails on the platform’s side is refunded in full; one you cancel pays for the calls it had made. A run usually settles well below its estimate.
The balance must cover the ceiling. A run is refused when the account holds fewer credits than it would reserve, even if it would have cost less.
A reading is reused when nothing it read has changed: the same episode, not edited since, the same Task revision and the same model, in the same account, and an earlier reading that ended well (one that failed or was cut off by its cap is read again). A reused reading is not charged again. The run still reserves its whole estimate first, and neither the estimate nor the run says how many readings were reused.
Preparation is not charged. It reserves and spends none of the account’s credits. Nothing in the API says so: the balance just does not move while it runs.
--max-cost caps a run you start: what it may reserve and spend. The run that starts by itself sets its own cap, at its estimate.
No command shows what one run settled at. A check run carries no credits. qualia credit --json (client.credits.get()) gives the account’s balance: read it before the run and after it ends. Other work in the account moves the same balance, so the difference is this run’s only when nothing else ran.

Over plain HTTP

Every step but the upload is one or two calls, from any language. Send the key in the X-API-Key header. Everything on the path lives under /v1/accounts/{account}, where {account} is the account_id that GET /v1/whoami answers; - in that place of the path also means the key’s own account. The qualia.v1 reference has every call and body.

Step	Call under `/v1/accounts/{account}`
Find a dataset by its slug	`GET /datasets?filter=slug = "session-0412"`
Preparation	`GET /datasets/{dataset}/preparation`
Runs on a dataset	`GET /checks/-/runs?filter=dataset = "accounts/{account}/datasets/{dataset}"`
One run; stop it	`GET /checks/vlm/runs/{run}`; `POST /checks/vlm/runs/{run}:cancel`
Create, edit, publish a Task	`POST /tasks`; `PATCH /tasks/{task}`; `POST /tasks/{task}:publish`
Link episodes to a Task	`POST /datasets/{dataset}/episodes:assignTask`
Estimate the VLM check	`POST /checks/vlm:run` with `{"dataset": "accounts/…/datasets/…", "validateOnly": true}`
Run it	the same call with `"maxCost"` and no `validateOnly`
Episodes	`GET /datasets/{dataset}/episodes?pageSize=1000&filter=child_count = 0`
Verdicts, passes too	`GET /datasets/{dataset}/checks/graph.evaluate/findings?pageSize=1000`
Failures, with moments	`GET /datasets/{dataset}/issues?filter=check = "accounts/{account}/checks/vlm"`
Keep, reject	`POST /datasets/{dataset}/episodes:keep`; `POST /datasets/{dataset}/episodes:reject`
Preview, then pin	`POST /datasets/{dataset}/versionPreviews`; `POST /datasets/{dataset}/versionPreviews/{id}:publish`
Versions; what one holds	`GET /datasets/{dataset}/versions`; `GET /datasets/{dataset}/episodes?version={tag}`
Render a version	`POST /datasets/{dataset}/versions/{tag}/materializations/lerobot-v3:start`
Its state; its files	`GET` the same without `:start`; `GET` the same with `/artifacts` in its place
Balance	`GET /v1/credits`, outside the account

ACCOUNT=$(curl -s -H "X-API-Key: $QUALIA_API_KEY" "$QUALIA_BASE_URL/v1/whoami" | jq -r .account_id)
BASE="$QUALIA_BASE_URL/v1/accounts/$ACCOUNT"
curl -sG -H "X-API-Key: $QUALIA_API_KEY" "$BASE/datasets" --data-urlencode 'filter=slug = "session-0412"'
curl -sG -H "X-API-Key: $QUALIA_API_KEY" "$BASE/datasets/$DATASET/episodes" \
  --data-urlencode pageSize=1000 --data-urlencode 'filter=child_count = 0'
curl -s -H "X-API-Key: $QUALIA_API_KEY" -H "Content-Type: application/json" \
  -d "{\"dataset\": \"accounts/$ACCOUNT/datasets/$DATASET\", \"validateOnly\": true}" "$BASE/checks/vlm:run"

A filter is one query parameter, URL-encoded: slug = "session-0412", all_revisions = true (on findings, for the superseded ones too).
A list answers one page: ask for pageSize and send nextPageToken back as pageToken. The most a page holds differs by list: 1000 for episodes, findings, issues and runs, 500 for datasets and versions, 512 for a rendering’s files. A larger pageSize is refused (400), not cut down.
The episode list holds a take that was split beside its pieces: the take has childCount above 0, and each piece names it in parentEpisode. The dataset’s counts count the rows with childCount 0, and filter=child_count = 0 asks for those.
A count is a JSON string in some fields and a number in others: a dataset’s counts.episodes is "24", a version’s episodeCount is "24" and an estimate’s credits is "70", while a run’s vlm.episodeCount is 24. Convert every count before you compute with it.
A body with a field the call does not know is refused (400), so a misspelt validateOnly starts nothing.
The preview and the rendering answer at once and work in the background: read them again until READY (then PUBLISHED after the publish), or SUCCEEDED. Each file in the artifact list has a signed uri, its path and its sha256.
Errors come in two shapes: the API’s {"code": "NOT_FOUND", "message": "…", "retryable": false, "details": []} and the gateway’s {"detail": "…"}, for a missing key or an unknown route. Read code if the body has one, else detail.

An upload is many calls that hash, split and resume files. Use the CLI or the SDK for that step.

A new batch every day

Several uploads become one dataset by a merge. A merge takes at least two uploads, so the first batch is an upload on its own, the second one makes the merged dataset, and each later one is appended to it:

qualia dataset upload ./batch-1 batch-1 --standalone --json > day1.json
qualia dataset upload ./batch-2 batch-2 --standalone --json > day2.json
ONE=$(jq -r .dataset_id day1.json); TWO=$(jq -r .dataset_id day2.json)
qualia dataset merge "$ONE" "$TWO" --name shelf-restock --dry-run
MERGED=$(qualia dataset merge "$ONE" "$TWO" --name shelf-restock --yes --json | jq -r .name)
qualia dataset upload ./batch-3 batch-3 --merge-target "$MERGED" --json > day3.json
jq .merge_append day3.json

A merge is final. It cannot be undone as a whole, and the uploads become the merged dataset’s storage. --dry-run lists what differs between them first. Uploads merge only when they record the same cameras, features, channel names and frame rate, with poses in the same place (root or TCP).
After the merge, work on the merged dataset. Rulings, issues, edits and task links go through it and land in the upload that holds the episode. Checks, versions and downloads are the merged dataset’s own. A write that names an upload a merge took is refused. What you ruled before the merge reads the same after it, and a version tagged on an upload keeps resolving under the upload’s own name.
merge_append in the receipt says whether the append happened: appended, or action_needed with why. Then append it yourself: qualia dataset merge <UPLOAD> --into "$MERGED" --yes.
The merged dataset is a new dataset. It has its own id and slug, and its own tags: a version pinned on the first upload stays under that upload’s name (batch-1:1.0), and version list on the merged dataset does not show it.
Each appended upload is prepared on its own, where uploads are prepared: qualia dataset preparation <UPLOAD> reads it, with the dataset_id of its receipt. The automatic checks that follow run on the merged dataset as a whole. The paid one takes every linked episode in it, the old batches too, and reserves for all of them: 111 credits for 30 episodes of which 15 were new, in one case. What it can reuse is not charged again, but its estimate does not show that. To pay for the new batch only, cancel that run and start your own on the new upload’s episodes, as below.
Verdicts made before the merge are listed under the upload. dataset checks findings through the merged dataset lists only what runs on the merged dataset found, so a batch checked while it was still an upload looks unchecked there. Read the findings of each upload the merged dataset holds (its sourceDatasets) instead: that lists every verdict on the upload’s episodes, whichever run made it. An episode keeps the last part of its name through the merge, so join on that. Issues, with their moments, do read through the merged dataset.
Tell the batches apart by upload. Every episode names the upload it came in with, and a filter takes it: --filter 'upload = "accounts/<ACCOUNT>/datasets/<UPLOAD>"'. Its index counts inside its upload, as the folder did; position counts across the merged dataset. A download names it as qds_source_dataset_id.
A new version takes what is kept now, the new batch’s kept episodes with the old. qualia dataset version diff "$MERGED" shows what came in since the newest tag. Earlier tags do not change.

NEW="${MERGED%/*}/$(jq -r .dataset_id day3.json)"   # the new upload's whole name
qualia dataset assign-task -d "$MERGED" --task "$TASK" --label "pour coffee into the cup" \
  --filter "upload = \"$NEW\" AND task = \"pour coffee into the cup\""   # day one's Task
qualia check estimate "$MERGED" --filter "upload = \"$NEW\""
qualia check run "$MERGED" --filter "upload = \"$NEW\"" --max-cost 60 --wait
for UPLOAD in $(qualia dataset get "$MERGED" --json | jq -r '.sourceDatasets[]'); do
  qualia dataset checks findings "$UPLOAD" --check graph.evaluate -n 1000 --json
done | jq -s '[.[].checkFindings[]]' > verdicts.json   # every verdict of every batch

So each day is the same path with $MERGED as the dataset, with four differences:

Link only the new episodes, and to the Tasks of the first day. Keep each Task’s name ($TASK; qualia task list lists them) and put the new upload in the filter, as above. assign-task moves every episode it selects that links another Task. So a new Task, or a filter without upload, also moves the old batches’ episodes, and the verdicts they had against the old Task no longer stand.
Wait for the preparation of the new upload, then for the runs on $MERGED: prepared "$NEW", then followed "$MERGED" with that preparation’s createTime and heuristic vlm (step 5).
Check the new batch with a filter if you will not pay to reserve for the old ones.
Read the verdicts upload by upload.

Then rule, pin the next tag and download.

qualia dataset destination set "$MERGED" saves the merged dataset for later uploads without --merge-target. The saved destination applies only when the CLI runs against the same base URL and workspace and is given the account (QUALIA_ACCOUNT). With only a key and a base URL in the environment it is not used, so a script is surer with the flag.

day1 = client.data.upload("./batch-1", name="batch-1", standalone=True)
day2 = client.data.upload("./batch-2", name="batch-2", standalone=True)
merged = client.datasets.merge(
    [day1["dataset_id"], day2["dataset_id"]], display_name="shelf-restock"
)
day3 = client.data.upload("./batch-3", name="batch-3", merge_target=merged.name)
print(day3["merge_append"])

new = f"{merged.name.rsplit('/', 1)[0]}/{day3['dataset_id']}"  # the new upload's whole name
client.episodes.assign_task(
    merged.name,
    "pour coffee into the cup",
    filter=f'upload = "{new}" AND task = "pour coffee into the cup"',
    linked_task=task.name,  # day one's Task
)
started = client.checks.run(merged.name, filter=f'upload = "{new}"', max_cost=60)
for run in started.check_runs:  # run() returns at once: wait before reading
    if client.checks.wait(run, timeout=3600).state != "SUCCEEDED":
        raise RuntimeError(f"{run} did not succeed")  # its findings were not written
verdicts = [
    f
    for upload in client.datasets.get(merged.name).source_datasets
    for f in client.checks.list_findings(upload, "graph.evaluate", limit=1000).items
]

See qualia dataset merge and datasets.

Robots and grippers

Curation needs no robot. Set one up when the dataset is played on it:

ROBOT=$(qualia robot create lab-arm --urdf lab_arm.zip --json | jq -r .name)
qualia robot model "$ROBOT"
qualia robot configure "$ROBOT" --solver dls
GRIPPER=$(qualia gripper create parallel_gripper.urdf --json | jq -r .name)
qualia gripper tcp "$GRIPPER" tcp.json
qualia robot carry "$ROBOT" "$GRIPPER"
qualia dataset update "$DATASET" --target-robot "$ROBOT"

qualia robot create with --urdf imports the file and waits for it. qualia robot model shows the arms it guessed, and qualia robot configure corrects them. qualia gripper tcp says where the tool centre point is. qualia robot carry gives the robot its own copy of the gripper, so change the copy from then on. qualia dataset update picks the robot the dataset plays on.

robot = client.robots.create_from_urdf("lab_arm.zip")
model = client.robots.get_model(robot.name)
client.robots.update_model(robot.name, solver="dls", etag=model.etag)

gripper = client.robots.create_from_urdf("parallel_gripper.urdf", kind="gripper")
client.robots.set_gripper_tcp(gripper.name, file="tcp.json")
client.robots.set_gripper(robot.name, gripper.name)
client.datasets.update(dataset, target_robot=robot.name)

See robots.

Find your way around

To see	CLI	SDK
Workspaces	`qualia project list`	`client.projects.list()`
One workspace	`qualia project show`	`client.projects.get(project_id)`
What people and keys did	`qualia activity list`	`client.activity.list()`
Running work	`qualia status list`	`client.statuses.list()`
Stop running work	`qualia status cancel`	`client.statuses.cancel(name)`

qualia activity list --org --filter 'verb = KEEP AND actor = API_KEY' --json
qualia status list --running --json

Admin

Admin work needs an opt-in scope ticked when the key was made. The scope opens the door. The role of the key’s creator still decides: a Contributor’s key with members:write cannot change a role, and no key grants a role above its creator’s own. A guest’s key can do none of this. Activity records each write with the key’s name. credits:write is an ordinary scope that every new key holds.

People and roles (members:read, members:write): qualia member list, show, role, remove; client.members list, get, set_role, remove.
Invitations (members:read, members:write): qualia invitation list, send, withdraw; client.invitations list, send, withdraw.
Integration keys (integrations:write): qualia integration list, add, remove, default; client.integrations list, add, remove, set_default.

An integration key’s value goes in once and is never shown again, so pipe it in and keep it out of your shell history:

printf %s "$HF_TOKEN" | qualia integration add huggingface --name ci --secret-stdin

What a key cannot do

These stay in the dashboard:

Log in or sign up. A person makes the key.
Reset a password. That is the person’s.
Change appearance. Theme is a browser setting.
See presence. Who is on a page is shown to signed-in people.
Hold an edit lease. A person editing a take blocks a key’s write to it.
Create or delete an organization. A person does that.

Try the whole path

sdk/cli/examples/agent_path.sh, in the CLI’s source, runs most of these commands with --json against the stack named by QUALIA_BASE_URL and QUALIA_API_KEY: an upload, a robot and gripper, a task, checks, rulings, a version and an export. It makes its own data, prints a line per step, and exits non-zero if a step fails. Where a stack cannot finish a step, it says so.

On exit it deletes what it made and cancels an upload it did not finish. An upload’s repository and stored bytes stay, because no command deletes them. The script prints a line for each one.