Robots and Tasks

A robot is a URDF in your workspace. The trajectory views draw episodes on it, and the reach solver walks handheld takes onto its arms. Neither works without the robot’s URDF.

A Task describes what the robot does and what success and failure look like. A model reads this text and your reference takes to draft the steps and check every uploaded episode against the Task, so these words decide what gets flagged.

A Task is a draft you edit, published as numbered revisions that never change. A dataset links to a Task, and every episode with no link of its own reads it.

This guide adds a robot, picks it as a dataset’s robot, then creates, publishes and links a Task.

Add a robot

The Rigs page lists robots, and the UMI rigs you made. A rig is made from templates the workspace keeps. Open the menu on the + Rig button with its chevron, click New template, choose Robot, and choose a .urdf or .zip with its meshes (up to 256 MiB). The template opens while its model converts. Then click + Rig (or New rig in the menu), pick the robot template and, if you want them, a gripper template and a solver template. A gripper asks only for the link of each arm it attaches to (proposed for you, one per arm: the arm’s tool flange, the link its own gripper is bolted to, where the robot has one away from its last arm link, else that last link); its TCP, mount transform and position are set on the rig’s page afterwards. The rig opens when it is made. The workspace’s oldest robot is the default for handheld uploads that do not name one.

New template also makes a Gripper template from a .urdf or .zip with its meshes (up to 256 MiB), or a Solver template from a solver file (a .json of up to 16 KiB: {"solver": "pyroki" | "mink", "parameters": {...}}, with every parameter of the solver and its schema marker, or {"solver": "dls"}). The page says what a file lacks. Templates in the menu lists the robot templates, gripper templates and solver templates the workspace keeps. A template’s page imports, replaces and renames its model, and a gripper template’s page sets its TCP. A rig keeps its own copy of what it was made from, so changing or deleting a template changes no rig.

Configure a robot

Open a robot to see its 3D model and Configuration JSON. Pick a gripper or choose Upload gripper to import a new workspace source. The upload stages the selection; it does not change the robot until Save. Selecting a new gripper opens the JSON with mounting fields for each arm. The picker offers the gripper templates and the gripper this rig carries now; Save makes an independent copy. Changing the selection while editing asks before discarding the current draft. A gripper with measured mounts for this robot previews immediately. For other pairs, set attachment.parent_link and its transform to preview the gripper. Ctrl+Space offers compatible mounting links (the link the gripper bolts to first: the tool flange, where the URDF has one past the wrist), arm endpoints and jaw joints in their JSON values. With two arms, the jaw joint is offered first as a {side} template that maps the jaw on both arms, then as literal joint names; a suggestion edits the draft and writes nothing until Save. A null parent leaves that arm unmounted. Set the gripper link → TCP if it is not already calibrated. See Configuration JSON for the fields and setup workflow. Matrices are row-major 4×4 transforms in metres. A new TCP needs an explicit gripper URDF link and matrix; no placement is guessed. Valid edits update the preview; invalid JSON keeps the last valid preview. Discard drops the draft. After a save the bar beside Save says Saved until you change something again.

Saving can write the robot model and gripper copy in sequence. If a later step fails, the page says what was saved; reload before finishing the remaining configuration. Editing the robot’s owned gripper does not edit the workspace source. Replacing an owned gripper asks for confirmation. Delete a gripper template from its own page under Templates, or a UMI rig from the menu on the rig’s own page; only the account owner can, and for everyone else Delete is greyed out. A live dataset using it blocks deletion. To remove a rig’s own gripper, pick None (bare flanges) in its Gripper selector and save. A UMI rig is deleted like any gripper, and nothing makes it again. Older servers that do not support exact staged copying cannot save a staged selection.

The Solver page has its own JSON editor and Save and Discard controls. It shows the robot’s whole solver file: the solver and, for PyRoKi and Mink, its parameters. Edit any of them and save, or switch to another solver by typing its whole file; the page says what a file lacks, and refuses what the server refuses. Saving there shows a “Solver saved” message and Saved beside the buttons. Saving configuration or solver changes marks earlier reach results stale; it does not run the solver again. Below the editor, Apply a solver template gives the robot a copy of a solver template’s solver and parameters, the way to give it a PyRoKi or Mink solver file without typing one; editing or deleting the template changes nothing on the robot, and applying one marks earlier reach results stale. Apply waits while the file above has an unsaved edit: save or discard it first. A robot has no solver until you set one here, and the solver cannot run on a robot without one. Replace URDF imports a new model onto the same robot.

The robot card’s … menu offers Open, Copy, Rename and Delete. Only the account owner can delete a robot; a dataset still played on it blocks deletion. The robot cards and detail page do not list associated handheld uploads, but playback associations remain stored.

Configuration JSON

This is the dashboard editor’s schema; SDK and CLI operations have their own fields and methods. Use valid JSON without comments. Keep each existing arm exactly once; this editor does not add arms. Omitted settings retain their saved values, and unsupported fields are rejected.

• arms[].side identifies an existing arm. Use the sides already in the document.
• arm_end_link is the robot link at the end of the IK chain. It may differ from the physical attachment parent. If omitted, the saved endpoint is retained; an endpoint shared with an attachment follows its parent.
• attachment can be an object or null to remove it. Its parent_link is the robot’s physical mounting link; built_in is true only when the robot URDF already includes the gripper; and transform is the row-major 4×4 transform from that parent to the gripper link containing its TCP, with translation in metres. A built-in attachment uses the gripper link and identity transform.
• gripper_link_tcp maps existing hand sides to row-major 4×4 matrices locating the TCP in the gripper link, with translation in metres. Keep the existing hand keys. When first configuring an unplaced TCP for a staged or robot-owned gripper, replace that hand’s null with an object containing link, a link in the gripper URDF, and matrix, the transform. The editor accepts this object only for a first, unplaced hand.
• jaw can be null or an object with joint, open and shut. Name an actual jaw joint and use its measured positions in the joint’s native units.
• tool_frame is an optional advanced legacy frame on an arm, separate from the gripper TCP. Its xyz_m and rpy_rad each contain three numbers: translation in metres and roll, pitch, yaw in radians.

For example, this excerpt shows the shape of an initial TCP entry for a right hand. The link and identity matrix are illustrative, not a measured calibration; use the actual arm side, URDF link and measured transform for your gripper. Keep the rest of the existing document.

{
  "gripper_link_tcp": {
    "right": {
      "link": "gripper_link",
      "matrix": [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]
    }
  }
}

Choose or upload the gripper, configure its arm attachment and TCP, then review and Save. Use Move and Turn in the viewer when available, and use measured transforms or an imported calibration rather than guessing matrix values. Save writes the configuration; it does not run the solver.

For UMI recordings, set the shared tracking-controller → TCP calibration in UMI calibration on an applicable robot page. On the dataset’s Trajectories tab, move poses to the tool center point as needed, select the robot, and run the solver. Robot solver settings have their own Solver page, which shows the robot’s whole solver file (the solver and its Mink or PyRoKi parameters) to tune and save; neither solver settings nor tracker_to_tcp belong in Configuration JSON. See UMI calibration and dataset setup for details.

For scripts and agents, see the Python SDK robot resource and CLI robot commands. Their interfaces are not the dashboard JSON schema above.

Copy a robot

Choose Copy… from a robot card. Name the copy and keep its gripper, pick a different source, upload one, or leave the flanges bare. A copied gripper belongs to the copy, so edits do not change the original robot. The copy is not the workspace default and does not take the original’s datasets or reach results. If a gripper cannot be attached, the new robot remains available to configure.

UMI calibration

A UMI rig is made from a gripper template: choose New rig, UMI rig, pick the gripper template and name it, and the rig opens to its calibration. A workspace has the UMI rigs someone made and none otherwise. An upload is recorded on a rig, so make the rig before you upload; an upload that names a rig the workspace does not have is refused, and with several rigs the upload has to say which one.

On a UMI rig, or a robot carrying a gripper from one, UMI calibration shows the capture gripper in 3D and edits its tracker → TCP transforms, one section per rig. Edit JSON opens left and right matrices in metres; Save validates and imports them using the captured revision, and Discard drops the draft. To create an initial calibration, use Import calibration and review the file before importing. Changes affect existing datasets only when their poses are moved again. A robot’s gripper link → TCP and the shared tracker → TCP are separate settings.

Pick the rig a dataset plays on

The list shows each rig with the gripper it carries, for example rig · gripper pair, or rig · its own gripper. A rig whose gripper has no TCP says so after it (franka-rig-2 · robotiq · no TCP), so you see it before Run solver refuses it. The note appears for the other rigs a moment after you open the list. The gripper is picked on the rig’s page; a dataset picks only the rig.

A dataset with hand poses (a UMI recording, marked UMI · Beta) is set up on its trajectories tab, in the Setup strip at the top:

1. Play on rig: pick the rig and its configured gripper from the list. The pick saves at once. Click (?) beside any row for what it means.
2. Poses: one line says where the poses stand: at the tool center point · 43 takes, moving to the tool center point, at the gripper root, or, when the picked rig has no TCP, at the gripper root · no TCP on this rig. Picking a rig whose arms all have a TCP moves the poses to the tool center point by itself, once, when they are not there yet. The move uses the calibration of the gripper the recording was captured with. The (?) explains the steps without equipment names. It starts nothing for a rig with no TCP (the line then offers Move to the gripper’s tool center point if you want it anyway), and never moves them back. While it runs, Run solver waits. The ⋯ menu has Move again and Back to the gripper root.
3. Solver: the rig’s IK solver, shown but not a choice (a dataset has no solver of its own; you set it on the rig’s Solver page), then Run solver. When it cannot be pressed, the reason is written beside it: no rig is picked, the poses are still being moved, the poses are at the TCP but the rig has no TCP, the rig has no solver, the setup is not verified, or your role cannot change the dataset. With the poses at the gripper root it can still be pressed. If where the gripper mounts on the rig is a guess, it warns you. If the rig or solver status could not be read, the row offers Retry setup check, which reads them again and changes nothing. Below it, Placement opens to Forward x, Left y, Up z (metres) and Heading (degrees), which move both hand paths together from the rig’s home midpoint. Click Save placement.

The solver runs only on takes tagged umi or egocentric. A dataset that holds hand poses but carries neither tag has the strip, with Play on rig and Poses, and no Solver row: there is no Run solver button and no solver settings. Add the UMI tag on the dataset’s settings tab, page collection, to get them.

Moving a merged dataset to the tool center point is not available yet. Its strip shows where its poses stand, read only. The overview of a UMI dataset links to Setup only while the setup is unfinished (no robot, a setup that is not verified), and says nothing once it is ready. Its settings tab, page collection, shows a read-only summary with a link to Setup.

Any other dataset (teleop, rollout, with no hand poses) has no Setup strip and no solver. Its trajectories tab says Play on rig <rig> · <gripper> with Change rig (or Choose rig when none is selected), which opens the picker on the settings tab, page collection, in the Played on panel. It shows No rig when none is picked.

With no pick, a dataset shows the rig it was recorded on. Picking a rig starts nothing: no reach run is queued and no issue is raised.

The same pick sets the gripper in the episode viewer’s trajectory card. The card draws the Played on rig’s gripper at its TCP: the gripper the rig carries, or its own fingers when it carries none, and says which in the line under its title (rig · its own gripper). A handheld take is drawn with that gripper too, whatever kind it is, not the rig’s stock hand. Change the pick and the card follows.

An upload that was merged into another dataset takes its robot from the merged dataset. Set it there.

If the account has no robots yet, the panel links to the Rigs page.

Create and publish a Task

Tasks are on the Tasks page.

1. Click + task. An Untitled task is created and opens.
2. Click the name at the top to rename it.
3. Fill the fields on Overview. Description and Success are required.

Field	What it holds, and how the checks use it
Description	What the robot does, e.g. “restock the shelf with bulbs from the crate”. The model reads it as the goal
Scene	The workspace, the objects that matter, how an episode starts, what varies. The model reads it to tell variation from a problem
Steps	The steps in order, one line per step. The model checks the order of actions against them
Failure	How a failure looks on camera, e.g. “a bulb falls off the shelf”. The model flags an episode that shows one
Success	How the end state looks when done, e.g. “every bulb rests on the shelf”. Every episode is checked against it

Steps can repeat. One line per step is enough. Write a repeated motion once and say when it ends: “Pick a bulb from the crate and place it on the shelf. Repeat until the crate is empty.” Set min/max counts on the step in the graph. The graph is optional. It can be generated later from this text and reference takes, then edited by hand. A repeated pick-and-place shows a worked graph.

The draft saves itself. The status beside Publish says when it did, or Not saved with a retry.

4. Click Publish. In the dialog, write What changed (up to 1000 characters) and click Publish revision N.

Publish is greyed out when nothing changed since the last revision, or when something publishing needs is missing. Open the Info drawer: its Issues panel lists each gap. Click one to jump to where it is.

The other tabs:

• Graph: the steps as a state machine. Optional. When it has steps, it needs one start, a success outcome, every step reachable and no loops.
• References: pick example episodes from a dataset: 1–3 successes, and 1–2 takes per failure or recovery mode. Draft with VLM fills the fields from these picks for you to review. Estimate draft prices it first. It never publishes.
• Revisions: every published revision, newest first. Each one opens on its own page.
• Linked: the datasets and episodes in the workspace linked to this Task.

The Tasks list shows each task as draft, rev N, or rev N with unpublished changes.

A repeated pick-and-place

A graph has no loops, so a repeat is a count, not an edge back. When each repeated step’s count allows more than one (a maxCount above 1, or none with a minCount), a take may run the steps again: pick_bulb → place_bulb → pick_bulb needs no edge from place_bulb back to pick_bulb. Publishing refuses that edge: the procedure loops (pick_bulb → place_bulb → pick_bulb): let a step happen several times instead.

A crate of 3 bulbs, as a task file:

description: Restock the shelf with bulbs from the crate
scene: A crate of 3 bulbs on the left of the table, an empty shelf on the right
steps: |-
  Pick a bulb from the crate and place it on the shelf.
  Repeat until the crate is empty.
success: Every bulb rests on the shelf and the crate is empty
failure: A bulb falls off the table
procedure:
  steps:
    - id: start
      kind: START
    - id: pick_bulb
      kind: ACTION
      description: Pick a bulb from the crate
      minCount: 3
      maxCount: 3
    - id: place_bulb
      kind: ACTION
      description: Place the bulb on the shelf
      minCount: 3
      maxCount: 3
    - id: done
      kind: SUCCESS
      description: Every bulb rests on the shelf
    - id: dropped
      kind: FAILURE
      description: A bulb falls off the table
  transitions:
    - { source: start, target: pick_bulb }
    - { source: pick_bulb, target: place_bulb }
    - { source: place_bulb, target: done }
    - { source: pick_bulb, target: dropped }
    - { source: place_bulb, target: dropped }

qualia task edit <TASK_ID> restock.yaml
qualia task publish <TASK_ID> -m "Steps as a graph"

The file replaces the whole draft, so it holds the five fields as well as the procedure. minCount and maxCount both 3 is exactly 3. minCount alone has no upper limit, for a crate that is not always full. maxCount alone allows zero. A step with neither happens exactly once.

A good take walks start → pick_bulb → place_bulb → pick_bulb → place_bulb → pick_bulb → place_bulb → done. The checks read other takes like this:

Take	Finding
pick_bulb has no count and repeats	pick_bulb follows place_bulb, which the task does not allow: a count above 1 on pick_bulb lets it repeat
The last bulb is picked, never placed	done follows pick_bulb, which the task does not allow
A bulb falls	task review: the walk ended in failure step dropped: <what the model saw>
2 bulbs placed, counts set by a person	only 2 of 3 place_bulb found
4 bulbs placed, counts set by a person	4 place_bulb found, at most 3 allowed

Only a count a person sets is enforced: set it under Count on the step in the Graph tab. Counts saved with the CLI or the SDK still let steps repeat, but the check only reports what it saw: 3 place_bulb found, not counted.

One step, “Pick a bulb from the crate and place it on the shelf”, with the same count also works, and is the smaller graph.

Link a Task to a dataset

On an upload’s or a merged dataset’s overview tab, the task panel shows the linked Task. A mixture has no picker: it reads the Tasks of its uploads.

1. Click Select task (or Change task when one is linked).
2. Pick the Task. The link saves at once.
3. If some episodes link the old Task themselves, a dialog asks whether to move them too: Move them or Keep theirs.

A link is not a label. The episodes’ task label, which training reads as the language instruction, stays as it is. To set the label too, use qualia dataset assign-task below.

Link matching episodes in the Task’s … menu links every unlinked episode and upload whose task label names the Task. Labels that are exactly the Task’s name come ticked. Near misses (case, spacing, punctuation) are listed for you to tick. Publishing a Task links the exact matches on its own.

A Task’s name is at most 32 characters. A Task that preparation drafted from an upload’s own instructions keeps each whole instruction as a linked label, listed on its Linked tab. An episode whose task label is one of them links as one labelled with the name does. Only the task label is compared, never the episode’s instruction: an episode with an instruction and no task label does not link this way.

From the CLI

Every qualia task command acts in the API key’s own account; no --account is needed. See Authentication.

# create a draft, fill it, publish revision 1
qualia task create --name "Cube in box" \
  -d "Pick the cube off the table and put it in the box"
qualia task edit <TASK_ID> --success "The cube rests inside the box"
qualia task publish <TASK_ID> -m "First revision"

# link an upload to it: every episode with no link of its own reads it
qualia dataset link-task <DATASET_ID> <TASK_ID>

# or label and link the kept episodes in one go
qualia dataset assign-task -d <DATASET_ID> --task <TASK_ID> --filter 'state = KEPT'

Use link-task to say which Task an upload was recorded against. It changes no label. Use assign-task when the episodes also need the task label training reads. See qualia task for every command.

A link keeps the revision the Task was at when it was made, and checks judge that revision. Publishing a new revision changes no link and starts no check. qualia dataset get shows when a newer one is published; relink the dataset and rerun its checks in one step:

qualia dataset relink-task <DATASET_ID> --validate-only   # what it would cost
qualia dataset relink-task <DATASET_ID>

On the dataset page, the Task panel says so too (“Restock shelf v4 is out; checks ran on v3”). An editor relinks there with Relink to v4 and rerun checks, after confirming the credit bound.

Robots have commands of their own: qualia robot and qualia gripper make and set them up, and qualia dataset update --target-robot picks the robot a dataset plays on.

From the SDK

from qualia import Qualia

client = Qualia()
draft = {"description": "Put the cube in the box", "success": "The cube is in the box"}
task = client.tasks.create("Cube in box", draft=draft)
client.tasks.publish(task.id, note="First revision")
robot = "accounts/<ACCOUNT_UUID>/robots/<ROBOT_ID>"
client.datasets.update("<DATASET_ID>", linked_task=task.name, target_robot=robot)

The client acts in the API key’s own account; account= is an optional override.