Checks_RunDatasetChecks
const url = 'https://junction-api.qualiastudios.dev/v1/accounts/example/datasets/example/checks:run';const options = { method: 'POST', headers: {'X-API-Key': '<X-API-Key>', 'Content-Type': 'application/json'}, body: '{"dataset":"example","checkIds":["example"],"categories":["example"],"all":true,"selection":{"names":{"names":["example"]},"filter":"example"},"maxEpisodes":1,"maxCost":"example","validateOnly":true,"model":"example","requestId":"example","kind":"KIND_UNSPECIFIED"}'};
try { const response = await fetch(url, options); const data = await response.json(); console.log(data);} catch (error) { console.error(error);}curl --request POST \ --url https://junction-api.qualiastudios.dev/v1/accounts/example/datasets/example/checks:run \ --header 'Content-Type: application/json' \ --header 'X-API-Key: <X-API-Key>' \ --data '{ "dataset": "example", "checkIds": [ "example" ], "categories": [ "example" ], "all": true, "selection": { "names": { "names": [ "example" ] }, "filter": "example" }, "maxEpisodes": 1, "maxCost": "example", "validateOnly": true, "model": "example", "requestId": "example", "kind": "KIND_UNSPECIFIED" }'Runs catalog checks over a dataset as one run: the union of
check_ids, categories and all, deduplicated, on the episodes of
selection, sampled to max_episodes. The selection is resolved and
pinned before the estimate (check versions, Task revisions, episodes),
and the run reads exactly what was pinned. One model call per episode
reads every selected VLM check. An episode a check cannot read (no
published Task for a Task check) is skipped for that check, never
passed. all and a category take the checks that can run and list the
rest in skipped_checks. INVALID_ARGUMENT, before anything starts, for
an empty selection, an unknown check id or category, a named check that
cannot run on this dataset, a selection none of whose checks can run, or
when no episode is eligible for any selected check;
FAILED_PRECONDITION on a deployment with no checks runner, and, for a
person’s run, while a selected check is still running on this dataset:
the message names that run, which DatasetCheck.running_check_run also
lists. An estimate (validate_only) is answered either way. A retry with
the first call’s request_id is answered with its run, not refused. With
validate_only nothing starts and nothing is reserved: the response
carries the resolved selection and pipeline-api’s bound for it, the
same bound a run reserves. A run needs max_cost at or above that
bound. Each run is a CheckRun under checks/vlm.
Authorizations
Section titled “Authorizations ”Parameters
Section titled “ Parameters ”Path Parameters
Section titled “Path Parameters ”The account id.
The dataset id.
Request Body required
Section titled “Request Body required ”Request for Checks.RunDatasetChecks.
object
Accounts/{account}/datasets/{dataset}
Check ids to run, e.g. task.graph.
Categories to run every runnable check of the selected kind.
Run every runnable check of the selected kind.
Only these episodes (names under dataset, or a ListEpisodes filter);
unset means all.
object
Episode names under the request’s parent.
object
Accounts/{account}/datasets/{dataset}/episodes/{episode}, at most 10000.
A ListEpisodes filter over the parent (e.g. what the filter bar leaves).
Sample at most this many eligible episodes, evenly spread and the same for the same request. Must be positive; unset reads all of them.
The most credits the run may reserve and spend. Required to start a
VLM run, and must cover the bound; optional with validate_only.
Heuristic runs have a bound of zero and do not require max_cost.
Resolve the selection and answer the bound, starting and reserving nothing.
The Qualia model tier that reads the VLM checks, one of CheckEstimate.models; a legacy vendor model id is still accepted. Empty: the runner’s default.
Idempotency key (AIP-155): a retry with the same id within 24 hours returns the first call’s run and its pinned selection instead of starting a second one. Required to start a run.
Kind selected by all and categories. Unset: the kind check_ids names;
else VLM when a VLM check they reach can run here, else heuristic. A run
reads one kind; the checks it leaves out are listed in skipped_checks.
Explicit check_ids may name either kind, but cannot mix kinds in one run.
Responses
Section titled “ Responses ”OK
Response for Checks.RunDatasetChecks.
object
The resolved checks, by category then check id.
One selected check, as the run pins it.
object
Accounts/{account}/datasets/{dataset}/checks/{check}
The check version pinned.
How it reads an episode.
Selected episodes it will read.
Selected episodes it skips, with skip_reason.
Why it skips them (e.g. no published Task); empty when it skips none.
Episodes selected: after the selection and sample cap.
Episodes of the selection that no selected check can read.
What the run reserves: pipeline-api’s bound for the pinned selection.
object
Credits the run would reserve: on a merged dataset, the sum over its uploads’ runs.
Episodes the model would read: linked to a published task, in a published version, with a camera.
Episodes of the selection it would leave out: not linked to a task, linked to one with no published revision, not in a published version, or without a camera.
The Qualia model tier it would read with (“qualia-vlm-fast”): the request’s, else the template’s default.
Every Qualia model tier this deployment’s VLM check offers, the default first. A request may still name a legacy vendor model id (deprecated).
Accounts/{account}/checks/vlm/runs/{run}: the run; empty with
validate_only.
Checks all or categories reached that the run leaves out, by category
then check id, with why. An id in check_ids that cannot run is refused
instead. The VLM check’s own steps (graph.observe, graph.evaluate) run
only inside it and are left out unlisted.
A check the selection reached that the run leaves out.
object
Accounts/{account}/datasets/{dataset}/checks/{check}
Its active version.
How it reads an episode.
Why the run leaves it out: it cannot run here, or it is of the other kind and runs as its own run.
Example
{ "checks": [ { "kind": "KIND_UNSPECIFIED" } ], "skippedChecks": [ { "kind": "KIND_UNSPECIFIED" } ]}default
Section titled “default ”Default error response
A refusal from the API itself.
object
The gRPC code name in upper case, e.g. NOT_FOUND.
Whether the same call may succeed if retried.
Empty, except when a dataset copy or move is refused: then one google.rpc.ErrorInfo with the refusal’s reason code, its domain and, in metadata, the datasets it names.
object
A refusal from the gateway before the call reaches the API, for example 401 bad or missing API key, 403 key lacks the scope or environment closed, 413 body too large.
object
What went wrong.
Example generated
{ "code": "example", "message": "example", "retryable": true, "details": [ {} ]}