Skip to main content
The tasks client methods are currently in BETA. The API may change without notice. A one-time warning is emitted on first use.
A task is an automated job that either evaluates data on a project or dataset (template_evaluation, code_evaluation) or runs an experiment over a dataset (run_experiment). The Get, Update, Delete, TriggerRun, and ListRuns methods accept either a task name or an ID — when a name is passed, the parent Space (name or ID) is also required so the SDK can resolve the name to a unique ID. GetRun, CancelRun, and WaitForRun take strict run IDs. Task types use different create methods: CreateEvaluationTask for template_evaluation and code_evaluation tasks, CreateRunExperimentTask for run_experiment tasks. An update with no patch fields returns tasks.ErrNoUpdateFields; a WaitForRun that exceeds its timeout returns an error wrapping tasks.ErrWaitTimeout.

List Tasks

List returns a paginated list of tasks. Space, when non-empty, accepts a space name or ID and restricts results to that space. Project and Dataset, when non-empty, accept a name or ID and restrict results to tasks attached to that project or dataset. Signature:
Usage Example:

Get a Task

Get returns a single task, resolving by name or ID. Space is required when Task is a name. Signature:
Usage Example:

Create an Evaluation Task

CreateEvaluationTask creates a new template_evaluation or code_evaluation task and returns it. Exactly one of Project or Dataset must be set (name or ID; Space is required when either is a name). At least one evaluator is required. Dataset-based tasks require at least one entry in ExperimentIDs; SamplingRate and IsContinuous apply only to project-based tasks. Signature:
Usage Example:

Create a Run-Experiment Task

CreateRunExperimentTask creates a new run_experiment task and returns it. Dataset accepts a name or ID; Space is required when Dataset is a name. RunConfiguration must hold exactly one variant — populate it via FromLlmGenerationRunConfig or FromTemplateEvaluationRunConfig. Signature:
Usage Example:

Update a Task

Update updates an existing task and returns it. Task accepts a name or ID; Space is required when Task is a name. The SDK fetches the task first to determine its type: Name applies to all tasks; SamplingRate, IsContinuous, QueryFilter, and Evaluators apply only to evaluation tasks; RunConfiguration applies only to run_experiment tasks. Leave a patch field nil to preserve its current value. A request with no patch fields returns tasks.ErrNoUpdateFields without contacting the server. Signature:
Usage Example:

Delete a Task

Delete irreversibly removes a task and all its associated resources (runs, configurations, etc.). Task accepts a name or ID; Space is required when Task is a name. It returns only an error. Signature:
Usage Example:

Trigger a Run

TriggerRun triggers a new run of a task and returns it (initially in pending status). Task accepts a name or ID; Space is required when Task is a name. The SDK fetches the task first to determine its type: DataStartTime, DataEndTime, MaxSpans, OverrideEvaluations, and ExperimentIDs apply only to evaluation tasks; ExperimentName (required), DatasetVersionID, ExampleIDs, MaxExamples, TracingMetadata, and EvaluationTaskIDs apply only to run_experiment tasks. Signature:
Usage Example:

List Runs

ListRuns returns a paginated list of a task’s runs, newest first. Task accepts a name or ID; Space is required when Task is a name. Signature:
Usage Example:

Get a Run

GetRun returns a single task run with its current status and statistics. Use it to poll a run triggered by TriggerRun (or use WaitForRun). RunID is a strict ID — no name resolution is performed. Signature:
Usage Example:

Cancel a Run

CancelRun cancels a pending or running task run and returns it. RunID is a strict ID. Signature:
Usage Example:

Wait for a Run

WaitForRun polls a task run until it reaches a terminal state (completed, failed, or cancelled) and returns it. It polls every PollInterval (default 5 s) for up to Timeout (default 10 m); on expiry it returns an error wrapping tasks.ErrWaitTimeout. Cancelling ctx stops the wait with ctx’s error. Signature:
Usage Example: