Evaluators
Evaluator definitions — the metrics and judges used in evaluation runs.
List Evaluator Catalog Types
List the JSON schema types the evaluator catalog understands.
List Evaluator Catalog Templates
List evaluator templates from the catalog.
Fetch Evaluator Catalog Template
Fetch one evaluator template by key.
List Evaluator Catalog Presets
List presets defined against one evaluator template.
Fetch Evaluator Catalog Preset
Fetch one evaluator preset by template and preset key.
Create Evaluator
Create an evaluator artifact, its first variant, and its initial revision.
Fetch Evaluator
Fetch an evaluator artifact by id.
Edit Evaluator
Edit an evaluator artifact's metadata.
Archive Evaluator
Soft-delete an evaluator artifact.
Unarchive Evaluator
Restore a soft-deleted evaluator artifact.
Query Evaluators
Query evaluator artifacts with filters and pagination.
Create Evaluator Variant
Create a new variant on an existing evaluator.
Fetch Evaluator Variant
Fetch an evaluator variant by id.
Edit Evaluator Variant
Edit a variant's metadata.
Archive Evaluator Variant
Soft-delete an evaluator variant.
Unarchive Evaluator Variant
Restore a soft-deleted evaluator variant.
Query Evaluator Variants
Query evaluator variants with filters, reference scoping, and pagination.
Fork Evaluator Variant
Fork an evaluator variant into a new variant.
Retrieve Evaluator Revision
Retrieve one evaluator revision, either directly or via an environment key.
Deploy Evaluator Revision
Pin an evaluator revision into an environment revision under a key.
Create Evaluator Revision
Create and commit the initial revision for an evaluator variant.
Fetch Evaluator Revision
Fetch a specific evaluator revision by id.
Edit Evaluator Revision
Edit a revision's metadata.
Archive Evaluator Revision
Soft-delete an evaluator revision.
Unarchive Evaluator Revision
Restore a soft-deleted evaluator revision.
Query Evaluator Revisions
Query evaluator revisions with filters, reference scoping, and pagination.
Commit Evaluator Revision
Commit a new revision on an evaluator variant.
Log Evaluator Revisions
List the revision log of an evaluator variant.
Resolve Evaluator Revision
Resolve embedded references on an evaluator revision's `data`.
Create Simple Evaluator
Create an evaluator via the simple surface.
List Evaluator Templates
List the legacy built-in evaluator templates.
Fetch Simple Evaluator
Fetch one evaluator via the simple surface.
Edit Simple Evaluator
Edit an evaluator via the simple surface.
Archive Simple Evaluator
Soft-delete an evaluator via the simple surface.
Unarchive Simple Evaluator
Restore a soft-deleted evaluator via the simple surface.
Query Simple Evaluators
Query evaluators via the simple surface with filters and pagination.