evaluators

Creates, updates, deletes, gets or lists an evaluators resource.

Overview

Name	evaluators
Type	Resource
Id	azure.ai_evaluation.evaluators

Fields

The following fields are returned by SELECT queries:

get_evaluator_version

Name	Datatype	Description
id	string	Asset ID, a unique identifier for the asset.
name	string	The name of the resource. Required.
display_name	string	Display Name for evaluator. It helps to find the evaluator easily in AI Foundry. It does not need to be unique.
categories	array	The categories of the evaluator. Required.
created_at	integer	Creation date/time of the evaluator. Required.
created_by	string	Creator of the evaluator. Required.
definition	object	Definition of the evaluator. Required.
description	string	The asset description text.
evaluator_type	string	The type of the evaluator. Required. Known values are: "builtin" and "custom". (builtin, custom)
metadata	object	Metadata about the evaluator.
modified_at	integer	Last modified date/time of the evaluator. Required.
tags	object	Tag dictionary. Tags can be added, removed, and updated.
version	string	The version of the resource. Required.

list_versions

Name	Datatype	Description
id	string	Asset ID, a unique identifier for the asset.
name	string	The name of the resource. Required.
display_name	string	Display Name for evaluator. It helps to find the evaluator easily in AI Foundry. It does not need to be unique.
categories	array	The categories of the evaluator. Required.
created_at	integer	Creation date/time of the evaluator. Required.
created_by	string	Creator of the evaluator. Required.
definition	object	Definition of the evaluator. Required.
description	string	The asset description text.
evaluator_type	string	The type of the evaluator. Required. Known values are: "builtin" and "custom". (builtin, custom)
metadata	object	Metadata about the evaluator.
modified_at	integer	Last modified date/time of the evaluator. Required.
tags	object	Tag dictionary. Tags can be added, removed, and updated.
version	string	The version of the resource. Required.

list_latest_versions

Name	Datatype	Description
id	string	Asset ID, a unique identifier for the asset.
name	string	The name of the resource. Required.
display_name	string	Display Name for evaluator. It helps to find the evaluator easily in AI Foundry. It does not need to be unique.
categories	array	The categories of the evaluator. Required.
created_at	integer	Creation date/time of the evaluator. Required.
created_by	string	Creator of the evaluator. Required.
definition	object	Definition of the evaluator. Required.
description	string	The asset description text.
evaluator_type	string	The type of the evaluator. Required. Known values are: "builtin" and "custom". (builtin, custom)
metadata	object	Metadata about the evaluator.
modified_at	integer	Last modified date/time of the evaluator. Required.
tags	object	Tag dictionary. Tags can be added, removed, and updated.
version	string	The version of the resource. Required.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get_evaluator_version	select	name, version, endpoint		Get the specific version of the EvaluatorVersion. The service returns 404 Not Found error if the EvaluatorVersion does not exist.
list_versions	select	name, endpoint	type, limit	List all versions of the given evaluator.
list_latest_versions	select	endpoint	type, limit	List the latest version of each evaluator.
create_evaluator_version	exec	name, endpoint		Create a new EvaluatorVersion with auto incremented version id.
delete_evaluator_version	exec	name, version, endpoint		Delete the specific version of the EvaluatorVersion. The service returns 204 No Content if the EvaluatorVersion was deleted successfully or if the EvaluatorVersion does not exist.
update_evaluator_version	exec	name, version, endpoint		Update an existing EvaluatorVersion with the given version id.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
endpoint	string	The service endpoint host (no scheme), e.g. myaccount.table.cosmos.azure.com:443 - value of the client endpoint parameter. (default: )
name	string	The name of the resource. Required.
version	string	The version of the EvaluatorVersion to update. Required.
limit	integer	A limit on the number of objects to be returned. Limit can range between 1 and 100, and the default is 20. Default value is None.
type	string	Filter evaluators by type. Possible values: 'all', 'custom', 'builtin'. Is one of the following types: Literal["builtin"], Literal["custom"], Literal["all"], str Default value is None.

SELECT examples

get_evaluator_version

Get the specific version of the EvaluatorVersion. The service returns 404 Not Found error if the EvaluatorVersion does not exist.

SELECT
id,
name,
display_name,
categories,
created_at,
created_by,
definition,
description,
evaluator_type,
metadata,
modified_at,
tags,
version
FROM azure.ai_evaluation.evaluators
WHERE name = '{{ name }}' -- required
AND version = '{{ version }}' -- required
AND endpoint = '{{ endpoint }}' -- required
;

list_versions

List all versions of the given evaluator.

SELECT
id,
name,
display_name,
categories,
created_at,
created_by,
definition,
description,
evaluator_type,
metadata,
modified_at,
tags,
version
FROM azure.ai_evaluation.evaluators
WHERE name = '{{ name }}' -- required
AND endpoint = '{{ endpoint }}' -- required
AND type = '{{ type }}'
AND limit = '{{ limit }}'
;

list_latest_versions

List the latest version of each evaluator.

SELECT
id,
name,
display_name,
categories,
created_at,
created_by,
definition,
description,
evaluator_type,
metadata,
modified_at,
tags,
version
FROM azure.ai_evaluation.evaluators
WHERE endpoint = '{{ endpoint }}' -- required
AND type = '{{ type }}'
AND limit = '{{ limit }}'
;

Lifecycle Methods

create_evaluator_version

Create a new EvaluatorVersion with auto incremented version id.

EXEC azure.ai_evaluation.evaluators.create_evaluator_version 
@name='{{ name }}' --required, 
@endpoint='{{ endpoint }}' --required
;

delete_evaluator_version

Delete the specific version of the EvaluatorVersion. The service returns 204 No Content if the EvaluatorVersion was deleted successfully or if the EvaluatorVersion does not exist.

EXEC azure.ai_evaluation.evaluators.delete_evaluator_version 
@name='{{ name }}' --required, 
@version='{{ version }}' --required, 
@endpoint='{{ endpoint }}' --required
;

update_evaluator_version

Update an existing EvaluatorVersion with the given version id.

EXEC azure.ai_evaluation.evaluators.update_evaluator_version 
@name='{{ name }}' --required, 
@version='{{ version }}' --required, 
@endpoint='{{ endpoint }}' --required
;