datasets

Creates, updates, deletes, gets or lists a datasets resource.

Overview

Name	datasets
Type	Resource
Id	azure.ai_evaluation.datasets

Fields

The following fields are returned by SELECT queries:

get_version

Name	Datatype	Description
id	string	Asset ID, a unique identifier for the asset.
name	string	The name of the resource. Required.
connectionName	string	The Azure Storage Account connection name. Required if startPendingUploadVersion was not called before creating the Dataset.
dataUri	string	URI of the data. Example: https://go.microsoft.com/fwlink/?linkid=2202330 _. Required.
description	string	The asset description text.
isReference	boolean	Indicates if the dataset holds a reference to the storage, or the dataset manages storage itself. If true, the underlying data will not be deleted when the dataset version is deleted.
tags	object	Tag dictionary. Tags can be added, removed, and updated.
type	string	Dataset type. Required. Known values are: "uri_file" and "uri_folder".
version	string	The version of the resource. Required.

list_versions

Name	Datatype	Description
id	string	Asset ID, a unique identifier for the asset.
name	string	The name of the resource. Required.
connectionName	string	The Azure Storage Account connection name. Required if startPendingUploadVersion was not called before creating the Dataset.
dataUri	string	URI of the data. Example: https://go.microsoft.com/fwlink/?linkid=2202330 _. Required.
description	string	The asset description text.
isReference	boolean	Indicates if the dataset holds a reference to the storage, or the dataset manages storage itself. If true, the underlying data will not be deleted when the dataset version is deleted.
tags	object	Tag dictionary. Tags can be added, removed, and updated.
type	string	Dataset type. Required. Known values are: "uri_file" and "uri_folder".
version	string	The version of the resource. Required.

list_latest

Name	Datatype	Description
id	string	Asset ID, a unique identifier for the asset.
name	string	The name of the resource. Required.
connectionName	string	The Azure Storage Account connection name. Required if startPendingUploadVersion was not called before creating the Dataset.
dataUri	string	URI of the data. Example: https://go.microsoft.com/fwlink/?linkid=2202330 _. Required.
description	string	The asset description text.
isReference	boolean	Indicates if the dataset holds a reference to the storage, or the dataset manages storage itself. If true, the underlying data will not be deleted when the dataset version is deleted.
tags	object	Tag dictionary. Tags can be added, removed, and updated.
type	string	Dataset type. Required. Known values are: "uri_file" and "uri_folder".
version	string	The version of the resource. Required.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get_version	select	name, version, endpoint		Get the specific version of the DatasetVersion. The service returns 404 Not Found error if the DatasetVersion does not exist.
list_versions	select	name, endpoint		List all versions of the given DatasetVersion.
list_latest	select	endpoint		List the latest version of each DatasetVersion.
delete_version	exec	name, version, endpoint		Delete the specific version of the DatasetVersion. The service returns 204 No Content if the DatasetVersion was deleted successfully or if the DatasetVersion does not exist.
create_or_update_version	exec	name, version, endpoint, dataUri, type		Create a new or update an existing DatasetVersion with the given version id.
get_credentials	exec	name, version, endpoint		Get the SAS credential to access the storage account associated with a Dataset version.
start_pending_upload_version	exec	name, version, endpoint, pendingUploadType		Start a new or get an existing pending upload of a dataset for a specific version.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
endpoint	string	The service endpoint host (no scheme), e.g. myaccount.table.cosmos.azure.com:443 - value of the client endpoint parameter. (default: )
name	string	The name of the resource. Required.
version	string	The specific version id of the DatasetVersion to operate on. Required.

SELECT examples

get_version

Get the specific version of the DatasetVersion. The service returns 404 Not Found error if the DatasetVersion does not exist.

SELECT
id,
name,
connectionName,
dataUri,
description,
isReference,
tags,
type,
version
FROM azure.ai_evaluation.datasets
WHERE name = '{{ name }}' -- required
AND version = '{{ version }}' -- required
AND endpoint = '{{ endpoint }}' -- required
;

list_versions

List all versions of the given DatasetVersion.

SELECT
id,
name,
connectionName,
dataUri,
description,
isReference,
tags,
type,
version
FROM azure.ai_evaluation.datasets
WHERE name = '{{ name }}' -- required
AND endpoint = '{{ endpoint }}' -- required
;

list_latest

List the latest version of each DatasetVersion.

SELECT
id,
name,
connectionName,
dataUri,
description,
isReference,
tags,
type,
version
FROM azure.ai_evaluation.datasets
WHERE endpoint = '{{ endpoint }}' -- required
;

Lifecycle Methods

delete_version

Delete the specific version of the DatasetVersion. The service returns 204 No Content if the DatasetVersion was deleted successfully or if the DatasetVersion does not exist.

EXEC azure.ai_evaluation.datasets.delete_version 
@name='{{ name }}' --required, 
@version='{{ version }}' --required, 
@endpoint='{{ endpoint }}' --required
;

create_or_update_version

Create a new or update an existing DatasetVersion with the given version id.

EXEC azure.ai_evaluation.datasets.create_or_update_version 
@name='{{ name }}' --required, 
@version='{{ version }}' --required, 
@endpoint='{{ endpoint }}' --required 
@@json=
'{
"dataUri": "{{ dataUri }}", 
"type": "{{ type }}", 
"connectionName": "{{ connectionName }}", 
"description": "{{ description }}", 
"tags": "{{ tags }}"
}'
;

get_credentials

Get the SAS credential to access the storage account associated with a Dataset version.

EXEC azure.ai_evaluation.datasets.get_credentials 
@name='{{ name }}' --required, 
@version='{{ version }}' --required, 
@endpoint='{{ endpoint }}' --required
;

start_pending_upload_version

Start a new or get an existing pending upload of a dataset for a specific version.

EXEC azure.ai_evaluation.datasets.start_pending_upload_version 
@name='{{ name }}' --required, 
@version='{{ version }}' --required, 
@endpoint='{{ endpoint }}' --required 
@@json=
'{
"pendingUploadId": "{{ pendingUploadId }}", 
"connectionName": "{{ connectionName }}", 
"pendingUploadType": "{{ pendingUploadType }}"
}'
;