red_teams

Creates, updates, deletes, gets or lists a red_teams resource.

Overview

Name	red_teams
Type	Resource
Id	azure.ai_evaluation.red_teams

Fields

The following fields are returned by SELECT queries:

get

Name	Datatype	Description
id	string	Identifier of the red team. Required.
applicationScenario	string	Application scenario for the red team operation, to generate scenario specific attacks.
attackStrategies	array	List of attack strategies or nested lists of attack strategies. Required.
displayName	string	Display name of the red-team scan.
numTurns	integer	Number of simulation rounds. Required.
outputs	object	Read-only result outputs. Example: { 'evaluationResultId': 'azureai://accounts/{AccountName}/projects/{myproject}/evaluationresults/{name}/versions/{version}', 'logId': 'azureai://accounts/{AccountName}/projects/{myproject}/datasets/{dataset-name}/versions/{dataset-version}' }. Required.
properties	object	Red team's properties. Unlike tags, properties are add-only. Once added, a property cannot be removed.
riskCategories	array	List of risk categories to generate attack objectives for. Required.
simulationOnly	boolean	Simulation-only or Simulation + Evaluation. Default false, if true the scan outputs conversation not evaluation result. Required.
status	string	Status of the red-team. It is set by service and is read-only.
systemData	object	Metadata containing createdBy and modifiedBy information.
tags	object	Red team's tags. Unlike properties, tags are fully mutable.
target	object	Target configuration for the red-team run. Required.

get_jail_break_dataset_with_type

Name	Datatype	Description
value	string

get_attack_objectives

Name	Datatype	Description
Id	string	The unique identifier. Required.
Messages	array	The messages. Required.
Metadata	object	The metadata.
Modality	string	The modality. Required.
Source	array	List of sources. Required.

get_template_parameters_image

Name	Datatype	Description
value	string

list

Name	Datatype	Description
id	string	Identifier of the red team. Required.
applicationScenario	string	Application scenario for the red team operation, to generate scenario specific attacks.
attackStrategies	array	List of attack strategies or nested lists of attack strategies. Required.
displayName	string	Display name of the red-team scan.
numTurns	integer	Number of simulation rounds. Required.
outputs	object	Read-only result outputs. Example: { 'evaluationResultId': 'azureai://accounts/{AccountName}/projects/{myproject}/evaluationresults/{name}/versions/{version}', 'logId': 'azureai://accounts/{AccountName}/projects/{myproject}/datasets/{dataset-name}/versions/{dataset-version}' }. Required.
properties	object	Red team's properties. Unlike tags, properties are add-only. Once added, a property cannot be removed.
riskCategories	array	List of risk categories to generate attack objectives for. Required.
simulationOnly	boolean	Simulation-only or Simulation + Evaluation. Default false, if true the scan outputs conversation not evaluation result. Required.
status	string	Status of the red-team. It is set by service and is read-only.
systemData	object	Metadata containing createdBy and modifiedBy information.
tags	object	Red team's tags. Unlike properties, tags are fully mutable.
target	object	Target configuration for the red-team run. Required.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get	select	name, endpoint		Get a redteam by name.
get_jail_break_dataset_with_type	select	type_name, endpoint		Get the jailbreak dataset with type.
get_attack_objectives	select	riskCategory, endpoint	lang, strategy, targetType	Get the attack objectives.
get_template_parameters_image	select	path, endpoint		Get the template parameters image.
list	select	endpoint	top, skip, maxpagesize	List a redteam by name.
get_jail_break_dataset	exec	endpoint		Get the jailbreak dataset.
get_template_parameters_with_type	exec	type_name, endpoint		Get template parameters with type.
get_template_parameters	exec	endpoint		Get template parameters.
create_run	exec	endpoint, id, numTurns, attackStrategies, simulationOnly, riskCategories, target		Creates a redteam run.
upload_run	exec	endpoint, id		Upload the result to a redteam run.
upload_update_run	exec	name, endpoint, id		Update the uploaded the result to an redteam run.
submit_simulation	exec	endpoint		Submit a request for simulation.
operation_results	exec	operation_id, endpoint		Poll for the operation results.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
endpoint	string	The service endpoint host (no scheme), e.g. myaccount.table.cosmos.azure.com:443 - value of the client endpoint parameter. (default: )
name	string	Name of the redteam run to update. Required.
operation_id	string	Operation ID for the polling operation. Required.
path	string	Image path. Required.
riskCategory	string	Risk category for the attack objectives. Required.
type_name	string	Type for the template parameters. Required.
lang	string	The language for the attack objectives dataset, defaults to 'en'. Default value is None.
maxpagesize	integer
skip	integer	The number of result items to skip. Default value is None.
strategy	string	The strategy. Default value is None.
targetType	string	The target, model/agent. Default value is None.
top	integer	The number of result items to return. Default value is None.

SELECT examples

get

Get a redteam by name.

SELECT
id,
applicationScenario,
attackStrategies,
displayName,
numTurns,
outputs,
properties,
riskCategories,
simulationOnly,
status,
systemData,
tags,
target
FROM azure.ai_evaluation.red_teams
WHERE name = '{{ name }}' -- required
AND endpoint = '{{ endpoint }}' -- required
;

get_jail_break_dataset_with_type

Get the jailbreak dataset with type.

SELECT
value
FROM azure.ai_evaluation.red_teams
WHERE type_name = '{{ type_name }}' -- required
AND endpoint = '{{ endpoint }}' -- required
;

get_attack_objectives

Get the attack objectives.

SELECT
Id,
Messages,
Metadata,
Modality,
Source
FROM azure.ai_evaluation.red_teams
WHERE riskCategory = '{{ riskCategory }}' -- required
AND endpoint = '{{ endpoint }}' -- required
AND lang = '{{ lang }}'
AND strategy = '{{ strategy }}'
AND targetType = '{{ targetType }}'
;

get_template_parameters_image

Get the template parameters image.

SELECT
value
FROM azure.ai_evaluation.red_teams
WHERE path = '{{ path }}' -- required
AND endpoint = '{{ endpoint }}' -- required
;

list

List a redteam by name.

SELECT
id,
applicationScenario,
attackStrategies,
displayName,
numTurns,
outputs,
properties,
riskCategories,
simulationOnly,
status,
systemData,
tags,
target
FROM azure.ai_evaluation.red_teams
WHERE endpoint = '{{ endpoint }}' -- required
AND top = '{{ top }}'
AND skip = '{{ skip }}'
AND maxpagesize = '{{ maxpagesize }}'
;

Lifecycle Methods

get_jail_break_dataset

Get the jailbreak dataset.

EXEC azure.ai_evaluation.red_teams.get_jail_break_dataset 
@endpoint='{{ endpoint }}' --required
;

get_template_parameters_with_type

Get template parameters with type.

EXEC azure.ai_evaluation.red_teams.get_template_parameters_with_type 
@type_name='{{ type_name }}' --required, 
@endpoint='{{ endpoint }}' --required
;

get_template_parameters

Get template parameters.

EXEC azure.ai_evaluation.red_teams.get_template_parameters 
@endpoint='{{ endpoint }}' --required
;

create_run

Creates a redteam run.

EXEC azure.ai_evaluation.red_teams.create_run 
@endpoint='{{ endpoint }}' --required 
@@json=
'{
"id": "{{ id }}", 
"displayName": "{{ displayName }}", 
"numTurns": {{ numTurns }}, 
"attackStrategies": "{{ attackStrategies }}", 
"simulationOnly": {{ simulationOnly }}, 
"riskCategories": "{{ riskCategories }}", 
"applicationScenario": "{{ applicationScenario }}", 
"tags": "{{ tags }}", 
"properties": "{{ properties }}", 
"target": "{{ target }}"
}'
;

upload_run

Upload the result to a redteam run.

EXEC azure.ai_evaluation.red_teams.upload_run 
@endpoint='{{ endpoint }}' --required 
@@json=
'{
"id": "{{ id }}", 
"displayName": "{{ displayName }}", 
"numTurns": {{ numTurns }}, 
"attackStrategies": "{{ attackStrategies }}", 
"simulationOnly": {{ simulationOnly }}, 
"riskCategories": "{{ riskCategories }}", 
"applicationScenario": "{{ applicationScenario }}", 
"tags": "{{ tags }}", 
"properties": "{{ properties }}", 
"status": "{{ status }}", 
"outputs": "{{ outputs }}", 
"target": "{{ target }}"
}'
;

upload_update_run

Update the uploaded the result to an redteam run.

EXEC azure.ai_evaluation.red_teams.upload_update_run 
@name='{{ name }}' --required, 
@endpoint='{{ endpoint }}' --required 
@@json=
'{
"id": "{{ id }}", 
"displayName": "{{ displayName }}", 
"numTurns": {{ numTurns }}, 
"attackStrategies": "{{ attackStrategies }}", 
"simulationOnly": {{ simulationOnly }}, 
"riskCategories": "{{ riskCategories }}", 
"applicationScenario": "{{ applicationScenario }}", 
"tags": "{{ tags }}", 
"properties": "{{ properties }}", 
"status": "{{ status }}", 
"outputs": "{{ outputs }}", 
"target": "{{ target }}"
}'
;

submit_simulation

Submit a request for simulation.

EXEC azure.ai_evaluation.red_teams.submit_simulation 
@endpoint='{{ endpoint }}' --required 
@@json=
'{
"Headers": "{{ Headers }}", 
"Params": "{{ Params }}", 
"TemplateParameters": "{{ TemplateParameters }}", 
"CustomizationParameters": "{{ CustomizationParameters }}", 
"Json": "{{ Json }}", 
"Url": "{{ Url }}", 
"TemplateKey": "{{ TemplateKey }}", 
"SimulationType": "{{ SimulationType }}", 
"IsMicrosoftTenant": {{ IsMicrosoftTenant }}, 
"SubscriptionId": "{{ SubscriptionId }}", 
"ResourceGroupName": "{{ ResourceGroupName }}", 
"WorkspaceName": "{{ WorkspaceName }}"
}'
;

operation_results

Poll for the operation results.

EXEC azure.ai_evaluation.red_teams.operation_results 
@operation_id='{{ operation_id }}' --required, 
@endpoint='{{ endpoint }}' --required
;