Create an Agent Observability annotation queue
ADVERTENCIANote: This endpoint is in preview and is subject to change.
If you have any feedback, contact Datadog support.
POST
https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queueshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queueshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queueshttps://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queueshttps://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queueshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queueshttps://api.uk1.datadoghq.com/api/v2/llm-obs/v1/annotation-queueshttps://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queueshttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queuesDescripción general
Create an annotation queue. The name and project_id fields are required.
An optional annotation_schema can be provided to define the labels for the queue.
Optional access-control fields can restrict annotation access to reviewers or assignees.
Fields such as created_by, owned_by, created_at, modified_by,
and modified_at are inferred by the backend.
Solicitud
Body Data (required)
Create annotation queue payload.
Campo
Type
Descripción
data [required]object
Data object for creating an Agent Observability annotation queue.
attributes [required]object
Attributes for creating an Agent Observability annotation queue.
annotation_schemaobject
Schema defining the labels for an annotation queue.
label_schemas [required][object]
List of label schema definitions.
descriptionstring
Description of the label.
has_assessmentboolean
Whether this label includes an assessment field.
has_reasoningboolean
Whether this label includes a reasoning field.
idstring
Unique identifier of the label schema. Assigned by the server if not provided.
is_assessmentboolean
Whether the boolean label represents an assessment. Requires has_assessment to be true.
is_integerboolean
Whether score values must be integers. Applicable to score-type labels.
is_requiredboolean
Whether this label is required for an annotation.
maxdouble
Maximum value for score-type labels.
mindouble
Minimum value for score-type labels.
name [required]string
Name of the label. Must match the pattern ^[a-zA-Z0-9_-]+$ and be unique within the queue.
type [required]enum
Type of a label in an annotation queue label schema.
Allowed enum values:
score, categorical, boolean, textvalues[string]
Allowed values for categorical-type labels. Must contain at least one non-empty, unique value.
descriptionstring
Description of the annotation queue.
name [required]string
Name of the annotation queue.
project_id [required]string
Identifier of the project this queue belongs to.
restrict_to_assigneesboolean
Whether annotation access is restricted to assigned users.
restrict_to_reviewersboolean
Whether annotation access is restricted to queue reviewers.
reviewer_emails[string]
Email addresses of reviewers who can access the annotation queue.
type [required]enum
Resource type of an Agent Observability annotation queue.
Allowed enum values:
queues{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"description": "Rating of the response quality.",
"has_assessment": false,
"has_reasoning": false,
"id": "abc-123",
"is_assessment": false,
"is_integer": false,
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score",
"values": [
"good",
"bad",
"neutral"
]
}
]
},
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"type": "queues"
}
}Respuesta
Created
Response containing a single Agent Observability annotation queue.
Campo
Type
Descripción
data [required]object
Data object for an Agent Observability annotation queue.
attributes [required]object
Attributes of an Agent Observability annotation queue.
annotation_schemaobject
Schema defining the labels for an annotation queue.
label_schemas [required][object]
List of label schema definitions.
descriptionstring
Description of the label.
has_assessmentboolean
Whether this label includes an assessment field.
has_reasoningboolean
Whether this label includes a reasoning field.
idstring
Unique identifier of the label schema. Assigned by the server if not provided.
is_assessmentboolean
Whether the boolean label represents an assessment. Requires has_assessment to be true.
is_integerboolean
Whether score values must be integers. Applicable to score-type labels.
is_requiredboolean
Whether this label is required for an annotation.
maxdouble
Maximum value for score-type labels.
mindouble
Minimum value for score-type labels.
name [required]string
Name of the label. Must match the pattern ^[a-zA-Z0-9_-]+$ and be unique within the queue.
type [required]enum
Type of a label in an annotation queue label schema.
Allowed enum values:
score, categorical, boolean, textvalues[string]
Allowed values for categorical-type labels. Must contain at least one non-empty, unique value.
can_manage_access [required]boolean
Whether the current caller can manage access for the annotation queue.
created_at [required]date-time
Timestamp when the queue was created.
created_by [required]string
Identifier of the user who created the queue.
description [required]string
Description of the annotation queue.
modified_at [required]date-time
Timestamp when the queue was last modified.
modified_by [required]string
Identifier of the user who last modified the queue.
name [required]string
Name of the annotation queue.
owned_by [required]string
Identifier of the user who owns the queue.
project_id [required]string
Identifier of the project this queue belongs to.
restrict_to_assignees [required]boolean
Whether annotation access is restricted to assigned users.
restrict_to_reviewers [required]boolean
Whether annotation access is restricted to queue reviewers.
reviewer_emails[string]
Email addresses of reviewers for the annotation queue. Returned only when the caller can manage queue access.
id [required]string
Unique identifier of the annotation queue.
type [required]enum
Resource type of an Agent Observability annotation queue.
Allowed enum values:
queues{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"description": "Rating of the response quality.",
"has_assessment": false,
"has_reasoning": false,
"id": "abc-123",
"is_assessment": false,
"is_integer": false,
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score",
"values": [
"good",
"bad",
"neutral"
]
}
]
},
"can_manage_access": true,
"created_at": "2024-01-15T10:30:00Z",
"created_by": "00000000-0000-0000-0000-000000000002",
"description": "Queue for annotating customer support traces",
"modified_at": "2024-01-15T10:30:00Z",
"modified_by": "00000000-0000-0000-0000-000000000002",
"name": "My annotation queue",
"owned_by": "00000000-0000-0000-0000-000000000002",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"id": "00000000-0000-0000-0000-000000000001",
"type": "queues"
}
}Bad Request
API error response.
Campo
Type
Descripción
errors [required][object]
A list of errors.
detailstring
A human-readable explanation specific to this occurrence of the error.
metaobject
Non-standard meta-information about the error
sourceobject
References to the source of the error.
headerstring
A string indicating the name of a single request header which caused the error.
parameterstring
A string indicating which URI query parameter caused the error.
pointerstring
A JSON pointer to the value in the request document that caused the error.
statusstring
Status code of the response.
titlestring
Short human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Unauthorized
API error response.
Campo
Type
Descripción
errors [required][object]
A list of errors.
detailstring
A human-readable explanation specific to this occurrence of the error.
metaobject
Non-standard meta-information about the error
sourceobject
References to the source of the error.
headerstring
A string indicating the name of a single request header which caused the error.
parameterstring
A string indicating which URI query parameter caused the error.
pointerstring
A JSON pointer to the value in the request document that caused the error.
statusstring
Status code of the response.
titlestring
Short human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Forbidden
API error response.
Campo
Type
Descripción
errors [required][object]
A list of errors.
detailstring
A human-readable explanation specific to this occurrence of the error.
metaobject
Non-standard meta-information about the error
sourceobject
References to the source of the error.
headerstring
A string indicating the name of a single request header which caused the error.
parameterstring
A string indicating which URI query parameter caused the error.
pointerstring
A JSON pointer to the value in the request document that caused the error.
statusstring
Status code of the response.
titlestring
Short human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Too many requests
API error response.
Campo
Type
Descripción
errors [required][string]
A list of errors.
{
"errors": [
"Bad Request"
]
}Ejemplo de código
Create an Agent Observability annotation queue
## default
# Set your Datadog site and credentials
export DD_SITE="datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF
## Create queue with access controls
# Set your Datadog site and credentials
export DD_SITE="datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "Restricted annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"type": "queues"
}
}
EOF
## Create queue with annotation schema
# Set your Datadog site and credentials
export DD_SITE="datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
},
{
"name": "sentiment",
"type": "categorical",
"values": [
"positive",
"negative",
"neutral"
]
}
]
},
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF## default
# Set your Datadog site and credentials
export DD_SITE="us3.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF
## Create queue with access controls
# Set your Datadog site and credentials
export DD_SITE="us3.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "Restricted annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"type": "queues"
}
}
EOF
## Create queue with annotation schema
# Set your Datadog site and credentials
export DD_SITE="us3.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
},
{
"name": "sentiment",
"type": "categorical",
"values": [
"positive",
"negative",
"neutral"
]
}
]
},
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF## default
# Set your Datadog site and credentials
export DD_SITE="us5.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF
## Create queue with access controls
# Set your Datadog site and credentials
export DD_SITE="us5.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "Restricted annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"type": "queues"
}
}
EOF
## Create queue with annotation schema
# Set your Datadog site and credentials
export DD_SITE="us5.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
},
{
"name": "sentiment",
"type": "categorical",
"values": [
"positive",
"negative",
"neutral"
]
}
]
},
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF## default
# Set your Datadog site and credentials
export DD_SITE="datadoghq.eu"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF
## Create queue with access controls
# Set your Datadog site and credentials
export DD_SITE="datadoghq.eu"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "Restricted annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"type": "queues"
}
}
EOF
## Create queue with annotation schema
# Set your Datadog site and credentials
export DD_SITE="datadoghq.eu"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
},
{
"name": "sentiment",
"type": "categorical",
"values": [
"positive",
"negative",
"neutral"
]
}
]
},
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF## default
# Set your Datadog site and credentials
export DD_SITE="ap1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF
## Create queue with access controls
# Set your Datadog site and credentials
export DD_SITE="ap1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "Restricted annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"type": "queues"
}
}
EOF
## Create queue with annotation schema
# Set your Datadog site and credentials
export DD_SITE="ap1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
},
{
"name": "sentiment",
"type": "categorical",
"values": [
"positive",
"negative",
"neutral"
]
}
]
},
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF## default
# Set your Datadog site and credentials
export DD_SITE="ap2.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF
## Create queue with access controls
# Set your Datadog site and credentials
export DD_SITE="ap2.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "Restricted annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"type": "queues"
}
}
EOF
## Create queue with annotation schema
# Set your Datadog site and credentials
export DD_SITE="ap2.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
},
{
"name": "sentiment",
"type": "categorical",
"values": [
"positive",
"negative",
"neutral"
]
}
]
},
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF## default
# Set your Datadog site and credentials
export DD_SITE="uk1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF
## Create queue with access controls
# Set your Datadog site and credentials
export DD_SITE="uk1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "Restricted annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"type": "queues"
}
}
EOF
## Create queue with annotation schema
# Set your Datadog site and credentials
export DD_SITE="uk1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
},
{
"name": "sentiment",
"type": "categorical",
"values": [
"positive",
"negative",
"neutral"
]
}
]
},
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF## default
# Set your Datadog site and credentials
export DD_SITE="ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF
## Create queue with access controls
# Set your Datadog site and credentials
export DD_SITE="ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "Restricted annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"type": "queues"
}
}
EOF
## Create queue with annotation schema
# Set your Datadog site and credentials
export DD_SITE="ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
},
{
"name": "sentiment",
"type": "categorical",
"values": [
"positive",
"negative",
"neutral"
]
}
]
},
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF## default
# Set your Datadog site and credentials
export DD_SITE="us2.ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOF
## Create queue with access controls
# Set your Datadog site and credentials
export DD_SITE="us2.ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"description": "Queue for annotating customer support traces",
"name": "Restricted annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002",
"restrict_to_assignees": false,
"restrict_to_reviewers": true,
"reviewer_emails": [
"reviewer@example.com"
]
},
"type": "queues"
}
}
EOF
## Create queue with annotation schema
# Set your Datadog site and credentials
export DD_SITE="us2.ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
},
{
"name": "sentiment",
"type": "categorical",
"values": [
"positive",
"negative",
"neutral"
]
}
]
},
"description": "Queue for annotating customer support traces",
"name": "My annotation queue",
"project_id": "00000000-0000-0000-0000-000000000002"
},
"type": "queues"
}
}
EOFCreate an Agent Observability annotation queue
// Create an Agent Observability annotation queue returns "Created" response
package main
import (
"context"
"encoding/json"
"fmt"
"os"
"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)
func main() {
body := datadogV2.LLMObsAnnotationQueueRequest{
Data: datadogV2.LLMObsAnnotationQueueDataRequest{
Attributes: datadogV2.LLMObsAnnotationQueueDataAttributesRequest{
AnnotationSchema: &datadogV2.LLMObsAnnotationSchema{
LabelSchemas: []datadogV2.LLMObsLabelSchema{
{
Description: datadog.PtrString("Rating of the response quality."),
HasAssessment: datadog.PtrBool(false),
HasReasoning: datadog.PtrBool(false),
Id: datadog.PtrString("abc-123"),
IsAssessment: datadog.PtrBool(false),
IsInteger: datadog.PtrBool(false),
IsRequired: datadog.PtrBool(true),
Max: datadog.PtrFloat64(5.0),
Min: datadog.PtrFloat64(0.0),
Name: "quality",
Type: datadogV2.LLMOBSLABELSCHEMATYPE_SCORE,
Values: []string{
"good",
"bad",
"neutral",
},
},
},
},
Description: datadog.PtrString("Queue for annotating customer support traces"),
Name: "My annotation queue",
ProjectId: "00000000-0000-0000-0000-000000000002",
RestrictToAssignees: datadog.PtrBool(false),
RestrictToReviewers: datadog.PtrBool(true),
ReviewerEmails: []string{
"reviewer@example.com",
},
},
Type: datadogV2.LLMOBSANNOTATIONQUEUETYPE_QUEUES,
},
}
ctx := datadog.NewDefaultContext(context.Background())
ctx = context.WithValue(ctx, datadog.ContextAccessToken, os.Getenv("DD_BEARER_TOKEN"))
configuration := datadog.NewConfiguration()
configuration.SetUnstableOperationEnabled("v2.CreateLLMObsAnnotationQueue", true)
apiClient := datadog.NewAPIClient(configuration)
api := datadogV2.NewAgentObservabilityApi(apiClient)
resp, r, err := api.CreateLLMObsAnnotationQueue(ctx, body)
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `AgentObservabilityApi.CreateLLMObsAnnotationQueue`: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
responseContent, _ := json.MarshalIndent(resp, "", " ")
fmt.Fprintf(os.Stdout, "Response from `AgentObservabilityApi.CreateLLMObsAnnotationQueue`:\n%s\n", responseContent)
}
Instructions
First install the library and its dependencies and then save the example to main.go and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"Create an Agent Observability annotation queue
// Create an Agent Observability annotation queue returns "Created" response
import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.AgentObservabilityApi;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueDataAttributesRequest;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueDataRequest;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueRequest;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueResponse;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueType;
import com.datadog.api.client.v2.model.LLMObsAnnotationSchema;
import com.datadog.api.client.v2.model.LLMObsLabelSchema;
import com.datadog.api.client.v2.model.LLMObsLabelSchemaType;
import java.util.Arrays;
import java.util.Collections;
public class Example {
public static void main(String[] args) {
ApiClient defaultClient = ApiClient.getDefaultApiClient();
defaultClient.setAccessToken(System.getenv("DD_BEARER_TOKEN"));
defaultClient.setUnstableOperationEnabled("v2.createLLMObsAnnotationQueue", true);
AgentObservabilityApi apiInstance = new AgentObservabilityApi(defaultClient);
LLMObsAnnotationQueueRequest body =
new LLMObsAnnotationQueueRequest()
.data(
new LLMObsAnnotationQueueDataRequest()
.attributes(
new LLMObsAnnotationQueueDataAttributesRequest()
.annotationSchema(
new LLMObsAnnotationSchema()
.labelSchemas(
Collections.singletonList(
new LLMObsLabelSchema()
.description("Rating of the response quality.")
.hasAssessment(false)
.hasReasoning(false)
.id("abc-123")
.isAssessment(false)
.isInteger(false)
.isRequired(true)
.max(5.0)
.name("quality")
.type(LLMObsLabelSchemaType.SCORE)
.values(Arrays.asList("good", "bad", "neutral")))))
.description("Queue for annotating customer support traces")
.name("My annotation queue")
.projectId("00000000-0000-0000-0000-000000000002")
.restrictToAssignees(false)
.restrictToReviewers(true)
.reviewerEmails(Collections.singletonList("reviewer@example.com")))
.type(LLMObsAnnotationQueueType.QUEUES));
try {
LLMObsAnnotationQueueResponse result = apiInstance.createLLMObsAnnotationQueue(body);
System.out.println(result);
} catch (ApiException e) {
System.err.println(
"Exception when calling AgentObservabilityApi#createLLMObsAnnotationQueue");
System.err.println("Status code: " + e.getCode());
System.err.println("Reason: " + e.getResponseBody());
System.err.println("Response headers: " + e.getResponseHeaders());
e.printStackTrace();
}
}
}
Instructions
First install the library and its dependencies and then save the example to Example.java and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"Create an Agent Observability annotation queue
"""
Create an Agent Observability annotation queue returns "Created" response
"""
from os import environ
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.agent_observability_api import AgentObservabilityApi
from datadog_api_client.v2.model.llm_obs_annotation_queue_data_attributes_request import (
LLMObsAnnotationQueueDataAttributesRequest,
)
from datadog_api_client.v2.model.llm_obs_annotation_queue_data_request import LLMObsAnnotationQueueDataRequest
from datadog_api_client.v2.model.llm_obs_annotation_queue_request import LLMObsAnnotationQueueRequest
from datadog_api_client.v2.model.llm_obs_annotation_queue_type import LLMObsAnnotationQueueType
from datadog_api_client.v2.model.llm_obs_annotation_schema import LLMObsAnnotationSchema
from datadog_api_client.v2.model.llm_obs_label_schema import LLMObsLabelSchema
from datadog_api_client.v2.model.llm_obs_label_schema_type import LLMObsLabelSchemaType
body = LLMObsAnnotationQueueRequest(
data=LLMObsAnnotationQueueDataRequest(
attributes=LLMObsAnnotationQueueDataAttributesRequest(
annotation_schema=LLMObsAnnotationSchema(
label_schemas=[
LLMObsLabelSchema(
description="Rating of the response quality.",
has_assessment=False,
has_reasoning=False,
id="abc-123",
is_assessment=False,
is_integer=False,
is_required=True,
max=5.0,
min=0.0,
name="quality",
type=LLMObsLabelSchemaType.SCORE,
values=[
"good",
"bad",
"neutral",
],
),
],
),
description="Queue for annotating customer support traces",
name="My annotation queue",
project_id="00000000-0000-0000-0000-000000000002",
restrict_to_assignees=False,
restrict_to_reviewers=True,
reviewer_emails=[
"reviewer@example.com",
],
),
type=LLMObsAnnotationQueueType.QUEUES,
),
)
configuration = Configuration()
configuration.access_token = environ["DD_BEARER_TOKEN"]
configuration.unstable_operations["create_llm_obs_annotation_queue"] = True
with ApiClient(configuration) as api_client:
api_instance = AgentObservabilityApi(api_client)
response = api_instance.create_llm_obs_annotation_queue(body=body)
print(response)
Instructions
First install the library and its dependencies and then save the example to example.py and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"Create an Agent Observability annotation queue
# Create an Agent Observability annotation queue returns "Created" response
require "datadog_api_client"
DatadogAPIClient.configure do |config|
config.access_token = ENV["DD_BEARER_TOKEN"]
config.unstable_operations["v2.create_llm_obs_annotation_queue".to_sym] = true
end
api_instance = DatadogAPIClient::V2::AgentObservabilityAPI.new
body = DatadogAPIClient::V2::LLMObsAnnotationQueueRequest.new({
data: DatadogAPIClient::V2::LLMObsAnnotationQueueDataRequest.new({
attributes: DatadogAPIClient::V2::LLMObsAnnotationQueueDataAttributesRequest.new({
annotation_schema: DatadogAPIClient::V2::LLMObsAnnotationSchema.new({
label_schemas: [
DatadogAPIClient::V2::LLMObsLabelSchema.new({
description: "Rating of the response quality.",
has_assessment: false,
has_reasoning: false,
id: "abc-123",
is_assessment: false,
is_integer: false,
is_required: true,
max: 5.0,
min: 0.0,
name: "quality",
type: DatadogAPIClient::V2::LLMObsLabelSchemaType::SCORE,
values: [
"good",
"bad",
"neutral",
],
}),
],
}),
description: "Queue for annotating customer support traces",
name: "My annotation queue",
project_id: "00000000-0000-0000-0000-000000000002",
restrict_to_assignees: false,
restrict_to_reviewers: true,
reviewer_emails: [
"reviewer@example.com",
],
}),
type: DatadogAPIClient::V2::LLMObsAnnotationQueueType::QUEUES,
}),
})
p api_instance.create_llm_obs_annotation_queue(body)
Instructions
First install the library and its dependencies and then save the example to example.rb and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"Create an Agent Observability annotation queue
// Create an Agent Observability annotation queue returns "Created" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_agent_observability::AgentObservabilityAPI;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueDataAttributesRequest;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueDataRequest;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueRequest;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueType;
use datadog_api_client::datadogV2::model::LLMObsAnnotationSchema;
use datadog_api_client::datadogV2::model::LLMObsLabelSchema;
use datadog_api_client::datadogV2::model::LLMObsLabelSchemaType;
#[tokio::main]
async fn main() {
let body = LLMObsAnnotationQueueRequest::new(LLMObsAnnotationQueueDataRequest::new(
LLMObsAnnotationQueueDataAttributesRequest::new(
"My annotation queue".to_string(),
"00000000-0000-0000-0000-000000000002".to_string(),
)
.annotation_schema(LLMObsAnnotationSchema::new(vec![LLMObsLabelSchema::new(
"quality".to_string(),
LLMObsLabelSchemaType::SCORE,
)
.description("Rating of the response quality.".to_string())
.has_assessment(false)
.has_reasoning(false)
.id("abc-123".to_string())
.is_assessment(false)
.is_integer(false)
.is_required(true)
.max(5.0 as f64)
.min(0.0 as f64)
.values(vec![
"good".to_string(),
"bad".to_string(),
"neutral".to_string(),
])]))
.description("Queue for annotating customer support traces".to_string())
.restrict_to_assignees(false)
.restrict_to_reviewers(true)
.reviewer_emails(vec!["reviewer@example.com".to_string()]),
LLMObsAnnotationQueueType::QUEUES,
));
let mut configuration = datadog::Configuration::new();
configuration.set_unstable_operation_enabled("v2.create_llm_obs_annotation_queue", true);
let api = AgentObservabilityAPI::with_config(configuration);
let resp = api.create_llm_obs_annotation_queue(body).await;
if let Ok(value) = resp {
println!("{:#?}", value);
} else {
println!("{:#?}", resp.unwrap_err());
}
}
Instructions
First install the library and its dependencies and then save the example to src/main.rs and run following commands:
DD_SITE="datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us3.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us5.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="datadoghq.eu" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ap1.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ap2.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="uk1.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runCreate an Agent Observability annotation queue
/**
* Create an Agent Observability annotation queue returns "Created" response
*/
import { client, v2 } from "@datadog/datadog-api-client";
const configuration = client.createConfiguration({
authMethods: {
AuthZ: {
accessToken: process.env.DD_BEARER_TOKEN as string,
},
},
});
configuration.unstableOperations["v2.createLLMObsAnnotationQueue"] = true;
const apiInstance = new v2.AgentObservabilityApi(configuration);
const params: v2.AgentObservabilityApiCreateLLMObsAnnotationQueueRequest = {
body: {
data: {
attributes: {
annotationSchema: {
labelSchemas: [
{
description: "Rating of the response quality.",
hasAssessment: false,
hasReasoning: false,
id: "abc-123",
isAssessment: false,
isInteger: false,
isRequired: true,
max: 5.0,
min: 0.0,
name: "quality",
type: "score",
values: ["good", "bad", "neutral"],
},
],
},
description: "Queue for annotating customer support traces",
name: "My annotation queue",
projectId: "00000000-0000-0000-0000-000000000002",
restrictToAssignees: false,
restrictToReviewers: true,
reviewerEmails: ["reviewer@example.com"],
},
type: "queues",
},
},
};
apiInstance
.createLLMObsAnnotationQueue(params)
.then((data: v2.LLMObsAnnotationQueueResponse) => {
console.log(
"API called successfully. Returned data: " + JSON.stringify(data)
);
})
.catch((error: any) => console.error(error));
Instructions
First install the library and its dependencies and then save the example to example.ts and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"