Update annotation queue label schema
WARNINGNote: This endpoint is in preview and is subject to change.
If you have any feedback, contact Datadog support.
PUT
https://api.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schemahttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schemahttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schemahttps://api.datadoghq.eu/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schemahttps://api.ap1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schemahttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schemahttps://api.uk1.datadoghq.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schemahttps://api.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schemahttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/annotation-queues/{queue_id}/label-schemaOverview
Create or replace the label schema for a given annotation queue.
The label schema defines the labels annotators can apply to interactions in the queue.
Label names must be unique within the queue and match the pattern ^[a-zA-Z0-9_-]+$.
Each label must have a valid type: score, categorical, boolean, or text.
Arguments
Path Parameters
Name
Type
Description
queue_id [required]string
The ID of the Agent Observability annotation queue.
Request
Body Data (required)
Update label schema payload.
Field
Type
Description
data [required]object
Data object for updating an annotation queue label schema.
attributes [required]object
Attributes for updating an annotation queue label schema.
annotation_schema [required]object
Schema defining the labels for an annotation queue.
label_schemas [required][object]
List of label schema definitions.
descriptionstring
Description of the label.
has_assessmentboolean
Whether this label includes an assessment field.
has_reasoningboolean
Whether this label includes a reasoning field.
idstring
Unique identifier of the label schema. Assigned by the server if not provided.
is_assessmentboolean
Whether the boolean label represents an assessment. Requires has_assessment to be true.
is_integerboolean
Whether score values must be integers. Applicable to score-type labels.
is_requiredboolean
Whether this label is required for an annotation.
maxdouble
Maximum value for score-type labels.
mindouble
Minimum value for score-type labels.
name [required]string
Name of the label. Must match the pattern ^[a-zA-Z0-9_-]+$ and be unique within the queue.
type [required]enum
Type of a label in an annotation queue label schema.
Allowed enum values:
score, categorical, boolean, textvalues[string]
Allowed values for categorical-type labels. Must contain at least one non-empty, unique value.
type [required]enum
Resource type of an Agent Observability annotation queue.
Allowed enum values:
queuesBoolean label schema
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_assessment": true,
"is_assessment": true,
"is_required": true,
"name": "is_correct",
"type": "boolean"
}
]
}
},
"type": "queues"
}
}Categorical label schema
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"name": "sentiment",
"type": "categorical",
"values": [
"positive",
"negative",
"neutral"
]
}
]
}
},
"type": "queues"
}
}default
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
}
]
}
},
"type": "queues"
}
}Score label schema
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_reasoning": true,
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score"
}
]
}
},
"type": "queues"
}
}Text label schema
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"is_required": false,
"name": "feedback",
"type": "text"
}
]
}
},
"type": "queues"
}
}Response
OK
Response containing the label schema of an annotation queue.
Field
Type
Description
data [required]object
Data object for an annotation queue label schema.
attributes [required]object
Attributes of an annotation queue label schema.
annotation_schema [required]object
Schema defining the labels for an annotation queue.
label_schemas [required][object]
List of label schema definitions.
descriptionstring
Description of the label.
has_assessmentboolean
Whether this label includes an assessment field.
has_reasoningboolean
Whether this label includes a reasoning field.
idstring
Unique identifier of the label schema. Assigned by the server if not provided.
is_assessmentboolean
Whether the boolean label represents an assessment. Requires has_assessment to be true.
is_integerboolean
Whether score values must be integers. Applicable to score-type labels.
is_requiredboolean
Whether this label is required for an annotation.
maxdouble
Maximum value for score-type labels.
mindouble
Minimum value for score-type labels.
name [required]string
Name of the label. Must match the pattern ^[a-zA-Z0-9_-]+$ and be unique within the queue.
type [required]enum
Type of a label in an annotation queue label schema.
Allowed enum values:
score, categorical, boolean, textvalues[string]
Allowed values for categorical-type labels. Must contain at least one non-empty, unique value.
id [required]string
Unique identifier of the annotation queue.
type [required]enum
Resource type of an Agent Observability annotation queue.
Allowed enum values:
queues{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"description": "Rating of the response quality.",
"has_assessment": false,
"has_reasoning": false,
"id": "abc-123",
"is_assessment": false,
"is_integer": false,
"is_required": true,
"max": 5,
"min": 0,
"name": "quality",
"type": "score",
"values": [
"good",
"bad",
"neutral"
]
}
]
}
},
"id": "00000000-0000-0000-0000-000000000001",
"type": "queues"
}
}Bad Request
API error response.
Field
Type
Description
errors [required][object]
A list of errors.
detailstring
A human-readable explanation specific to this occurrence of the error.
metaobject
Non-standard meta-information about the error
sourceobject
References to the source of the error.
headerstring
A string indicating the name of a single request header which caused the error.
parameterstring
A string indicating which URI query parameter caused the error.
pointerstring
A JSON pointer to the value in the request document that caused the error.
statusstring
Status code of the response.
titlestring
Short human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Unauthorized
API error response.
Field
Type
Description
errors [required][object]
A list of errors.
detailstring
A human-readable explanation specific to this occurrence of the error.
metaobject
Non-standard meta-information about the error
sourceobject
References to the source of the error.
headerstring
A string indicating the name of a single request header which caused the error.
parameterstring
A string indicating which URI query parameter caused the error.
pointerstring
A JSON pointer to the value in the request document that caused the error.
statusstring
Status code of the response.
titlestring
Short human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Forbidden
API error response.
Field
Type
Description
errors [required][object]
A list of errors.
detailstring
A human-readable explanation specific to this occurrence of the error.
metaobject
Non-standard meta-information about the error
sourceobject
References to the source of the error.
headerstring
A string indicating the name of a single request header which caused the error.
parameterstring
A string indicating which URI query parameter caused the error.
pointerstring
A JSON pointer to the value in the request document that caused the error.
statusstring
Status code of the response.
titlestring
Short human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Not Found
API error response.
Field
Type
Description
errors [required][object]
A list of errors.
detailstring
A human-readable explanation specific to this occurrence of the error.
metaobject
Non-standard meta-information about the error
sourceobject
References to the source of the error.
headerstring
A string indicating the name of a single request header which caused the error.
parameterstring
A string indicating which URI query parameter caused the error.
pointerstring
A JSON pointer to the value in the request document that caused the error.
statusstring
Status code of the response.
titlestring
Short human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Too many requests
API error response.
Field
Type
Description
errors [required][string]
A list of errors.
{
"errors": [
"Bad Request"
]
}Internal Server Error
API error response.
Field
Type
Description
errors [required][object]
A list of errors.
detailstring
A human-readable explanation specific to this occurrence of the error.
metaobject
Non-standard meta-information about the error
sourceobject
References to the source of the error.
headerstring
A string indicating the name of a single request header which caused the error.
parameterstring
A string indicating which URI query parameter caused the error.
pointerstring
A JSON pointer to the value in the request document that caused the error.
statusstring
Status code of the response.
titlestring
Short human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Code Example
# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Set your Datadog site and credentials
export DD_SITE="datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X PUT "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues/${queue_id}/label-schema" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_assessment": true,
"is_assessment": true,
"is_required": true,
"name": "is_correct",
"type": "boolean"
}
]
}
},
"type": "queues"
}
}
EOF# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Set your Datadog site and credentials
export DD_SITE="us3.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X PUT "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues/${queue_id}/label-schema" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_assessment": true,
"is_assessment": true,
"is_required": true,
"name": "is_correct",
"type": "boolean"
}
]
}
},
"type": "queues"
}
}
EOF# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Set your Datadog site and credentials
export DD_SITE="us5.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X PUT "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues/${queue_id}/label-schema" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_assessment": true,
"is_assessment": true,
"is_required": true,
"name": "is_correct",
"type": "boolean"
}
]
}
},
"type": "queues"
}
}
EOF# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Set your Datadog site and credentials
export DD_SITE="datadoghq.eu"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X PUT "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues/${queue_id}/label-schema" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_assessment": true,
"is_assessment": true,
"is_required": true,
"name": "is_correct",
"type": "boolean"
}
]
}
},
"type": "queues"
}
}
EOF# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Set your Datadog site and credentials
export DD_SITE="ap1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X PUT "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues/${queue_id}/label-schema" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_assessment": true,
"is_assessment": true,
"is_required": true,
"name": "is_correct",
"type": "boolean"
}
]
}
},
"type": "queues"
}
}
EOF# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Set your Datadog site and credentials
export DD_SITE="ap2.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X PUT "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues/${queue_id}/label-schema" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_assessment": true,
"is_assessment": true,
"is_required": true,
"name": "is_correct",
"type": "boolean"
}
]
}
},
"type": "queues"
}
}
EOF# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Set your Datadog site and credentials
export DD_SITE="uk1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X PUT "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues/${queue_id}/label-schema" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_assessment": true,
"is_assessment": true,
"is_required": true,
"name": "is_correct",
"type": "boolean"
}
]
}
},
"type": "queues"
}
}
EOF# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Set your Datadog site and credentials
export DD_SITE="ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X PUT "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues/${queue_id}/label-schema" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_assessment": true,
"is_assessment": true,
"is_required": true,
"name": "is_correct",
"type": "boolean"
}
]
}
},
"type": "queues"
}
}
EOF# Path parameters
export queue_id="00000000-0000-0000-0000-000000000001"
# Set your Datadog site and credentials
export DD_SITE="us2.ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X PUT "https://api.${DD_SITE}/api/v2/llm-obs/v1/annotation-queues/${queue_id}/label-schema" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"annotation_schema": {
"label_schemas": [
{
"has_assessment": true,
"is_assessment": true,
"is_required": true,
"name": "is_correct",
"type": "boolean"
}
]
}
},
"type": "queues"
}
}
EOF// Update annotation queue label schema returns "OK" response
package main
import (
"context"
"encoding/json"
"fmt"
"os"
"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)
func main() {
body := datadogV2.LLMObsAnnotationQueueLabelSchemaUpdateRequest{
Data: datadogV2.LLMObsAnnotationQueueLabelSchemaUpdateData{
Attributes: datadogV2.LLMObsAnnotationQueueLabelSchemaUpdateAttributes{
AnnotationSchema: datadogV2.LLMObsAnnotationSchema{
LabelSchemas: []datadogV2.LLMObsLabelSchema{
{
Description: datadog.PtrString("Rating of the response quality."),
HasAssessment: datadog.PtrBool(false),
HasReasoning: datadog.PtrBool(false),
Id: datadog.PtrString("abc-123"),
IsAssessment: datadog.PtrBool(false),
IsInteger: datadog.PtrBool(false),
IsRequired: datadog.PtrBool(true),
Max: datadog.PtrFloat64(5.0),
Min: datadog.PtrFloat64(0.0),
Name: "quality",
Type: datadogV2.LLMOBSLABELSCHEMATYPE_SCORE,
Values: []string{
"good",
"bad",
"neutral",
},
},
},
},
},
Type: datadogV2.LLMOBSANNOTATIONQUEUETYPE_QUEUES,
},
}
ctx := datadog.NewDefaultContext(context.Background())
ctx = context.WithValue(ctx, datadog.ContextAccessToken, os.Getenv("DD_BEARER_TOKEN"))
configuration := datadog.NewConfiguration()
configuration.SetUnstableOperationEnabled("v2.UpdateLLMObsAnnotationQueueLabelSchema", true)
apiClient := datadog.NewAPIClient(configuration)
api := datadogV2.NewAgentObservabilityApi(apiClient)
resp, r, err := api.UpdateLLMObsAnnotationQueueLabelSchema(ctx, "queue_id", body)
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `AgentObservabilityApi.UpdateLLMObsAnnotationQueueLabelSchema`: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
responseContent, _ := json.MarshalIndent(resp, "", " ")
fmt.Fprintf(os.Stdout, "Response from `AgentObservabilityApi.UpdateLLMObsAnnotationQueueLabelSchema`:\n%s\n", responseContent)
}
Instructions
First install the library and its dependencies and then save the example to main.go and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"// Update annotation queue label schema returns "OK" response
import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.AgentObservabilityApi;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueLabelSchemaResponse;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueLabelSchemaUpdateAttributes;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueLabelSchemaUpdateData;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueLabelSchemaUpdateRequest;
import com.datadog.api.client.v2.model.LLMObsAnnotationQueueType;
import com.datadog.api.client.v2.model.LLMObsAnnotationSchema;
import com.datadog.api.client.v2.model.LLMObsLabelSchema;
import com.datadog.api.client.v2.model.LLMObsLabelSchemaType;
import java.util.Arrays;
import java.util.Collections;
public class Example {
public static void main(String[] args) {
ApiClient defaultClient = ApiClient.getDefaultApiClient();
defaultClient.setAccessToken(System.getenv("DD_BEARER_TOKEN"));
defaultClient.setUnstableOperationEnabled("v2.updateLLMObsAnnotationQueueLabelSchema", true);
AgentObservabilityApi apiInstance = new AgentObservabilityApi(defaultClient);
LLMObsAnnotationQueueLabelSchemaUpdateRequest body =
new LLMObsAnnotationQueueLabelSchemaUpdateRequest()
.data(
new LLMObsAnnotationQueueLabelSchemaUpdateData()
.attributes(
new LLMObsAnnotationQueueLabelSchemaUpdateAttributes()
.annotationSchema(
new LLMObsAnnotationSchema()
.labelSchemas(
Collections.singletonList(
new LLMObsLabelSchema()
.description("Rating of the response quality.")
.hasAssessment(false)
.hasReasoning(false)
.id("abc-123")
.isAssessment(false)
.isInteger(false)
.isRequired(true)
.max(5.0)
.name("quality")
.type(LLMObsLabelSchemaType.SCORE)
.values(Arrays.asList("good", "bad", "neutral"))))))
.type(LLMObsAnnotationQueueType.QUEUES));
try {
LLMObsAnnotationQueueLabelSchemaResponse result =
apiInstance.updateLLMObsAnnotationQueueLabelSchema(
"00000000-0000-0000-0000-000000000001", body);
System.out.println(result);
} catch (ApiException e) {
System.err.println(
"Exception when calling AgentObservabilityApi#updateLLMObsAnnotationQueueLabelSchema");
System.err.println("Status code: " + e.getCode());
System.err.println("Reason: " + e.getResponseBody());
System.err.println("Response headers: " + e.getResponseHeaders());
e.printStackTrace();
}
}
}
Instructions
First install the library and its dependencies and then save the example to Example.java and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java""""
Update annotation queue label schema returns "OK" response
"""
from os import environ
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.agent_observability_api import AgentObservabilityApi
from datadog_api_client.v2.model.llm_obs_annotation_queue_label_schema_update_attributes import (
LLMObsAnnotationQueueLabelSchemaUpdateAttributes,
)
from datadog_api_client.v2.model.llm_obs_annotation_queue_label_schema_update_data import (
LLMObsAnnotationQueueLabelSchemaUpdateData,
)
from datadog_api_client.v2.model.llm_obs_annotation_queue_label_schema_update_request import (
LLMObsAnnotationQueueLabelSchemaUpdateRequest,
)
from datadog_api_client.v2.model.llm_obs_annotation_queue_type import LLMObsAnnotationQueueType
from datadog_api_client.v2.model.llm_obs_annotation_schema import LLMObsAnnotationSchema
from datadog_api_client.v2.model.llm_obs_label_schema import LLMObsLabelSchema
from datadog_api_client.v2.model.llm_obs_label_schema_type import LLMObsLabelSchemaType
body = LLMObsAnnotationQueueLabelSchemaUpdateRequest(
data=LLMObsAnnotationQueueLabelSchemaUpdateData(
attributes=LLMObsAnnotationQueueLabelSchemaUpdateAttributes(
annotation_schema=LLMObsAnnotationSchema(
label_schemas=[
LLMObsLabelSchema(
description="Rating of the response quality.",
has_assessment=False,
has_reasoning=False,
id="abc-123",
is_assessment=False,
is_integer=False,
is_required=True,
max=5.0,
min=0.0,
name="quality",
type=LLMObsLabelSchemaType.SCORE,
values=[
"good",
"bad",
"neutral",
],
),
],
),
),
type=LLMObsAnnotationQueueType.QUEUES,
),
)
configuration = Configuration()
configuration.access_token = environ["DD_BEARER_TOKEN"]
configuration.unstable_operations["update_llm_obs_annotation_queue_label_schema"] = True
with ApiClient(configuration) as api_client:
api_instance = AgentObservabilityApi(api_client)
response = api_instance.update_llm_obs_annotation_queue_label_schema(queue_id="queue_id", body=body)
print(response)
Instructions
First install the library and its dependencies and then save the example to example.py and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"# Update annotation queue label schema returns "OK" response
require "datadog_api_client"
DatadogAPIClient.configure do |config|
config.access_token = ENV["DD_BEARER_TOKEN"]
config.unstable_operations["v2.update_llm_obs_annotation_queue_label_schema".to_sym] = true
end
api_instance = DatadogAPIClient::V2::AgentObservabilityAPI.new
body = DatadogAPIClient::V2::LLMObsAnnotationQueueLabelSchemaUpdateRequest.new({
data: DatadogAPIClient::V2::LLMObsAnnotationQueueLabelSchemaUpdateData.new({
attributes: DatadogAPIClient::V2::LLMObsAnnotationQueueLabelSchemaUpdateAttributes.new({
annotation_schema: DatadogAPIClient::V2::LLMObsAnnotationSchema.new({
label_schemas: [
DatadogAPIClient::V2::LLMObsLabelSchema.new({
description: "Rating of the response quality.",
has_assessment: false,
has_reasoning: false,
id: "abc-123",
is_assessment: false,
is_integer: false,
is_required: true,
max: 5.0,
min: 0.0,
name: "quality",
type: DatadogAPIClient::V2::LLMObsLabelSchemaType::SCORE,
values: [
"good",
"bad",
"neutral",
],
}),
],
}),
}),
type: DatadogAPIClient::V2::LLMObsAnnotationQueueType::QUEUES,
}),
})
p api_instance.update_llm_obs_annotation_queue_label_schema("queue_id", body)
Instructions
First install the library and its dependencies and then save the example to example.rb and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"// Update annotation queue label schema returns "OK" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_agent_observability::AgentObservabilityAPI;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueLabelSchemaUpdateAttributes;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueLabelSchemaUpdateData;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueLabelSchemaUpdateRequest;
use datadog_api_client::datadogV2::model::LLMObsAnnotationQueueType;
use datadog_api_client::datadogV2::model::LLMObsAnnotationSchema;
use datadog_api_client::datadogV2::model::LLMObsLabelSchema;
use datadog_api_client::datadogV2::model::LLMObsLabelSchemaType;
#[tokio::main]
async fn main() {
let body = LLMObsAnnotationQueueLabelSchemaUpdateRequest::new(
LLMObsAnnotationQueueLabelSchemaUpdateData::new(
LLMObsAnnotationQueueLabelSchemaUpdateAttributes::new(LLMObsAnnotationSchema::new(
vec![
LLMObsLabelSchema::new("quality".to_string(), LLMObsLabelSchemaType::SCORE)
.description("Rating of the response quality.".to_string())
.has_assessment(false)
.has_reasoning(false)
.id("abc-123".to_string())
.is_assessment(false)
.is_integer(false)
.is_required(true)
.max(5.0 as f64)
.min(0.0 as f64)
.values(vec![
"good".to_string(),
"bad".to_string(),
"neutral".to_string(),
]),
],
)),
LLMObsAnnotationQueueType::QUEUES,
),
);
let mut configuration = datadog::Configuration::new();
configuration
.set_unstable_operation_enabled("v2.update_llm_obs_annotation_queue_label_schema", true);
let api = AgentObservabilityAPI::with_config(configuration);
let resp = api
.update_llm_obs_annotation_queue_label_schema("queue_id".to_string(), body)
.await;
if let Ok(value) = resp {
println!("{:#?}", value);
} else {
println!("{:#?}", resp.unwrap_err());
}
}
Instructions
First install the library and its dependencies and then save the example to src/main.rs and run following commands:
DD_SITE="datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us3.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us5.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="datadoghq.eu" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ap1.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ap2.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="uk1.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run/**
* Update annotation queue label schema returns "OK" response
*/
import { client, v2 } from "@datadog/datadog-api-client";
const configuration = client.createConfiguration({
authMethods: {
AuthZ: {
accessToken: process.env.DD_BEARER_TOKEN as string,
},
},
});
configuration.unstableOperations["v2.updateLLMObsAnnotationQueueLabelSchema"] =
true;
const apiInstance = new v2.AgentObservabilityApi(configuration);
const params: v2.AgentObservabilityApiUpdateLLMObsAnnotationQueueLabelSchemaRequest =
{
body: {
data: {
attributes: {
annotationSchema: {
labelSchemas: [
{
description: "Rating of the response quality.",
hasAssessment: false,
hasReasoning: false,
id: "abc-123",
isAssessment: false,
isInteger: false,
isRequired: true,
max: 5.0,
min: 0.0,
name: "quality",
type: "score",
values: ["good", "bad", "neutral"],
},
],
},
},
type: "queues",
},
},
queueId: "queue_id",
};
apiInstance
.updateLLMObsAnnotationQueueLabelSchema(params)
.then((data: v2.LLMObsAnnotationQueueLabelSchemaResponse) => {
console.log(
"API called successfully. Returned data: " + JSON.stringify(data)
);
})
.catch((error: any) => console.error(error));
Instructions
First install the library and its dependencies and then save the example to example.ts and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"