Create an Agent Observability experiment
https://api.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.datadoghq.eu/api/v2/llm-obs/v1/experimentshttps://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.uk1.datadoghq.com/api/v2/llm-obs/v1/experimentshttps://api.ddog-gov.com/api/v2/llm-obs/v1/experimentshttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experiments概要
Create a new Agent Observability experiment.
リクエスト
Body Data (required)
Create experiment payload.
data [required]Data object for creating an Agent Observability experiment.
attributes [required]Attributes for creating an Agent Observability experiment.
configConfiguration parameters for the experiment.
dataset_idIdentifier of the dataset used in this experiment.
dataset_versionVersion of the dataset to use. Defaults to the current version if not specified.
descriptionDescription of the experiment.
ensure_uniqueWhether to ensure the experiment name is unique. Defaults to true.
metadataArbitrary metadata associated with the experiment.
name [required]Name of the experiment.
parent_experiment_idIdentifier of the parent (baseline) experiment this experiment is run against.
project_id [required]Identifier of the project this experiment belongs to.
run_countNumber of runs configured for this experiment.
type [required]Resource type of an Agent Observability experiment.
experiments{
"data": {
"attributes": {
"config": {},
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"dataset_version": "integer",
"description": "string",
"ensure_unique": false,
"metadata": {},
"name": "My Experiment v1",
"parent_experiment_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
"run_count": "integer"
},
"type": "experiments"
}
}応答
OK
Response containing a single Agent Observability experiment.
data [required]Data object for an Agent Observability experiment.
attributes [required]Attributes of an Agent Observability experiment.
aggregate_dataPre-computed aggregate metrics for this experiment run, including eval score distributions, token costs, and error rates.
authorUser data for the author of an experiment. Only present when include[user_data] is true.
emailEmail address of the user.
handleUsername or handle associated with the user's Datadog account.
iconURL of the user's icon.
idUnique identifier of the user.
nameDisplay name of the user.
config [required]Configuration parameters for the experiment.
created_at [required]Timestamp when the experiment was created.
dataset_id [required]Identifier of the dataset used in this experiment.
dataset_nameName of the dataset used in this experiment.
Only present when include[dataset_names] is true.
dataset_versionVersion of the dataset used in this experiment.
deleted_atTimestamp when the experiment was soft-deleted, if applicable.
description [required]Description of the experiment.
errorError message describing why the experiment failed, if applicable.
experimentLogical name of the experiment, shared across all runs of the same pipeline.
metadata [required]Arbitrary metadata associated with the experiment.
name [required]Name of the experiment.
parent_experiment_idIdentifier of the parent (baseline) experiment this experiment was run against, if any.
project_id [required]Identifier of the project this experiment belongs to.
run_countExpected number of runs for this experiment.
statusExecution status of an Agent Observability experiment.
running, completed, failed, interruptedupdated_at [required]Timestamp when the experiment was last updated.
id [required]Unique identifier of the experiment.
type [required]Resource type of an Agent Observability experiment.
experiments{
"data": {
"attributes": {
"aggregate_data": {},
"author": {
"email": "jane.doe@example.com",
"handle": "jane.doe@example.com",
"icon": "https://example.com/icon.png",
"id": "00000000-0000-0000-0000-000000000010",
"name": "Jane Doe"
},
"config": {},
"created_at": "2024-01-15T10:30:00Z",
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"dataset_name": "string",
"dataset_version": "integer",
"deleted_at": "2019-09-19T10:00:00.000Z",
"description": "",
"error": "string",
"experiment": "my-pipeline",
"metadata": {},
"name": "My Experiment v1",
"parent_experiment_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
"run_count": "integer",
"status": "completed",
"updated_at": "2024-01-15T10:30:00Z"
},
"id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
"type": "experiments"
}
}Created
Response containing a single Agent Observability experiment.
data [required]Data object for an Agent Observability experiment.
attributes [required]Attributes of an Agent Observability experiment.
aggregate_dataPre-computed aggregate metrics for this experiment run, including eval score distributions, token costs, and error rates.
authorUser data for the author of an experiment. Only present when include[user_data] is true.
emailEmail address of the user.
handleUsername or handle associated with the user's Datadog account.
iconURL of the user's icon.
idUnique identifier of the user.
nameDisplay name of the user.
config [required]Configuration parameters for the experiment.
created_at [required]Timestamp when the experiment was created.
dataset_id [required]Identifier of the dataset used in this experiment.
dataset_nameName of the dataset used in this experiment.
Only present when include[dataset_names] is true.
dataset_versionVersion of the dataset used in this experiment.
deleted_atTimestamp when the experiment was soft-deleted, if applicable.
description [required]Description of the experiment.
errorError message describing why the experiment failed, if applicable.
experimentLogical name of the experiment, shared across all runs of the same pipeline.
metadata [required]Arbitrary metadata associated with the experiment.
name [required]Name of the experiment.
parent_experiment_idIdentifier of the parent (baseline) experiment this experiment was run against, if any.
project_id [required]Identifier of the project this experiment belongs to.
run_countExpected number of runs for this experiment.
statusExecution status of an Agent Observability experiment.
running, completed, failed, interruptedupdated_at [required]Timestamp when the experiment was last updated.
id [required]Unique identifier of the experiment.
type [required]Resource type of an Agent Observability experiment.
experiments{
"data": {
"attributes": {
"aggregate_data": {},
"author": {
"email": "jane.doe@example.com",
"handle": "jane.doe@example.com",
"icon": "https://example.com/icon.png",
"id": "00000000-0000-0000-0000-000000000010",
"name": "Jane Doe"
},
"config": {},
"created_at": "2024-01-15T10:30:00Z",
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"dataset_name": "string",
"dataset_version": "integer",
"deleted_at": "2019-09-19T10:00:00.000Z",
"description": "",
"error": "string",
"experiment": "my-pipeline",
"metadata": {},
"name": "My Experiment v1",
"parent_experiment_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
"run_count": "integer",
"status": "completed",
"updated_at": "2024-01-15T10:30:00Z"
},
"id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
"type": "experiments"
}
}Bad Request
API error response.
errors [required]A list of errors.
detailA human-readable explanation specific to this occurrence of the error.
metaNon-standard meta-information about the error
sourceReferences to the source of the error.
headerA string indicating the name of a single request header which caused the error.
parameterA string indicating which URI query parameter caused the error.
pointerA JSON pointer to the value in the request document that caused the error.
statusStatus code of the response.
titleShort human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Unauthorized
API error response.
errors [required]A list of errors.
detailA human-readable explanation specific to this occurrence of the error.
metaNon-standard meta-information about the error
sourceReferences to the source of the error.
headerA string indicating the name of a single request header which caused the error.
parameterA string indicating which URI query parameter caused the error.
pointerA JSON pointer to the value in the request document that caused the error.
statusStatus code of the response.
titleShort human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Forbidden
API error response.
errors [required]A list of errors.
detailA human-readable explanation specific to this occurrence of the error.
metaNon-standard meta-information about the error
sourceReferences to the source of the error.
headerA string indicating the name of a single request header which caused the error.
parameterA string indicating which URI query parameter caused the error.
pointerA JSON pointer to the value in the request document that caused the error.
statusStatus code of the response.
titleShort human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Too many requests
API error response.
errors [required]A list of errors.
{
"errors": [
"Bad Request"
]
}コード例
# Set your Datadog site and credentials
export DD_SITE="datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"name": "My Experiment v1",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
},
"type": "experiments"
}
}
EOF# Set your Datadog site and credentials
export DD_SITE="us3.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"name": "My Experiment v1",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
},
"type": "experiments"
}
}
EOF# Set your Datadog site and credentials
export DD_SITE="us5.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"name": "My Experiment v1",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
},
"type": "experiments"
}
}
EOF# Set your Datadog site and credentials
export DD_SITE="datadoghq.eu"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"name": "My Experiment v1",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
},
"type": "experiments"
}
}
EOF# Set your Datadog site and credentials
export DD_SITE="ap1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"name": "My Experiment v1",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
},
"type": "experiments"
}
}
EOF# Set your Datadog site and credentials
export DD_SITE="ap2.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"name": "My Experiment v1",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
},
"type": "experiments"
}
}
EOF# Set your Datadog site and credentials
export DD_SITE="uk1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"name": "My Experiment v1",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
},
"type": "experiments"
}
}
EOF# Set your Datadog site and credentials
export DD_SITE="ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"name": "My Experiment v1",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
},
"type": "experiments"
}
}
EOF# Set your Datadog site and credentials
export DD_SITE="us2.ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/experiments" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
"data": {
"attributes": {
"dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
"name": "My Experiment v1",
"project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
},
"type": "experiments"
}
}
EOF// Create an Agent Observability experiment returns "OK" response
package main
import (
"context"
"encoding/json"
"fmt"
"os"
"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)
func main() {
body := datadogV2.LLMObsExperimentRequest{
Data: datadogV2.LLMObsExperimentDataRequest{
Attributes: datadogV2.LLMObsExperimentDataAttributesRequest{
DatasetId: datadog.PtrString("9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"),
Name: "My Experiment v1",
ParentExperimentId: datadog.PtrString("3fd6b5e0-8910-4b1c-a7d0-5b84de329012"),
ProjectId: "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
},
Type: datadogV2.LLMOBSEXPERIMENTTYPE_EXPERIMENTS,
},
}
ctx := datadog.NewDefaultContext(context.Background())
ctx = context.WithValue(ctx, datadog.ContextAccessToken, os.Getenv("DD_BEARER_TOKEN"))
configuration := datadog.NewConfiguration()
configuration.SetUnstableOperationEnabled("v2.CreateLLMObsExperiment", true)
apiClient := datadog.NewAPIClient(configuration)
api := datadogV2.NewAgentObservabilityApi(apiClient)
resp, r, err := api.CreateLLMObsExperiment(ctx, body)
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `AgentObservabilityApi.CreateLLMObsExperiment`: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
responseContent, _ := json.MarshalIndent(resp, "", " ")
fmt.Fprintf(os.Stdout, "Response from `AgentObservabilityApi.CreateLLMObsExperiment`:\n%s\n", responseContent)
}
Instructions
First install the library and its dependencies and then save the example to main.go and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"// Create an Agent Observability experiment returns "OK" response
import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.AgentObservabilityApi;
import com.datadog.api.client.v2.model.LLMObsExperimentDataAttributesRequest;
import com.datadog.api.client.v2.model.LLMObsExperimentDataRequest;
import com.datadog.api.client.v2.model.LLMObsExperimentRequest;
import com.datadog.api.client.v2.model.LLMObsExperimentResponse;
import com.datadog.api.client.v2.model.LLMObsExperimentType;
public class Example {
public static void main(String[] args) {
ApiClient defaultClient = ApiClient.getDefaultApiClient();
defaultClient.setAccessToken(System.getenv("DD_BEARER_TOKEN"));
defaultClient.setUnstableOperationEnabled("v2.createLLMObsExperiment", true);
AgentObservabilityApi apiInstance = new AgentObservabilityApi(defaultClient);
LLMObsExperimentRequest body =
new LLMObsExperimentRequest()
.data(
new LLMObsExperimentDataRequest()
.attributes(
new LLMObsExperimentDataAttributesRequest()
.datasetId("9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d")
.name("My Experiment v1")
.parentExperimentId("3fd6b5e0-8910-4b1c-a7d0-5b84de329012")
.projectId("a33671aa-24fd-4dcd-9b33-a8ec7dde7751"))
.type(LLMObsExperimentType.EXPERIMENTS));
try {
LLMObsExperimentResponse result = apiInstance.createLLMObsExperiment(body);
System.out.println(result);
} catch (ApiException e) {
System.err.println("Exception when calling AgentObservabilityApi#createLLMObsExperiment");
System.err.println("Status code: " + e.getCode());
System.err.println("Reason: " + e.getResponseBody());
System.err.println("Response headers: " + e.getResponseHeaders());
e.printStackTrace();
}
}
}
Instructions
First install the library and its dependencies and then save the example to Example.java and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java""""
Create an Agent Observability experiment returns "OK" response
"""
from os import environ
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.agent_observability_api import AgentObservabilityApi
from datadog_api_client.v2.model.llm_obs_experiment_data_attributes_request import LLMObsExperimentDataAttributesRequest
from datadog_api_client.v2.model.llm_obs_experiment_data_request import LLMObsExperimentDataRequest
from datadog_api_client.v2.model.llm_obs_experiment_request import LLMObsExperimentRequest
from datadog_api_client.v2.model.llm_obs_experiment_type import LLMObsExperimentType
body = LLMObsExperimentRequest(
data=LLMObsExperimentDataRequest(
attributes=LLMObsExperimentDataAttributesRequest(
dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
name="My Experiment v1",
parent_experiment_id="3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
),
type=LLMObsExperimentType.EXPERIMENTS,
),
)
configuration = Configuration()
configuration.access_token = environ["DD_BEARER_TOKEN"]
configuration.unstable_operations["create_llm_obs_experiment"] = True
with ApiClient(configuration) as api_client:
api_instance = AgentObservabilityApi(api_client)
response = api_instance.create_llm_obs_experiment(body=body)
print(response)
Instructions
First install the library and its dependencies and then save the example to example.py and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"# Create an Agent Observability experiment returns "OK" response
require "datadog_api_client"
DatadogAPIClient.configure do |config|
config.access_token = ENV["DD_BEARER_TOKEN"]
config.unstable_operations["v2.create_llm_obs_experiment".to_sym] = true
end
api_instance = DatadogAPIClient::V2::AgentObservabilityAPI.new
body = DatadogAPIClient::V2::LLMObsExperimentRequest.new({
data: DatadogAPIClient::V2::LLMObsExperimentDataRequest.new({
attributes: DatadogAPIClient::V2::LLMObsExperimentDataAttributesRequest.new({
dataset_id: "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
name: "My Experiment v1",
parent_experiment_id: "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
project_id: "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
}),
type: DatadogAPIClient::V2::LLMObsExperimentType::EXPERIMENTS,
}),
})
p api_instance.create_llm_obs_experiment(body)
Instructions
First install the library and its dependencies and then save the example to example.rb and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"// Create an Agent Observability experiment returns "OK" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_agent_observability::AgentObservabilityAPI;
use datadog_api_client::datadogV2::model::LLMObsExperimentDataAttributesRequest;
use datadog_api_client::datadogV2::model::LLMObsExperimentDataRequest;
use datadog_api_client::datadogV2::model::LLMObsExperimentRequest;
use datadog_api_client::datadogV2::model::LLMObsExperimentType;
#[tokio::main]
async fn main() {
let body = LLMObsExperimentRequest::new(LLMObsExperimentDataRequest::new(
LLMObsExperimentDataAttributesRequest::new(
"My Experiment v1".to_string(),
"a33671aa-24fd-4dcd-9b33-a8ec7dde7751".to_string(),
)
.dataset_id("9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d".to_string())
.parent_experiment_id("3fd6b5e0-8910-4b1c-a7d0-5b84de329012".to_string()),
LLMObsExperimentType::EXPERIMENTS,
));
let mut configuration = datadog::Configuration::new();
configuration.set_unstable_operation_enabled("v2.create_llm_obs_experiment", true);
let api = AgentObservabilityAPI::with_config(configuration);
let resp = api.create_llm_obs_experiment(body).await;
if let Ok(value) = resp {
println!("{:#?}", value);
} else {
println!("{:#?}", resp.unwrap_err());
}
}
Instructions
First install the library and its dependencies and then save the example to src/main.rs and run following commands:
DD_SITE="datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us3.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us5.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="datadoghq.eu" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ap1.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ap2.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="uk1.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run/**
* Create an Agent Observability experiment returns "OK" response
*/
import { client, v2 } from "@datadog/datadog-api-client";
const configuration = client.createConfiguration({
authMethods: {
AuthZ: {
accessToken: process.env.DD_BEARER_TOKEN as string,
},
},
});
configuration.unstableOperations["v2.createLLMObsExperiment"] = true;
const apiInstance = new v2.AgentObservabilityApi(configuration);
const params: v2.AgentObservabilityApiCreateLLMObsExperimentRequest = {
body: {
data: {
attributes: {
datasetId: "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d",
name: "My Experiment v1",
parentExperimentId: "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
projectId: "a33671aa-24fd-4dcd-9b33-a8ec7dde7751",
},
type: "experiments",
},
},
};
apiInstance
.createLLMObsExperiment(params)
.then((data: v2.LLMObsExperimentResponse) => {
console.log(
"API called successfully. Returned data: " + JSON.stringify(data)
);
})
.catch((error: any) => console.error(error));
Instructions
First install the library and its dependencies and then save the example to example.ts and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"