사용자 지정 평가자 구성 조회
https://api.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}https://api.us3.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}https://api.us5.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}https://api.datadoghq.eu/api/unstable/llm-obs/config/evaluators/custom/{eval_name}https://api.ap1.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}https://api.ap2.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}https://api.uk1.datadoghq.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}https://api.ddog-gov.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}https://api.us2.ddog-gov.com/api/unstable/llm-obs/config/evaluators/custom/{eval_name}개요
이름으로 사용자 지정 Agent Observability 평가자 구성을 검색합니다.
인수
경로 파라미터
eval_name [required]The name of the custom Agent Observability evaluator configuration.
응답
OK
Response containing a custom Agent Observability evaluator configuration.
data [required]Data object for a custom Agent Observability evaluator configuration.
attributes [required]Attributes of a custom Agent Observability evaluator configuration.
categoryCategory of the evaluator.
created_at [required]Timestamp when the evaluator configuration was created.
created_byA Datadog user associated with a custom evaluator configuration.
emailEmail address of the user.
eval_name [required]Name of the custom evaluator.
last_updated_byA Datadog user associated with a custom evaluator configuration.
emailEmail address of the user.
llm_judge_configLLM judge configuration for a custom evaluator.
assessment_criteriaCriteria used to assess the pass/fail result of a custom evaluator.
max_thresholdMaximum numeric threshold for a passing result.
min_thresholdMinimum numeric threshold for a passing result.
pass_valuesSpecific output values considered as a passing result.
pass_whenWhen true, a boolean output of true is treated as passing.
context_queryQuery used to extract additional context for the evaluation.
inference_params [required]LLM inference parameters for a custom evaluator.
frequency_penaltyFrequency penalty to reduce repetition.
max_tokensMaximum number of tokens to generate.
presence_penaltyPresence penalty to reduce repetition.
temperatureSampling temperature for the LLM.
top_kTop-k sampling parameter.
top_pTop-p (nucleus) sampling parameter.
last_used_library_prompt_template_nameName of the last library prompt template used.
modified_library_prompt_templateWhether the library prompt template was modified.
output_schemaJSON schema describing the expected output format of the LLM judge.
parsing_typeOutput parsing type for a custom LLM judge evaluator.
structured_output, json, keyword_searchprompt_templateList of messages forming the LLM judge prompt template.
contentText content of the message.
contentsMulti-part content blocks for the message.
type [required]Content block type.
value [required]Value of a prompt message content block.
textText content of the message block.
tool_callA tool call within a prompt message.
argumentsJSON-encoded arguments for the tool call.
idUnique identifier of the tool call.
nameName of the tool being called.
typeType of the tool call.
tool_call_resultA tool call result within a prompt message.
nameName of the tool that produced this result.
resultThe result returned by the tool.
tool_idIdentifier of the tool call this result corresponds to.
typeType of the tool result.
role [required]Role of the message author.
target_queryQuery used to extract the target value to evaluate.
user_specified_json_post_processing_functionUser-provided function applied to post-process the JSON output of the LLM judge.
llm_providerLLM provider configuration for a custom evaluator.
bedrockAWS Bedrock-specific options for LLM provider configuration.
inference_profileBedrock inference profile identifier, such as an application inference profile ARN.
regionAWS region for Bedrock.
integration_account_idIntegration account identifier.
integration_providerName of the LLM integration provider.
openai, amazon-bedrock, anthropic, azure-openai, vertex-ai, llm-proxymodel_nameName of the LLM model.
vertex_aiGoogle Vertex AI-specific options for LLM provider configuration.
locationGoogle Cloud region.
projectGoogle Cloud project ID.
targetTarget application configuration for a custom evaluator.
application_name [required]Name of the ML application this evaluator targets.
enabled [required]Whether the evaluator is active for the target application.
eval_scopeScope at which to evaluate spans.
span, trace, sessionexperiment_project_idsExperiment project IDs this evaluator is scoped to.
filterFilter expression to select which spans to evaluate.
root_spans_onlyWhen true, only root spans are evaluated.
sampling_percentagePercentage of traces to evaluate. Must be greater than 0 and at most 100.
updated_at [required]Timestamp when the evaluator configuration was last updated.
id [required]Unique name identifier of the evaluator configuration.
type [required]Type of the custom Agent Observability evaluator configuration resource.
evaluator_config{
"data": {
"attributes": {
"category": "Custom",
"created_at": "2024-01-15T10:30:00Z",
"created_by": {
"email": "user@example.com"
},
"eval_name": "my-custom-evaluator",
"last_updated_by": {
"email": "user@example.com"
},
"llm_judge_config": {
"assessment_criteria": {
"max_threshold": 1,
"min_threshold": 0.7,
"pass_values": [
"pass",
"yes"
],
"pass_when": true
},
"context_query": "@input.context",
"inference_params": {
"frequency_penalty": 0,
"max_tokens": 1024,
"presence_penalty": 0,
"temperature": 0.7,
"top_k": 50,
"top_p": 1
},
"last_used_library_prompt_template_name": "sentiment-analysis-v1",
"modified_library_prompt_template": false,
"output_schema": {},
"parsing_type": "structured_output",
"prompt_template": [
{
"content": "Rate the quality of the following response:",
"contents": [
{
"type": "text",
"value": {
"text": "What is the sentiment of this review?",
"tool_call": {
"arguments": "{\"location\": \"San Francisco\"}",
"id": "call_abc123",
"name": "get_weather",
"type": "function"
},
"tool_call_result": {
"name": "get_weather",
"result": "sunny, 72F",
"tool_id": "call_abc123",
"type": "function"
}
}
}
],
"role": "user"
}
],
"target_query": "@output.value",
"user_specified_json_post_processing_function": "string"
},
"llm_provider": {
"bedrock": {
"inference_profile": "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123",
"region": "us-east-1"
},
"integration_account_id": "my-account-id",
"integration_provider": "openai",
"model_name": "gpt-4o",
"vertex_ai": {
"location": "us-central1",
"project": "my-gcp-project"
}
},
"target": {
"application_name": "my-llm-app",
"enabled": true,
"eval_scope": "span",
"experiment_project_ids": [],
"filter": "@service:my-service",
"root_spans_only": true,
"sampling_percentage": 50
},
"updated_at": "2024-01-15T10:30:00Z"
},
"id": "my-custom-evaluator",
"type": "evaluator_config"
}
}Bad Request
API error response.
errors [required]A list of errors.
detailA human-readable explanation specific to this occurrence of the error.
metaNon-standard meta-information about the error
sourceReferences to the source of the error.
headerA string indicating the name of a single request header which caused the error.
parameterA string indicating which URI query parameter caused the error.
pointerA JSON pointer to the value in the request document that caused the error.
statusStatus code of the response.
titleShort human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Unauthorized
API error response.
errors [required]A list of errors.
detailA human-readable explanation specific to this occurrence of the error.
metaNon-standard meta-information about the error
sourceReferences to the source of the error.
headerA string indicating the name of a single request header which caused the error.
parameterA string indicating which URI query parameter caused the error.
pointerA JSON pointer to the value in the request document that caused the error.
statusStatus code of the response.
titleShort human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Forbidden
API error response.
errors [required]A list of errors.
detailA human-readable explanation specific to this occurrence of the error.
metaNon-standard meta-information about the error
sourceReferences to the source of the error.
headerA string indicating the name of a single request header which caused the error.
parameterA string indicating which URI query parameter caused the error.
pointerA JSON pointer to the value in the request document that caused the error.
statusStatus code of the response.
titleShort human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Not Found
API error response.
errors [required]A list of errors.
detailA human-readable explanation specific to this occurrence of the error.
metaNon-standard meta-information about the error
sourceReferences to the source of the error.
headerA string indicating the name of a single request header which caused the error.
parameterA string indicating which URI query parameter caused the error.
pointerA JSON pointer to the value in the request document that caused the error.
statusStatus code of the response.
titleShort human-readable summary of the error.
{
"errors": [
{
"detail": "Missing required attribute in body",
"meta": {},
"source": {
"header": "Authorization",
"parameter": "limit",
"pointer": "/data/attributes/title"
},
"status": "400",
"title": "Bad Request"
}
]
}Too many requests
API error response.
errors [required]A list of errors.
{
"errors": [
"Bad Request"
]
}코드 예시
# Path parameters
export eval_name="my-custom-evaluator"
# Set your Datadog site and credentials
export DD_SITE="datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X GET "https://api.${DD_SITE}/api/unstable/llm-obs/config/evaluators/custom/${eval_name}" \
-H "Accept: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}"# Path parameters
export eval_name="my-custom-evaluator"
# Set your Datadog site and credentials
export DD_SITE="us3.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X GET "https://api.${DD_SITE}/api/unstable/llm-obs/config/evaluators/custom/${eval_name}" \
-H "Accept: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}"# Path parameters
export eval_name="my-custom-evaluator"
# Set your Datadog site and credentials
export DD_SITE="us5.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X GET "https://api.${DD_SITE}/api/unstable/llm-obs/config/evaluators/custom/${eval_name}" \
-H "Accept: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}"# Path parameters
export eval_name="my-custom-evaluator"
# Set your Datadog site and credentials
export DD_SITE="datadoghq.eu"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X GET "https://api.${DD_SITE}/api/unstable/llm-obs/config/evaluators/custom/${eval_name}" \
-H "Accept: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}"# Path parameters
export eval_name="my-custom-evaluator"
# Set your Datadog site and credentials
export DD_SITE="ap1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X GET "https://api.${DD_SITE}/api/unstable/llm-obs/config/evaluators/custom/${eval_name}" \
-H "Accept: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}"# Path parameters
export eval_name="my-custom-evaluator"
# Set your Datadog site and credentials
export DD_SITE="ap2.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X GET "https://api.${DD_SITE}/api/unstable/llm-obs/config/evaluators/custom/${eval_name}" \
-H "Accept: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}"# Path parameters
export eval_name="my-custom-evaluator"
# Set your Datadog site and credentials
export DD_SITE="uk1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X GET "https://api.${DD_SITE}/api/unstable/llm-obs/config/evaluators/custom/${eval_name}" \
-H "Accept: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}"# Path parameters
export eval_name="my-custom-evaluator"
# Set your Datadog site and credentials
export DD_SITE="ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X GET "https://api.${DD_SITE}/api/unstable/llm-obs/config/evaluators/custom/${eval_name}" \
-H "Accept: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}"# Path parameters
export eval_name="my-custom-evaluator"
# Set your Datadog site and credentials
export DD_SITE="us2.ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"
curl -X GET "https://api.${DD_SITE}/api/unstable/llm-obs/config/evaluators/custom/${eval_name}" \
-H "Accept: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}"// Get a custom evaluator configuration returns "OK" response
package main
import (
"context"
"encoding/json"
"fmt"
"os"
"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)
func main() {
ctx := datadog.NewDefaultContext(context.Background())
ctx = context.WithValue(ctx, datadog.ContextAccessToken, os.Getenv("DD_BEARER_TOKEN"))
configuration := datadog.NewConfiguration()
configuration.SetUnstableOperationEnabled("v2.GetLLMObsCustomEvalConfig", true)
apiClient := datadog.NewAPIClient(configuration)
api := datadogV2.NewAgentObservabilityApi(apiClient)
resp, r, err := api.GetLLMObsCustomEvalConfig(ctx, "eval_name")
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `AgentObservabilityApi.GetLLMObsCustomEvalConfig`: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
responseContent, _ := json.MarshalIndent(resp, "", " ")
fmt.Fprintf(os.Stdout, "Response from `AgentObservabilityApi.GetLLMObsCustomEvalConfig`:\n%s\n", responseContent)
}
Instructions
First install the library and its dependencies and then save the example to main.go and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"// Get a custom evaluator configuration returns "OK" response
import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.AgentObservabilityApi;
import com.datadog.api.client.v2.model.LLMObsCustomEvalConfigResponse;
public class Example {
public static void main(String[] args) {
ApiClient defaultClient = ApiClient.getDefaultApiClient();
defaultClient.setAccessToken(System.getenv("DD_BEARER_TOKEN"));
defaultClient.setUnstableOperationEnabled("v2.getLLMObsCustomEvalConfig", true);
AgentObservabilityApi apiInstance = new AgentObservabilityApi(defaultClient);
try {
LLMObsCustomEvalConfigResponse result =
apiInstance.getLLMObsCustomEvalConfig("my-custom-evaluator");
System.out.println(result);
} catch (ApiException e) {
System.err.println("Exception when calling AgentObservabilityApi#getLLMObsCustomEvalConfig");
System.err.println("Status code: " + e.getCode());
System.err.println("Reason: " + e.getResponseBody());
System.err.println("Response headers: " + e.getResponseHeaders());
e.printStackTrace();
}
}
}
Instructions
First install the library and its dependencies and then save the example to Example.java and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java""""
Get a custom evaluator configuration returns "OK" response
"""
from os import environ
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.agent_observability_api import AgentObservabilityApi
configuration = Configuration()
configuration.access_token = environ["DD_BEARER_TOKEN"]
configuration.unstable_operations["get_llm_obs_custom_eval_config"] = True
with ApiClient(configuration) as api_client:
api_instance = AgentObservabilityApi(api_client)
response = api_instance.get_llm_obs_custom_eval_config(
eval_name="eval_name",
)
print(response)
Instructions
First install the library and its dependencies and then save the example to example.py and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"# Get a custom evaluator configuration returns "OK" response
require "datadog_api_client"
DatadogAPIClient.configure do |config|
config.access_token = ENV["DD_BEARER_TOKEN"]
config.unstable_operations["v2.get_llm_obs_custom_eval_config".to_sym] = true
end
api_instance = DatadogAPIClient::V2::AgentObservabilityAPI.new
p api_instance.get_llm_obs_custom_eval_config("eval_name")
Instructions
First install the library and its dependencies and then save the example to example.rb and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"// Get a custom evaluator configuration returns "OK" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_agent_observability::AgentObservabilityAPI;
#[tokio::main]
async fn main() {
let mut configuration = datadog::Configuration::new();
configuration.set_unstable_operation_enabled("v2.get_llm_obs_custom_eval_config", true);
let api = AgentObservabilityAPI::with_config(configuration);
let resp = api
.get_llm_obs_custom_eval_config("eval_name".to_string())
.await;
if let Ok(value) = resp {
println!("{:#?}", value);
} else {
println!("{:#?}", resp.unwrap_err());
}
}
Instructions
First install the library and its dependencies and then save the example to src/main.rs and run following commands:
DD_SITE="datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us3.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us5.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="datadoghq.eu" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ap1.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ap2.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="uk1.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo runDD_SITE="us2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run/**
* Get a custom evaluator configuration returns "OK" response
*/
import { client, v2 } from "@datadog/datadog-api-client";
const configuration = client.createConfiguration({
authMethods: {
AuthZ: {
accessToken: process.env.DD_BEARER_TOKEN as string,
},
},
});
configuration.unstableOperations["v2.getLLMObsCustomEvalConfig"] = true;
const apiInstance = new v2.AgentObservabilityApi(configuration);
const params: v2.AgentObservabilityApiGetLLMObsCustomEvalConfigRequest = {
evalName: "eval_name",
};
apiInstance
.getLLMObsCustomEvalConfig(params)
.then((data: v2.LLMObsCustomEvalConfigResponse) => {
console.log(
"API called successfully. Returned data: " + JSON.stringify(data)
);
})
.catch((error: any) => console.error(error));
Instructions
First install the library and its dependencies and then save the example to example.ts and run following commands:
DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"