For AI agents: A markdown version of this page is available at https://dnd37xeb23o8n.cloudfront.net/stefon.simmons/docs-astro-web-10435/ko/api/latest/agent-observability/run-an-llm-inference.md. A documentation index is available at /llms.txt.
homeHomedocs문서apiAPI
Language
Datadog site

LLM 추론 실행

경고Note: This endpoint is in Preview and is subject to change. If you have any feedback, contact Datadog support.
POSThttps://api.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inferencehttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inferencehttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inferencehttps://api.datadoghq.eu/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inferencehttps://api.ap1.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inferencehttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inferencehttps://api.uk1.datadoghq.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inferencehttps://api.ddog-gov.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inferencehttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/integrations/{integration}/{account_id}/inference

개요

지정된 통합 및 계정을 통해 LLM 추론 요청을 실행하고 모델 응답과 토큰 사용량을 반환합니다.

인수

경로 파라미터

이름
유형
설명
integration [required]
enum

The name of the LLM integration.

Allowed enum values: openai, amazon_bedrock, anthropic, azure_openai, vertex_ai, llmproxy
account_id [required]
string

The ID of the integration account.

요청

Body Data (required)

추론 요청 파라미터입니다.

항목
유형
설명
anthropic_metadata
object

LLM 추론 요청을 위한 파라미터입니다.

azure_openai_metadata
object

Azure OpenAI-specific metadata for an integration account or inference request.

bedrock_metadata
object

Amazon Bedrock-specific metadata for an inference request.

frequency_penalty
double

Penalty for token frequency to reduce repetition.

json_schema
string

JSON schema for structured output, if supported by the model.

max_completion_tokens
int64

Maximum number of completion tokens to generate (alternative to max_tokens for some providers).

max_tokens
int64

Maximum number of tokens to generate.

messages [required]
[object]

List of messages in an inference conversation.

model_id [required]
string

The model identifier to use for inference.

openai_metadata
object

OpenAI-specific metadata for an inference request.

presence_penalty
double

Penalty for token presence to encourage topic diversity.

temperature
double

Sampling temperature between 0 and 2. Higher values produce more random output.

tools
[object]

List of tools available to the model.

top_k
int64

Top-K sampling parameter.

top_p
double

Nucleus sampling probability mass.

vertex_ai_metadata
object

Vertex AI-specific metadata for an integration account or inference request.

응답

OK

The result of an LLM inference request, including input parameters and the model response.

항목
유형
설명
anthropic_metadata
object

Anthropic-specific metadata for an inference request.

azure_openai_metadata
object

Azure OpenAI-specific metadata for an integration account or inference request.

bedrock_metadata
object

Amazon Bedrock-specific metadata for an inference request.

error_response
object

Error details returned when an inference provider returns an error.

frequency_penalty
double

Frequency penalty that was applied.

json_schema
string

JSON schema that was applied for structured output.

max_completion_tokens
int64

Maximum number of completion tokens that were configured.

max_tokens
int64

Maximum number of tokens that were configured.

messages [required]
[object]

List of messages in an inference conversation.

model_id [required]
string

The model identifier used for inference.

openai_metadata
object

OpenAI-specific metadata for an inference request.

presence_penalty
double

Presence penalty that was applied.

response [required]
object

The output of a completed LLM inference call.

temperature
double

Sampling temperature that was used.

tools
[object]

List of tools available to the model.

top_k
int64

Top-K sampling parameter that was used.

top_p
double

Nucleus sampling parameter that was used.

vertex_ai_metadata
object

Vertex AI-specific metadata for an integration account or inference request.

코드 예시

LLM 추론 실행
# Path parameters
export integration="openai"
export account_id="account-abc123"

# Set your Datadog site and credentials
export DD_SITE="datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/integrations/${integration}/${account_id}/inference" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "max_tokens": 256,
  "messages": [
    {
      "content": "What is the capital of France?",
      "role": "user"
    }
  ],
  "model_id": "gpt-4o",
  "temperature": 0.7
}
EOF
# Path parameters
export integration="openai"
export account_id="account-abc123"

# Set your Datadog site and credentials
export DD_SITE="us3.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/integrations/${integration}/${account_id}/inference" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "max_tokens": 256,
  "messages": [
    {
      "content": "What is the capital of France?",
      "role": "user"
    }
  ],
  "model_id": "gpt-4o",
  "temperature": 0.7
}
EOF
# Path parameters
export integration="openai"
export account_id="account-abc123"

# Set your Datadog site and credentials
export DD_SITE="us5.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/integrations/${integration}/${account_id}/inference" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "max_tokens": 256,
  "messages": [
    {
      "content": "What is the capital of France?",
      "role": "user"
    }
  ],
  "model_id": "gpt-4o",
  "temperature": 0.7
}
EOF
# Path parameters
export integration="openai"
export account_id="account-abc123"

# Set your Datadog site and credentials
export DD_SITE="datadoghq.eu"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/integrations/${integration}/${account_id}/inference" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "max_tokens": 256,
  "messages": [
    {
      "content": "What is the capital of France?",
      "role": "user"
    }
  ],
  "model_id": "gpt-4o",
  "temperature": 0.7
}
EOF
# Path parameters
export integration="openai"
export account_id="account-abc123"

# Set your Datadog site and credentials
export DD_SITE="ap1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/integrations/${integration}/${account_id}/inference" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "max_tokens": 256,
  "messages": [
    {
      "content": "What is the capital of France?",
      "role": "user"
    }
  ],
  "model_id": "gpt-4o",
  "temperature": 0.7
}
EOF
# Path parameters
export integration="openai"
export account_id="account-abc123"

# Set your Datadog site and credentials
export DD_SITE="ap2.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/integrations/${integration}/${account_id}/inference" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "max_tokens": 256,
  "messages": [
    {
      "content": "What is the capital of France?",
      "role": "user"
    }
  ],
  "model_id": "gpt-4o",
  "temperature": 0.7
}
EOF
# Path parameters
export integration="openai"
export account_id="account-abc123"

# Set your Datadog site and credentials
export DD_SITE="uk1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/integrations/${integration}/${account_id}/inference" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "max_tokens": 256,
  "messages": [
    {
      "content": "What is the capital of France?",
      "role": "user"
    }
  ],
  "model_id": "gpt-4o",
  "temperature": 0.7
}
EOF
# Path parameters
export integration="openai"
export account_id="account-abc123"

# Set your Datadog site and credentials
export DD_SITE="ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/integrations/${integration}/${account_id}/inference" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "max_tokens": 256,
  "messages": [
    {
      "content": "What is the capital of France?",
      "role": "user"
    }
  ],
  "model_id": "gpt-4o",
  "temperature": 0.7
}
EOF
# Path parameters
export integration="openai"
export account_id="account-abc123"

# Set your Datadog site and credentials
export DD_SITE="us2.ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/integrations/${integration}/${account_id}/inference" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "max_tokens": 256,
  "messages": [
    {
      "content": "What is the capital of France?",
      "role": "user"
    }
  ],
  "model_id": "gpt-4o",
  "temperature": 0.7
}
EOF