For AI agents: A markdown version of this page is available at https://dnd37xeb23o8n.cloudfront.net/stefon.simmons/docs-astro-web-10435/api/latest/agent-observability/upload-records-to-an-agent-observability-dataset.md. A documentation index is available at /llms.txt.
Language
Datadog site

Upload records to an Agent Observability dataset

WARNINGNote: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.
POSThttps://api.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.us3.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.us5.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.datadoghq.eu/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.ap1.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.ap2.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.uk1.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.ddog-gov.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.us2.ddog-gov.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload

Overview

Upload records to a dataset from a file. The request is a multipart/form-data upload containing a single file part. Currently only CSV is supported. The CSV must include an input column. Optional columns are id, expected_output, metadata, and tags.

The response is a Server-Sent Events stream (text/event-stream) emitting progress updates while records are processed. The stream emits the following named events:

  • progress: incremental record counts written so far.
  • completed: terminal event with a JSON body containing records_created.
  • error: terminal event with a JSON body containing an error message.

Arguments

Path Parameters

Name
Type
Description
project_id [required]
string

The ID of the Agent Observability project.

dataset_id [required]
string

The ID of the Agent Observability dataset.

Query Strings

Name
Type
Description
deduplicate
boolean

Whether to skip records whose input already exists in the dataset. Defaults to false.

Default: false
overwrite
boolean

Whether to overwrite existing records that share the same user-provided id. Defaults to true.

Default: true
tags
array

Tags to apply to every uploaded record, in addition to any tags defined on individual rows. Can be repeated, e.g. tags=env:prod&tags=team:ai.

include[user_data]
boolean

Whether to enrich the response with user metadata.

Request

Body Data (required)

Multipart upload payload containing the records file.

Field
Type
Description
file
binary

The records file to upload. Currently only CSV is supported. The file must include an input column. Optional columns include id, expected_output, metadata, and tags.

Response

OK

Code Example

# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"

# Set your Datadog site and credentials
export DD_SITE="datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \
-H "Accept: application/json" \
-H "Content-Type: multipart/form-data" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-F file=@string
# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"

# Set your Datadog site and credentials
export DD_SITE="us3.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \
-H "Accept: application/json" \
-H "Content-Type: multipart/form-data" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-F file=@string
# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"

# Set your Datadog site and credentials
export DD_SITE="us5.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \
-H "Accept: application/json" \
-H "Content-Type: multipart/form-data" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-F file=@string
# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"

# Set your Datadog site and credentials
export DD_SITE="datadoghq.eu"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \
-H "Accept: application/json" \
-H "Content-Type: multipart/form-data" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-F file=@string
# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"

# Set your Datadog site and credentials
export DD_SITE="ap1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \
-H "Accept: application/json" \
-H "Content-Type: multipart/form-data" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-F file=@string
# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"

# Set your Datadog site and credentials
export DD_SITE="ap2.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \
-H "Accept: application/json" \
-H "Content-Type: multipart/form-data" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-F file=@string
# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"

# Set your Datadog site and credentials
export DD_SITE="uk1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \
-H "Accept: application/json" \
-H "Content-Type: multipart/form-data" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-F file=@string
# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"

# Set your Datadog site and credentials
export DD_SITE="ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \
-H "Accept: application/json" \
-H "Content-Type: multipart/form-data" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-F file=@string
# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"

# Set your Datadog site and credentials
export DD_SITE="us2.ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \
-H "Accept: application/json" \
-H "Content-Type: multipart/form-data" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-F file=@string