Search Agent Observability spans - Agent Observability

WARNINGNote: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POSThttps://api.datadoghq.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.datadoghq.eu/api/v2/llm-obs/v1/spans/events/searchhttps://api.ap1.datadoghq.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.uk1.datadoghq.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.ddog-gov.com/api/v2/llm-obs/v1/spans/events/searchhttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/spans/events/search

Overview

Search Agent Observability spans using structured filters in the request body.

Request

Body Data (required)

Search spans payload.

Field

Type

Description

data [required]

object

Data object for an Agent Observability spans search request.

{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "query": "@session_id:abc123def456",
        "span_id": "abc123def456",
        "span_kind": "llm",
        "span_name": "llm_call",
        "tags": {
          "<any-key>": "string"
        },
        "to": "now",
        "trace_id": "trace-9a8b7c6d5e4f"
      },
      "options": {
        "include_attachments": true,
        "time_offset": 0
      },
      "page": {
        "cursor": "eyJzdGFydCI6MTAwfQ==",
        "limit": 10
      },
      "sort": "-start_ns"
    },
    "type": "spans"
  }
}

Response

Response containing a list of Agent Observability spans.

Field

Type

Description

data [required]

[object]

List of spans matching the query.

attributes [required]

object

Attributes of an Agent Observability span.

duration [required]

double

Duration of the span in nanoseconds.

evaluation

object

Evaluation metrics keyed by evaluator name.

input

object

Input or output content of an Agent Observability span.

intent

string

Detected intent of the span.

metadata

object

Arbitrary metadata associated with the span.

metrics

object

Numeric metrics associated with the span (e.g., token counts).

ml_app [required]

string

Name of the ML application this span belongs to.

model_name

string

Name of the model used in this span.

model_provider

string

Provider of the model used in this span.

name [required]

string

Name of the span.

output

object

Input or output content of an Agent Observability span.

parent_id

string

Identifier of the parent span, if any.

span_id [required]

string

Unique identifier of the span.

span_kind [required]

string

Kind of span (e.g., llm, agent, tool, task, workflow).

start_ns [required]

int64

Start time of the span in nanoseconds since Unix epoch.

status [required]

string

Status of the span (e.g., ok, error).

tags

[string]

Tags associated with the span.

tool_definitions

[object]

Tool definitions available to the span.

trace_id [required]

string

Trace identifier this span belongs to.

id [required]

string

Unique identifier of the span.

type [required]

enum

Resource type for an Agent Observability span.

Allowed enum values: span

links

object

Pagination links accompanying the spans response.

meta [required]

object

Metadata accompanying the spans response.

{
  "data": [
    {
      "attributes": {
        "duration": 1500000000,
        "evaluation": {
          "<any-key>": {
            "assessment": "pass",
            "eval_metric_type": "score",
            "reasoning": "string",
            "status": "string",
            "tags": [],
            "value": "undefined"
          }
        },
        "input": {
          "messages": [
            {
              "content": "string",
              "id": "string",
              "role": "string",
              "tool_calls": [
                {
                  "arguments": {},
                  "name": "string",
                  "tool_id": "string",
                  "type": "string"
                }
              ],
              "tool_results": [
                {
                  "name": "string",
                  "result": "string",
                  "tool_id": "string",
                  "type": "string"
                }
              ]
            }
          ],
          "value": "string"
        },
        "intent": "string",
        "metadata": {},
        "metrics": {
          "<any-key>": "number"
        },
        "ml_app": "my-llm-app",
        "model_name": "gpt-4o",
        "model_provider": "openai",
        "name": "llm_call",
        "output": {
          "messages": [
            {
              "content": "string",
              "id": "string",
              "role": "string",
              "tool_calls": [
                {
                  "arguments": {},
                  "name": "string",
                  "tool_id": "string",
                  "type": "string"
                }
              ],
              "tool_results": [
                {
                  "name": "string",
                  "result": "string",
                  "tool_id": "string",
                  "type": "string"
                }
              ]
            }
          ],
          "value": "string"
        },
        "parent_id": "string",
        "span_id": "abc123def456",
        "span_kind": "llm",
        "start_ns": 1705314600000000000,
        "status": "ok",
        "tags": [],
        "tool_definitions": [
          {
            "description": "string",
            "name": "string",
            "schema": {},
            "version": "string"
          }
        ],
        "trace_id": "trace-9a8b7c6d5e4f"
      },
      "id": "abc123def456",
      "type": "span"
    }
  ],
  "links": {
    "next": "https://api.datadoghq.com/api/v2/llm-obs/v1/spans/events?page[cursor]=eyJzdGFydCI6MTAwfQ=="
  },
  "meta": {
    "elapsed": 132,
    "page": {
      "after": "eyJzdGFydCI6MTAwfQ=="
    },
    "request_id": "req-abc123",
    "status": "done"
  }
}

Bad Request

API error response.

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Code Example

Search Agent Observability spans

## Search spans for an ML app

# Set your Datadog site and credentials
export DD_SITE="datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "span_kind": "llm",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 10
      }
    },
    "type": "spans"
  }
}
EOF

## Search all spans in a session

# Set your Datadog site and credentials
export DD_SITE="datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "query": "@session_id:abc123def456",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 50
      }
    },
    "type": "spans"
  }
}
EOF

## Search spans for an ML app

# Set your Datadog site and credentials
export DD_SITE="us3.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "span_kind": "llm",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 10
      }
    },
    "type": "spans"
  }
}
EOF

## Search all spans in a session

# Set your Datadog site and credentials
export DD_SITE="us3.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "query": "@session_id:abc123def456",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 50
      }
    },
    "type": "spans"
  }
}
EOF

## Search spans for an ML app

# Set your Datadog site and credentials
export DD_SITE="us5.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "span_kind": "llm",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 10
      }
    },
    "type": "spans"
  }
}
EOF

## Search all spans in a session

# Set your Datadog site and credentials
export DD_SITE="us5.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "query": "@session_id:abc123def456",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 50
      }
    },
    "type": "spans"
  }
}
EOF

## Search spans for an ML app

# Set your Datadog site and credentials
export DD_SITE="datadoghq.eu"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "span_kind": "llm",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 10
      }
    },
    "type": "spans"
  }
}
EOF

## Search all spans in a session

# Set your Datadog site and credentials
export DD_SITE="datadoghq.eu"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "query": "@session_id:abc123def456",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 50
      }
    },
    "type": "spans"
  }
}
EOF

## Search spans for an ML app

# Set your Datadog site and credentials
export DD_SITE="ap1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "span_kind": "llm",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 10
      }
    },
    "type": "spans"
  }
}
EOF

## Search all spans in a session

# Set your Datadog site and credentials
export DD_SITE="ap1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "query": "@session_id:abc123def456",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 50
      }
    },
    "type": "spans"
  }
}
EOF

## Search spans for an ML app

# Set your Datadog site and credentials
export DD_SITE="ap2.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "span_kind": "llm",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 10
      }
    },
    "type": "spans"
  }
}
EOF

## Search all spans in a session

# Set your Datadog site and credentials
export DD_SITE="ap2.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "query": "@session_id:abc123def456",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 50
      }
    },
    "type": "spans"
  }
}
EOF

## Search spans for an ML app

# Set your Datadog site and credentials
export DD_SITE="uk1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "span_kind": "llm",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 10
      }
    },
    "type": "spans"
  }
}
EOF

## Search all spans in a session

# Set your Datadog site and credentials
export DD_SITE="uk1.datadoghq.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "query": "@session_id:abc123def456",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 50
      }
    },
    "type": "spans"
  }
}
EOF

## Search spans for an ML app

# Set your Datadog site and credentials
export DD_SITE="ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "span_kind": "llm",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 10
      }
    },
    "type": "spans"
  }
}
EOF

## Search all spans in a session

# Set your Datadog site and credentials
export DD_SITE="ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "query": "@session_id:abc123def456",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 50
      }
    },
    "type": "spans"
  }
}
EOF

## Search spans for an ML app

# Set your Datadog site and credentials
export DD_SITE="us2.ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "ml_app": "my-llm-app",
        "span_kind": "llm",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 10
      }
    },
    "type": "spans"
  }
}
EOF

## Search all spans in a session

# Set your Datadog site and credentials
export DD_SITE="us2.ddog-gov.com"
# Use a Personal Access Token or Service Access Token
export DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>"

curl -X POST "https://api.${DD_SITE}/api/v2/llm-obs/v1/spans/events/search" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${DD_BEARER_TOKEN}" \
-d @- << EOF
{
  "data": {
    "attributes": {
      "filter": {
        "from": "now-900s",
        "query": "@session_id:abc123def456",
        "to": "now"
      },
      "options": {
        "include_attachments": true
      },
      "page": {
        "limit": 50
      }
    },
    "type": "spans"
  }
}
EOF

Search Agent Observability spans

// Search Agent Observability spans returns "OK" response

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"os"

	"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
	"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)

func main() {
	body := datadogV2.LLMObsSearchSpansRequest{
		Data: datadogV2.LLMObsSearchSpansRequestData{
			Attributes: datadogV2.LLMObsSearchSpansRequestAttributes{
				Filter: &datadogV2.LLMObsSpanFilter{
					From:     datadog.PtrString("now-900s"),
					MlApp:    datadog.PtrString("my-llm-app"),
					Query:    datadog.PtrString("@session_id:abc123def456"),
					SpanId:   datadog.PtrString("abc123def456"),
					SpanKind: datadog.PtrString("llm"),
					SpanName: datadog.PtrString("llm_call"),
					To:       datadog.PtrString("now"),
					TraceId:  datadog.PtrString("trace-9a8b7c6d5e4f"),
				},
				Options: &datadogV2.LLMObsSpanSearchOptions{
					IncludeAttachments: datadog.PtrBool(true),
					TimeOffset:         datadog.PtrInt64(0),
				},
				Page: &datadogV2.LLMObsSpanPageQuery{
					Cursor: datadog.PtrString("eyJzdGFydCI6MTAwfQ=="),
					Limit:  datadog.PtrInt64(10),
				},
				Sort: datadog.PtrString("-start_ns"),
			},
			Type: datadogV2.LLMOBSSEARCHSPANSREQUESTTYPE_SPANS,
		},
	}
	ctx := datadog.NewDefaultContext(context.Background())
	ctx = context.WithValue(ctx, datadog.ContextAccessToken, os.Getenv("DD_BEARER_TOKEN"))
	configuration := datadog.NewConfiguration()
	configuration.SetUnstableOperationEnabled("v2.SearchLLMObsSpans", true)
	apiClient := datadog.NewAPIClient(configuration)
	api := datadogV2.NewAgentObservabilityApi(apiClient)
	resp, r, err := api.SearchLLMObsSpans(ctx, body)

	if err != nil {
		fmt.Fprintf(os.Stderr, "Error when calling `AgentObservabilityApi.SearchLLMObsSpans`: %v\n", err)
		fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
	}

	responseContent, _ := json.MarshalIndent(resp, "", "  ")
	fmt.Fprintf(os.Stdout, "Response from `AgentObservabilityApi.SearchLLMObsSpans`:\n%s\n", responseContent)
}

Instructions

First install the library and its dependencies and then save the example to main.go and run following commands:

DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"

DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"

DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"

DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"

DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"

DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"

DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"

DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"

DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" go run "main.go"

Search Agent Observability spans

// Search Agent Observability spans returns "OK" response

import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.AgentObservabilityApi;
import com.datadog.api.client.v2.model.LLMObsSearchSpansRequest;
import com.datadog.api.client.v2.model.LLMObsSearchSpansRequestAttributes;
import com.datadog.api.client.v2.model.LLMObsSearchSpansRequestData;
import com.datadog.api.client.v2.model.LLMObsSearchSpansRequestType;
import com.datadog.api.client.v2.model.LLMObsSpanFilter;
import com.datadog.api.client.v2.model.LLMObsSpanPageQuery;
import com.datadog.api.client.v2.model.LLMObsSpanSearchOptions;
import com.datadog.api.client.v2.model.LLMObsSpansResponse;

public class Example {
  public static void main(String[] args) {
    ApiClient defaultClient = ApiClient.getDefaultApiClient();
    defaultClient.setAccessToken(System.getenv("DD_BEARER_TOKEN"));
    defaultClient.setUnstableOperationEnabled("v2.searchLLMObsSpans", true);
    AgentObservabilityApi apiInstance = new AgentObservabilityApi(defaultClient);

    LLMObsSearchSpansRequest body =
        new LLMObsSearchSpansRequest()
            .data(
                new LLMObsSearchSpansRequestData()
                    .attributes(
                        new LLMObsSearchSpansRequestAttributes()
                            .filter(
                                new LLMObsSpanFilter()
                                    .from("now-900s")
                                    .mlApp("my-llm-app")
                                    .query("@session_id:abc123def456")
                                    .spanId("abc123def456")
                                    .spanKind("llm")
                                    .spanName("llm_call")
                                    .to("now")
                                    .traceId("trace-9a8b7c6d5e4f"))
                            .options(
                                new LLMObsSpanSearchOptions()
                                    .includeAttachments(true)
                                    .timeOffset(0L))
                            .page(
                                new LLMObsSpanPageQuery().cursor("eyJzdGFydCI6MTAwfQ==").limit(10L))
                            .sort("-start_ns"))
                    .type(LLMObsSearchSpansRequestType.SPANS));

    try {
      LLMObsSpansResponse result = apiInstance.searchLLMObsSpans(body);
      System.out.println(result);
    } catch (ApiException e) {
      System.err.println("Exception when calling AgentObservabilityApi#searchLLMObsSpans");
      System.err.println("Status code: " + e.getCode());
      System.err.println("Reason: " + e.getResponseBody());
      System.err.println("Response headers: " + e.getResponseHeaders());
      e.printStackTrace();
    }
  }
}

Instructions

First install the library and its dependencies and then save the example to Example.java and run following commands:

DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"

DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"

DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"

DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"

DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"

DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"

DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"

DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"

DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" java "Example.java"

Search Agent Observability spans

"""
Search Agent Observability spans returns "OK" response
"""

from os import environ
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.agent_observability_api import AgentObservabilityApi
from datadog_api_client.v2.model.llm_obs_search_spans_request import LLMObsSearchSpansRequest
from datadog_api_client.v2.model.llm_obs_search_spans_request_attributes import LLMObsSearchSpansRequestAttributes
from datadog_api_client.v2.model.llm_obs_search_spans_request_data import LLMObsSearchSpansRequestData
from datadog_api_client.v2.model.llm_obs_search_spans_request_type import LLMObsSearchSpansRequestType
from datadog_api_client.v2.model.llm_obs_span_filter import LLMObsSpanFilter
from datadog_api_client.v2.model.llm_obs_span_page_query import LLMObsSpanPageQuery
from datadog_api_client.v2.model.llm_obs_span_search_options import LLMObsSpanSearchOptions

body = LLMObsSearchSpansRequest(
    data=LLMObsSearchSpansRequestData(
        attributes=LLMObsSearchSpansRequestAttributes(
            filter=LLMObsSpanFilter(
                _from="now-900s",
                ml_app="my-llm-app",
                query="@session_id:abc123def456",
                span_id="abc123def456",
                span_kind="llm",
                span_name="llm_call",
                to="now",
                trace_id="trace-9a8b7c6d5e4f",
            ),
            options=LLMObsSpanSearchOptions(
                include_attachments=True,
                time_offset=0,
            ),
            page=LLMObsSpanPageQuery(
                cursor="eyJzdGFydCI6MTAwfQ==",
                limit=10,
            ),
            sort="-start_ns",
        ),
        type=LLMObsSearchSpansRequestType.SPANS,
    ),
)

configuration = Configuration()
configuration.access_token = environ["DD_BEARER_TOKEN"]
configuration.unstable_operations["search_llm_obs_spans"] = True
with ApiClient(configuration) as api_client:
    api_instance = AgentObservabilityApi(api_client)
    response = api_instance.search_llm_obs_spans(body=body)

    print(response)

Instructions

First install the library and its dependencies and then save the example to example.py and run following commands:

DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"

DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"

DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"

DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"

DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"

DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"

DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"

DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"

DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" python3 "example.py"

Search Agent Observability spans

# Search Agent Observability spans returns "OK" response

require "datadog_api_client"
DatadogAPIClient.configure do |config|
  config.access_token = ENV["DD_BEARER_TOKEN"]
  config.unstable_operations["v2.search_llm_obs_spans".to_sym] = true
end
api_instance = DatadogAPIClient::V2::AgentObservabilityAPI.new

body = DatadogAPIClient::V2::LLMObsSearchSpansRequest.new({
  data: DatadogAPIClient::V2::LLMObsSearchSpansRequestData.new({
    attributes: DatadogAPIClient::V2::LLMObsSearchSpansRequestAttributes.new({
      filter: DatadogAPIClient::V2::LLMObsSpanFilter.new({
        from: "now-900s",
        ml_app: "my-llm-app",
        query: "@session_id:abc123def456",
        span_id: "abc123def456",
        span_kind: "llm",
        span_name: "llm_call",
        to: "now",
        trace_id: "trace-9a8b7c6d5e4f",
      }),
      options: DatadogAPIClient::V2::LLMObsSpanSearchOptions.new({
        include_attachments: true,
        time_offset: 0,
      }),
      page: DatadogAPIClient::V2::LLMObsSpanPageQuery.new({
        cursor: "eyJzdGFydCI6MTAwfQ==",
        limit: 10,
      }),
      sort: "-start_ns",
    }),
    type: DatadogAPIClient::V2::LLMObsSearchSpansRequestType::SPANS,
  }),
})
p api_instance.search_llm_obs_spans(body)

Instructions

First install the library and its dependencies and then save the example to example.rb and run following commands:

DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"

DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"

DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"

DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"

DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"

DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"

DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"

DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"

DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" rb "example.rb"

Search Agent Observability spans

// Search Agent Observability spans returns "OK" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_agent_observability::AgentObservabilityAPI;
use datadog_api_client::datadogV2::model::LLMObsSearchSpansRequest;
use datadog_api_client::datadogV2::model::LLMObsSearchSpansRequestAttributes;
use datadog_api_client::datadogV2::model::LLMObsSearchSpansRequestData;
use datadog_api_client::datadogV2::model::LLMObsSearchSpansRequestType;
use datadog_api_client::datadogV2::model::LLMObsSpanFilter;
use datadog_api_client::datadogV2::model::LLMObsSpanPageQuery;
use datadog_api_client::datadogV2::model::LLMObsSpanSearchOptions;

#[tokio::main]
async fn main() {
    let body = LLMObsSearchSpansRequest::new(LLMObsSearchSpansRequestData::new(
        LLMObsSearchSpansRequestAttributes::new()
            .filter(
                LLMObsSpanFilter::new()
                    .from("now-900s".to_string())
                    .ml_app("my-llm-app".to_string())
                    .query("@session_id:abc123def456".to_string())
                    .span_id("abc123def456".to_string())
                    .span_kind("llm".to_string())
                    .span_name("llm_call".to_string())
                    .to("now".to_string())
                    .trace_id("trace-9a8b7c6d5e4f".to_string()),
            )
            .options(
                LLMObsSpanSearchOptions::new()
                    .include_attachments(true)
                    .time_offset(0),
            )
            .page(
                LLMObsSpanPageQuery::new()
                    .cursor("eyJzdGFydCI6MTAwfQ==".to_string())
                    .limit(10),
            )
            .sort("-start_ns".to_string()),
        LLMObsSearchSpansRequestType::SPANS,
    ));
    let mut configuration = datadog::Configuration::new();
    configuration.set_unstable_operation_enabled("v2.search_llm_obs_spans", true);
    let api = AgentObservabilityAPI::with_config(configuration);
    let resp = api.search_llm_obs_spans(body).await;
    if let Ok(value) = resp {
        println!("{:#?}", value);
    } else {
        println!("{:#?}", resp.unwrap_err());
    }
}

Instructions

First install the library and its dependencies and then save the example to src/main.rs and run following commands:

DD_SITE="datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

DD_SITE="us3.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

DD_SITE="us5.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

DD_SITE="datadoghq.eu" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

DD_SITE="ap1.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

DD_SITE="ap2.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

DD_SITE="uk1.datadoghq.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

DD_SITE="ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

DD_SITE="us2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

Search Agent Observability spans

/**
 * Search Agent Observability spans returns "OK" response
 */

import { client, v2 } from "@datadog/datadog-api-client";

const configuration = client.createConfiguration({
  authMethods: {
    AuthZ: {
      accessToken: process.env.DD_BEARER_TOKEN as string,
    },
  },
});
configuration.unstableOperations["v2.searchLLMObsSpans"] = true;
const apiInstance = new v2.AgentObservabilityApi(configuration);

const params: v2.AgentObservabilityApiSearchLLMObsSpansRequest = {
  body: {
    data: {
      attributes: {
        filter: {
          from: "now-900s",
          mlApp: "my-llm-app",
          query: "@session_id:abc123def456",
          spanId: "abc123def456",
          spanKind: "llm",
          spanName: "llm_call",
          to: "now",
          traceId: "trace-9a8b7c6d5e4f",
        },
        options: {
          includeAttachments: true,
          timeOffset: 0,
        },
        page: {
          cursor: "eyJzdGFydCI6MTAwfQ==",
          limit: 10,
        },
        sort: "-start_ns",
      },
      type: "spans",
    },
  },
};

apiInstance
  .searchLLMObsSpans(params)
  .then((data: v2.LLMObsSpansResponse) => {
    console.log(
      "API called successfully. Returned data: " + JSON.stringify(data)
    );
  })
  .catch((error: any) => console.error(error));

Instructions

First install the library and its dependencies and then save the example to example.ts and run following commands:

DD_SITE="datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"

DD_SITE="us3.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"

DD_SITE="us5.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"

DD_SITE="datadoghq.eu" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"

DD_SITE="ap1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"

DD_SITE="ap2.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"

DD_SITE="uk1.datadoghq.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"

DD_SITE="ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"

DD_SITE="us2.ddog-gov.com" DD_BEARER_TOKEN="<PERSONAL_ACCESS_TOKEN OR SERVICE_ACCESS_TOKEN>" tsc "example.ts"