# LLM observability for AI apps on Lambda

Ivy records the LLM calls your Lambda functions make, from the **Vercel AI SDK** or any library that emits **OpenTelemetry GenAI** spans, inside the same trace as everything else the invocation did. You see the conversation, the response, tool calls and their results, token counts and time to first token, next to the DynamoDB query a tool ran.

## What you see

- **Each model call:** provider, model, settings, the messages sent, the response, input and output tokens, and time to first token for streamed calls.
- **Agent steps and tools:** the steps of a multi-step generation, each tool call with its arguments and result, and the AWS SDK or HTTP calls the tool made, nested under it.
- **The provider's HTTP request** (`POST api.openai.com`, …) under the model call that made it.
- **Totals:** tokens in and out for the whole trace in its header, and an "LLM usage" table per model.
- **In the service graph:** one node per provider, so you see where an invocation spent its time.

The same data is available to agents: [`get_trace_summary`](https://ivytrace.dev/features/ai-agents) prints lines like `chat gpt-4o-mini 812→24 tok tool:getOrder`.

## Setup

Attach the Ivy layer as usual ([auto-instrumentation](https://ivytrace.dev/features/auto-instrumentation)). Then turn on the AI SDK's telemetry:

**AI SDK 3 to 6:** pass `experimental_telemetry` on each call.

```ts
import { generateText } from "ai";

const result = await generateText({
  model,
  prompt,
  experimental_telemetry: { isEnabled: true },
});
```

**AI SDK 7:** register the OpenTelemetry integration once, at the top of the handler module.

```ts
import { registerTelemetry } from "ai";
import { OpenTelemetry } from "@ai-sdk/otel";

registerTelemetry(new OpenTelemetry());
```

You don't need an OpenTelemetry SDK. The layer registers a minimal tracer provider that libraries using `@opentelemetry/api` report to. If your app registers its own provider (for example with the AWS Distro for OpenTelemetry), Ivy steps aside. `IVY_CAPTURE_OTEL=false` turns this off.

## Supported conventions

| Source | Spans |
|---|---|
| Vercel AI SDK 3–6 | `ai.generateText`, `ai.streamText`, `ai.generateObject`, `ai.toolCall`, `ai.embed`, … |
| Vercel AI SDK 7 with `@ai-sdk/otel` | OpenTelemetry GenAI conventions: `invoke_agent`, `chat`, `execute_tool`, … |
| Other OpenTelemetry GenAI instrumentations | OpenAI, Anthropic and LangChain instrumentations that use `@opentelemetry/api` |

Attributes are stored as the library sent them and interpreted when read, so new instrumentations only need a change in how Ivy displays them.

## Privacy and limits

- Prompts and responses follow the same rules as other payloads: capped per attribute (`IVY_MAX_PAYLOAD_BYTES`), and string values under keys that match your redaction list are replaced. Token counts are kept.
- Long conversations can exceed the cap; raise `IVY_MAX_PAYLOAD_BYTES` if you need the full text.
- Ivy shows tokens, not cost estimates: model prices change too often to hard-code.
- Response streaming Lambda handlers (`awslambda.streamifyResponse`) are not traced yet. Streaming from the model inside a normal handler is.

---

Source: https://ivytrace.dev/features/llm-observability · Updated 2026-11-14