Concepts

​

What is AI Observability?

• What - The inputs and outputs of AI model calls
• How - Performance metrics like latency and token usage
• How much - Cost attribution and spending
• Why - Error tracking and debugging information

​

OpenTelemetry

• Vendor-neutral instrumentation
• Standard data formats
• Wide ecosystem support

​

Traces

Trace: handle-user-message
├── validate-input (2ms)
├── retrieve-context (15ms)
├── openai.chat.completions (1,234ms)
├── format-response (3ms)
└── send-response (5ms)

​

Trace ID

Trace ID: 4bf92f3577b34da6a3ce929d0e0e4736

​

Spans

• Name - What operation is being performed
• Start time - When it began
• Duration - How long it took
• Attributes - Key-value metadata
• Status - Success or error

​

Span Hierarchy

Trace: user-query
└── Span: process_query.workflow (parent)
    ├── Span: validate.task (child)
    ├── Span: llm_call.task (child)
    │   └── Span: openai.chat (grandchild - auto-instrumented)
    └── Span: format_response.task (child)

​

Span Attributes

{
    "gen_ai.system": "openai",
    "gen_ai.request.model": "gpt-4o",
    "gen_ai.usage.prompt_tokens": 150,
    "gen_ai.usage.completion_tokens": 200,
    "gen_ai.usage.cost": 0.0035
}

​

Workflows and Tasks

from anyway.sdk.decorators import workflow, task

@task(name="summarize")
def summarize(text):
    # LLM call - creates a child span
    return result

@workflow(name="process_document")
def process_document(doc):
    # Creates a parent span containing all tasks
    return summarize(doc)

import { withWorkflow, withTask } from "@anyway-sh/node-server-sdk";

async function summarize(text: string) {
  return withTask({ name: "summarize" }, async () => {
    // LLM call - creates a child span
    return result;
  });
}

async function processDocument(doc: string) {
  return withWorkflow({ name: "process_document" }, async () => {
    return summarize(doc);
  });
}

​

Metrics

• Counter - Values that only increase (total requests, total tokens)
• Gauge - Point-in-time values (queue size, active connections)
• Histogram - Distribution of values (latency percentiles)

​

Semantic Conventions

​

Next Steps