Export Traces to Langfuse / Jaeger

Tasks

Forward every governed LLM request and MCP tool call as an OpenTelemetry trace to the observability stack your platform team already runs. Each span carries the model, provider, latency, token counts, estimated cost, and the policy decision — so LLM traffic shows up in Langfuse, Jaeger, or Grafana Tempo alongside the rest of your telemetry.

note

Trace export is configured in the AIControls config file (or Helm values for self-hosted). It is off by default and adds no overhead when disabled. On managed SaaS, ask your Nirmata contact to enable it for your workspace.

What gets exported

One span per completed request, following the OpenTelemetry GenAI semantic conventions:

• LLM requests — chat <model> spans with gen_ai.request.model, gen_ai.system (provider), input/output/cache/reasoning token counts, gen_ai.usage.cost, finish reason, and the request duration.
• MCP tool calls — execute_tool <tool> spans linked to the LLM turn that requested them via gen_ai.tool.call.id.
• Governance outcome — aicontrols.decision (allow / deny / mutated / …) and the policy that produced it. Denials surface as WARNING-level observations in Langfuse, not errors.
• Session and user dimensions — session.id and user.id, so Langfuse's Sessions and Users views work out of the box.

Prompt and response text is not exported. Token counts, model names, cost, and decisions are metadata; prompt snippets are only included if you explicitly opt in with observability.otel.content.capturePrompts: true — think carefully before enabling that when the backend is a third-party SaaS. See the full attribute table in Trace Attributes.

Configure the exporter

Langfuse

Langfuse ingests OTLP over HTTP (http/protobuf — it does not accept gRPC) and authenticates with your project API key pair:

1. In Langfuse, go to Project Settings → API Keys and create a key pair (pk-lf-… / sk-lf-…).
2. Export the secret key as an environment variable on the AIControls host: export LANGFUSE_SECRET_KEY=sk-lf-… — never put it in the config file.
3. Add the export block to your config and restart AIControls.

observability:
  otel:
    enabled: true
    endpoint: https://cloud.langfuse.com/api/public/otel   # or your self-hosted Langfuse URL
    protocol: http/protobuf
    langfuse:
      publicKey: pk-lf-...
      secretKeyEnv: LANGFUSE_SECRET_KEY

LLM spans appear as generations with model, token, and cost columns populated; traces group by session in the Sessions view.

Jaeger / Tempo / OTLP

Any OTLP collector works over gRPC or HTTP. For Jaeger all-in-one or Grafana Tempo:

observability:
  otel:
    enabled: true
    endpoint: jaeger-collector:4317   # host:port for grpc
    protocol: grpc
    insecure: true                    # in-cluster plaintext collectors only

For a collector that needs auth, use explicit headers instead of the langfuse block:

observability:
  otel:
    enabled: true
    endpoint: https://tempo.example.com
    protocol: http/protobuf
    headers:
      Authorization: "Bearer <token>"

To smoke-test this locally before pointing at a production collector, run a Jaeger all-in-one container (docker run -p 4317:4317 -p 16686:16686 jaegertracing/all-in-one), set endpoint: localhost:4317 with insecure: true as above, send a few requests through AIControls, and confirm the traces show up in the Jaeger UI at http://localhost:16686.

Self-hosted (Helm)

Store the Langfuse secret key in a Kubernetes Secret and reference it from values:

kubectl create secret generic langfuse-key \
  --from-literal=secret-key=sk-lf-...

observability:
  otel:
    enabled: true
    endpoint: https://cloud.langfuse.com/api/public/otel
    protocol: http/protobuf
    langfuse:
      publicKey: pk-lf-...
      existingSecret: langfuse-key

Tuning

All settings below live under observability.otel in the config file:

Setting	Default	Notes
observability.otel.export.requestTypes	[llm, mcp]	Add egress to export outbound HTTP calls; drop mcp to reduce ingest volume
observability.otel.export.sampleRatio	1.0	Head sampling. Below 1.0 breaks cost summation in Langfuse dashboards — keep 1.0 unless volume forces otherwise
observability.otel.export.includeSystemCalls	false	Include AIControls' own internal LLM calls (session summaries, LLM-judge)
observability.otel.content.capturePrompts	false	Export prompt snippets as gen_ai.prompt. Denied requests carry the raw blocked prompt
observability.otel.content.captureToolArguments	false	Export MCP tool argument JSON

Reliability

The exporter can never block or slow a request: spans are synthesized after each request completes, batched in a bounded queue, and dropped if the backend is down or slow. Delivery is observable via the aicontrols_otel_export_spans_total Prometheus counter (outcome: enqueued / exported / failed). enqueued − exported − failed covers spans still queued or dropped — it equals drops exactly only when the exporter is idle, so watch for a steadily growing difference rather than treating the instantaneous value as a drop count. A failing endpoint logs one warning, not a flood.