Skip to content

Send traces with OpenTelemetry (OTLP)

OpenTelemetry is the spec-compliant way to send traces to Mibo. If your stack — agent framework, app server, LLM SDK, or workflow tool — emits OTel traces, you can point its OTLP/HTTP exporter at Mibo’s ingestion endpoint for passive evaluation. Mibo accepts both JSON and protobuf exports. It does not accept OTLP/gRPC.

The other supported path is Your API — Mibo’s canonical {spans:[...]} JSON posted directly. Pick OTLP when you have an exporter; Your API when you don’t (curl scripts, n8n Cloud, Flowise Cloud, or an environment where you cannot configure OTel). See Sending Traces for the decision matrix.

You configure your OTel exporter to send traces to Mibo’s HTTP endpoint. Mibo receives the OTLP envelope and stores it as a trace for passive testing.

[your instrumented system] --(OTLP/HTTP)--> [Mibo] --> [passive evaluation]

Most OTel SDKs accept these environment variables (Node, Python, Go, Java, etc.). This example uses the recommended protobuf transport:

Terminal window
OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://api.mibo-ai.com/public/traces
OTEL_EXPORTER_OTLP_TRACES_HEADERS=x-api-key=<YOUR_API_KEY>

Mibo accepts both OTLP/HTTP encodings:

  • Protobuf (http/protobuf) sends application/x-protobuf. Use it when your exporter supports protobuf.
  • JSON (http/json) sends application/json. Use it when your exporter supports JSON or when you want to inspect the request body while troubleshooting.

Mibo does not accept OTLP/gRPC. Each OTLP span also needs a non-zero hexadecimal traceId (32 characters) and spanId (16 characters). For Python, the first-party opentelemetry-exporter-otlp-proto-http package sends protobuf, which Mibo accepts. You do not need a custom exporter just to connect Python to Mibo. If you need a one-shot script without an OTel SDK, use Your API instead.

OTLP traces are routed to a Mibo agent by their service.name resource attribute. Each agent in Mibo has an OTLP service name field; when an OTLP trace arrives, Mibo looks up the agent in the request’s project whose OTLP service name equals service.name.

  1. Set OTLP service name on your agent. In the dashboard, open the agent and set its OTLP service name to a value of your choice (e.g. support-agent-prod). The value must be unique within the project.

  2. Configure your exporter to send the same value. Either of these works:

    Terminal window
    OTEL_SERVICE_NAME=support-agent-prod
    # or
    OTEL_RESOURCE_ATTRIBUTES=service.name=support-agent-prod
  3. Use an API key that’s allowed to write to that agent (a per-agent key or a multi-agent key whose allowlist includes it).

Failure modes:

  • 400 OTLP_SERVICE_NAME_REQUIRED — service.name resource attribute is missing.
  • 404 OTLP_PLATFORM_NOT_FOUND — no agent in this project has the matching OTLP service name.
  • 403 OTLP_PLATFORM_NOT_ALLOWED — match found, but the API key isn’t authorized for that agent.

n8n self-hosted has a built-in OTel tracer. See Send traces from n8n for the full setup — env vars, docker-compose, multi-backend config, and all n8n options.

Flowise self-hosted exposes the same OTel exporter variables. Set service.name to the OTLP service name configured on the target agent, then point the exporter at Mibo.

W3C trace-context propagation headers (traceparent, tracestate) are not currently supported for cross-service tracing. If the upstream caller injects a traceparent header into your service, Mibo ignores it — span hierarchy is built from parentSpanId on each span you export. Configure your SDK to extract traceparent into local spans within your service, establishing the parent-child relationship before exporting to Mibo.

Mibo’s assertions look up well-known OTel GenAI semantic-convention attributes. If your instrumentation follows the OTel GenAI conventions, everything works out of the box.

Span attribute Mibo behavior Assertion target
gen_ai.response.text Default text for semantic assertions; Mibo reads from the root span first, then falls back to the most-recent descendant span if absent semantic assertions, response_regex
gen_ai.tool.name Treats the span as a tool call; nested under the parent span’s tool_calls tool_call assertion
gen_ai.tool.call.arguments Parsed as JSON if possible, else stored raw under value tool_call.expected_arguments
gen_ai.usage.input_tokens / gen_ai.usage.output_tokens Pulled from root, summed across descendants if absent token_limit
http.response.status_code Pulled from root, fallback to any descendant span http_status
Span name Every non-tool span (at any depth) becomes a node_call keyed by name (substring, case-insensitive match). Nested spans match too — scope your expected_name accordingly. node_call.expected_name
Root span start/end_time_unix_nano Wall-clock duration of the trace, in milliseconds response_time.max_ms
parent_span_id Defines the span tree; tool-spans attach to their parent’s tool_calls (structural — not a target)

When you write a test case, Mibo uses the attributes above to evaluate assertions:

  • Response time: response_time { max_ms: 5000 } — Mibo computes (root.end - root.start) / 1e6 in milliseconds.
  • HTTP status: http_status { expected_status: 200 } — Mibo reads http.response.status_code from the root span, falling back to descendant spans if absent.
  • Token usage: See Token limit for a test case and its required trace data.
  • Span name matching: node_call { expected_name: "search" } — matches any span whose name contains “search” (substring match, case-insensitive).
  • Tool calls: tool_call { expected_name: "create_booking" } — matches a child span whose gen_ai.tool.name is exactly create_booking.
  • Text-based assertions: Any semantic or regex assertion uses gen_ai.response.text as the default source. Override per-assertion with target_node to point at a specific span name instead.

Beyond the standard semconv keys, you can attach arbitrary attributes to your spans and assert on them directly with json_match and json_schema. Combine target_node (substring match on span name) with field (literal attribute key) to point at any span’s attribute.

For example, if your agent’s orchestrator span carries an attribute like myco.workflow.status with values such as "READY" / "WAITING_FOR_USER", you can gate other assertions on it:

{
"target": "json_match",
"target_node": "orchestrator",
"field": "myco.workflow.status",
"expected_value": "READY",
"id": "ready_gate"
}

And validate the shape of a structured payload on another span:

{
"target": "json_schema",
"target_node": "generator",
"field": "myco.output_payload",
"depends_on": "ready_gate",
"schema": { "type": "array", "minItems": 1 }
}

Notes:

  • field is the literal attribute key, not a dot-path — OTel attribute names commonly contain dots (http.request.method, myco.foo.bar) and are looked up verbatim.
  • OTel attribute values can’t be dicts or lists-of-dicts. JSON-stringify structured payloads (e.g. myco.output_payload = JSON.stringify(payload)) — json_schema parses the string before validating.
  • target_node is a substring, case-insensitive match against span.name. Pick distinctive names to avoid collisions.
  • This works with any OTel-instrumented system, not just GenAI semconv — pick a namespace under your own company prefix (e.g. myco.*) to avoid clashing with future semconv keys.

Mibo requires every OTLP span to include a valid, non-zero traceId and spanId. For a request containing one trace, Mibo uses the x-request-id header when you provide one. Otherwise, it uses the first span’s traceId as the trace’s external identifier.

An exporter batch can contain spans from multiple traces. Mibo splits that batch into one trace per traceId and uses each traceId as that trace’s external identifier, even when the request includes x-request-id. The X-Mibo-Trace-Id response header is returned only for a successful single-trace request. Leave x-request-id unset for normal OTLP exports so each trace keeps its own traceId; Mibo merges later batches for that trace by span_id.

This is how OTel exporters work in practice:

  • SimpleSpanProcessor ships each span as it ends — a 10-span trace = 10 POSTs
  • BatchSpanProcessor flushes every scheduledDelayMillis (5s default) — a long-running trace splits across multiple batches

Either way, Mibo stitches them together on the server, so a single logical trace ends up as a single record in your dashboard.

Passive evaluation is debounced: Mibo waits ~5 s after the latest batch lands before starting evaluation, with a 60 s cap from the first batch. When at least one test applies, a multi-batch trace is normally evaluated once, after all batches have arrived. Idempotent retries that add no new spans don’t trigger another execution.

You don’t have to change SimpleSpanProcessor / BatchSpanProcessor settings to get the right behavior — the server handles assembly. Only consider tuning the exporter if your trace genuinely takes longer than ~60 s of wall-clock to finish.

The Mibo ingestion endpoint returns standard HTTP status codes. Your exporter should handle them as follows:

  • 200 OK — OTLP payload accepted. JSON clients receive {}; protobuf clients receive an empty OTLP response. See OTLP batch responses for partial acceptance.
  • 400 OTLP_SERVICE_NAME_REQUIRED — service.name resource attribute is missing.
  • 400 OTLP_NO_SPANS — the OTLP envelope contained zero spans.
  • 400 OTLP_ID_INVALID — a span is missing a valid, non-zero traceId or spanId, or has an invalid supplied parentSpanId.
  • 400 VALIDATION_ERROR — the OTLP envelope has an invalid shape.
  • 401 UNAUTHORIZED — API key is missing, invalid, revoked, or expired.
  • 403 EMAIL_NOT_VERIFIED — your email isn’t verified, so Mibo didn’t store or evaluate the trace. Open the verification link, return to Mibo, and click I’ve verified my email in the Verify your email banner. OTLP exporters don’t retry a 403. Repeat the source operation to emit a new trace, or manually resend the rejected payload if you retained it.
  • 403 OTLP_PLATFORM_NOT_ALLOWED — service.name matched an agent, but the API key isn’t authorized for it.
  • 404 OTLP_PLATFORM_NOT_FOUND — no agent in this project has an OTLP service name matching service.name. Set it in the dashboard or fix OTEL_SERVICE_NAME.
  • 413 PAYLOAD_TOO_LARGE — trace exceeds 30 MB decompressed. Reduce batch size via maxExportBatchSize.
  • 400 OTLP_PROTOBUF_INVALID — the protobuf request could not be decoded. Check that your exporter sends OTLP/HTTP protobuf with Content-Type: application/x-protobuf.
  • 500+ errors — transient server issue. Most OTel exporters retry automatically with exponential backoff — no action needed. The trace will be retried.

One OTLP request can contain more than one traceId. Mibo processes each trace separately. If some traces are accepted and others are rejected, the endpoint still returns 200 and reports the rejected spans in a partial-success response. Accepted traces are stored.

  • JSON clients receive partialSuccess.rejectedSpans and partialSuccess.errorMessage.
  • Protobuf clients receive ExportTraceServiceResponse.partialSuccess with rejectedSpans and errorMessage.

If every trace in the batch is rejected, Mibo returns the corresponding error status instead. For predictable trace identity, do not use one x-request-id value for a batch containing multiple traces; Mibo uses each trace’s traceId in that case.

OTel’s built-in batching and merging provide natural idempotency:

  • Mibo merges spans by the trace identity and span_id across multiple POSTs.
  • If a batch POST times out, resending it is safe — duplicate spans are deduplicated by span_id.
  • For a request containing one trace, x-request-id replaces traceId as the trace identity. A multi-trace batch ignores the header and preserves each trace’s traceId.

The endpoint accepts up to 30 MB decompressed per request. Verbose GenAI instrumentation (full prompts, completions, and tool arguments in attributes) can hit this ceiling with batches of 50–100 spans.

If you see HTTP 413 (Payload Too Large):

  1. Reduce maxExportBatchSize in your BatchSpanProcessor config — 10–20 spans per batch is a healthy default
  2. Alternatively, filter verbose attributes before export if your SDK supports it
  3. Consider sampling to export only a subset of traces to production if payload size is persistent

OTel GenAI instrumentation packages (@opentelemetry/instrumentation-openai, traceloop/instrumentation-langchain, and similar) capture request/response payloads by default — including prompts, completions, and tool arguments. If your system handles user PII:

  • Review your instrumentation SDK config before sending traces to Mibo — most packages support disabling request/response capture via environment variables or config flags
  • Understand what’s captured — prompts, tool arguments, and LLM responses can all contain sensitive user data
  • Know the visibility model — traces are encrypted at rest and in transit, but attribute contents are visible in the Mibo dashboard to any team member with access
  • Consider sampling strategies — in production, you can sample only a subset of traces to reduce PII exposure while still evaluating real interactions

Send a minimal OTLP trace to verify your endpoint and API key are configured correctly:

Terminal window
curl -X POST https://api.mibo-ai.com/public/traces \
-H "x-api-key: $MIBO_API_KEY" \
-H "content-type: application/json" \
-d '{
"resourceSpans": [{
"resource": {
"attributes": [
{ "key": "service.name", "value": { "stringValue": "my-agent" } }
]
},
"scopeSpans": [{
"spans": [{
"traceId": "5b8aa5a2d2c872e8321cf37308d69df2",
"spanId": "5b8aa5a2d2c872e8",
"name": "agent.run",
"startTimeUnixNano": "1717000000000000000",
"endTimeUnixNano": "1717000001500000000",
"attributes": [
{ "key": "gen_ai.response.text", "value": { "stringValue": "Hello!" } }
],
"status": { "code": 1 }
}]
}]
}]
}'
  • 200 OK — The trace was accepted. JSON clients receive {}; protobuf clients receive an empty OTLP response. It will appear in your Mibo dashboard within seconds.
  • 400 — service.name is missing, the envelope has no spans, the OTLP shape is invalid, or a protobuf body could not be decoded
  • 413 — Payload is too large; reduce batch size (see Payload size limits)

For evaluation timing and retry limits, see Timing of passive test execution.

Problem Likely cause Solution
400 OTLP_PROTOBUF_INVALID The protobuf body could not be decoded Check that the exporter uses OTLP/HTTP protobuf with Content-Type: application/x-protobuf.
400 VALIDATION_ERROR Your OTLP shape is malformed or missing required fields For JSON, check that you’re sending an OTLP/HTTP JSON envelope with resourceSpans. For protobuf, check the exporter and content type. See Error handling.
400 service name missing The OTLP payload has no service.name resource attribute Set OTEL_SERVICE_NAME or OTEL_RESOURCE_ATTRIBUTES=service.name=<YOUR_OTLP_SERVICE_NAME>, matching the target agent’s OTLP service name.
401 UNAUTHORIZED API key is missing, invalid, revoked, or expired Check the OTEL_EXPORTER_OTLP_TRACES_HEADERS configuration without printing the secret, then check the dashboard for expiration or revocation.
413 Payload Too Large Your batch contains too many spans or very large attributes Reduce maxExportBatchSize to 10–20 spans. Filter verbose attributes before export if possible. See Payload size limits.
Traces appear but tests don’t run Evaluation is pending, or no test is eligible and applicable Wait for the final OTLP batch, then follow If a trace appears without an evaluated result. Identical retries do not create another evaluation.
Traces missing gen_ai.* attributes Your instrumentation doesn’t follow OTel GenAI conventions Review your SDK config and enable the relevant instrumentation packages. See Attribute mapping.