Send traces with OpenTelemetry (OTLP)
OpenTelemetry is the spec-compliant way to send traces to Mibo. If your stack — agent framework, app server, LLM SDK, or workflow tool — emits OTel traces, you can point its OTLP/HTTP exporter at Mibo’s ingestion endpoint for passive evaluation. Mibo accepts both JSON and protobuf exports. It does not accept OTLP/gRPC.
The other supported path is Your API — Mibo’s canonical {spans:[...]} JSON posted directly. Pick OTLP when you have an exporter; Your API when you don’t (curl scripts, n8n Cloud, Flowise Cloud, or an environment where you cannot configure OTel). See Sending Traces for the decision matrix.
How it works
Section titled “How it works”You configure your OTel exporter to send traces to Mibo’s HTTP endpoint. Mibo receives the OTLP envelope and stores it as a trace for passive testing.
[your instrumented system] --(OTLP/HTTP)--> [Mibo] --> [passive evaluation]Exporter configuration
Section titled “Exporter configuration”Most OTel SDKs accept these environment variables (Node, Python, Go, Java, etc.). This example uses the recommended protobuf transport:
OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobufOTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://api.mibo-ai.com/public/tracesOTEL_EXPORTER_OTLP_TRACES_HEADERS=x-api-key=<YOUR_API_KEY>Choose a transport
Section titled “Choose a transport”Mibo accepts both OTLP/HTTP encodings:
- Protobuf (
http/protobuf) sendsapplication/x-protobuf. Use it when your exporter supports protobuf. - JSON (
http/json) sendsapplication/json. Use it when your exporter supports JSON or when you want to inspect the request body while troubleshooting.
Mibo does not accept OTLP/gRPC. Each OTLP span also needs a non-zero hexadecimal traceId (32 characters) and spanId (16 characters). For Python, the first-party opentelemetry-exporter-otlp-proto-http package sends protobuf, which Mibo accepts. You do not need a custom exporter just to connect Python to Mibo. If you need a one-shot script without an OTel SDK, use Your API instead.
Configuring trace routing
Section titled “Configuring trace routing”OTLP traces are routed to a Mibo agent by their service.name resource attribute. Each agent in Mibo has an OTLP service name field; when an OTLP trace arrives, Mibo looks up the agent in the request’s project whose OTLP service name equals service.name.
-
Set OTLP service name on your agent. In the dashboard, open the agent and set its OTLP service name to a value of your choice (e.g.
support-agent-prod). The value must be unique within the project. -
Configure your exporter to send the same value. Either of these works:
Terminal window OTEL_SERVICE_NAME=support-agent-prod# orOTEL_RESOURCE_ATTRIBUTES=service.name=support-agent-prod -
Use an API key that’s allowed to write to that agent (a per-agent key or a multi-agent key whose allowlist includes it).
Failure modes:
400 OTLP_SERVICE_NAME_REQUIRED—service.nameresource attribute is missing.404 OTLP_PLATFORM_NOT_FOUND— no agent in this project has the matchingOTLP service name.403 OTLP_PLATFORM_NOT_ALLOWED— match found, but the API key isn’t authorized for that agent.
Special integration setups
Section titled “Special integration setups”n8n self-hosted (≥ 2.19.0)
Section titled “n8n self-hosted (≥ 2.19.0)”n8n self-hosted has a built-in OTel tracer. See Send traces from n8n for the full setup — env vars, docker-compose, multi-backend config, and all n8n options.
Flowise self-hosted
Section titled “Flowise self-hosted”Flowise self-hosted exposes the same OTel exporter variables. Set service.name to the OTLP service name configured on the target agent, then point the exporter at Mibo.
Advanced configuration
Section titled “Advanced configuration”W3C traceparent propagation
Section titled “W3C traceparent propagation”W3C trace-context propagation headers (traceparent, tracestate) are not currently supported for cross-service tracing. If the upstream caller injects a traceparent header into your service, Mibo ignores it — span hierarchy is built from parentSpanId on each span you export. Configure your SDK to extract traceparent into local spans within your service, establishing the parent-child relationship before exporting to Mibo.
What Mibo reads from your spans
Section titled “What Mibo reads from your spans”Mibo’s assertions look up well-known OTel GenAI semantic-convention attributes. If your instrumentation follows the OTel GenAI conventions, everything works out of the box.
Attribute mapping
Section titled “Attribute mapping”| Span attribute | Mibo behavior | Assertion target |
|---|---|---|
gen_ai.response.text |
Default text for semantic assertions; Mibo reads from the root span first, then falls back to the most-recent descendant span if absent | semantic assertions, response_regex |
gen_ai.tool.name |
Treats the span as a tool call; nested under the parent span’s tool_calls |
tool_call assertion |
gen_ai.tool.call.arguments |
Parsed as JSON if possible, else stored raw under value |
tool_call.expected_arguments |
gen_ai.usage.input_tokens / gen_ai.usage.output_tokens |
Pulled from root, summed across descendants if absent | token_limit |
http.response.status_code |
Pulled from root, fallback to any descendant span | http_status |
Span name |
Every non-tool span (at any depth) becomes a node_call keyed by name (substring, case-insensitive match). Nested spans match too — scope your expected_name accordingly. |
node_call.expected_name |
Root span start/end_time_unix_nano |
Wall-clock duration of the trace, in milliseconds | response_time.max_ms |
parent_span_id |
Defines the span tree; tool-spans attach to their parent’s tool_calls |
(structural — not a target) |
Common assertion examples
Section titled “Common assertion examples”When you write a test case, Mibo uses the attributes above to evaluate assertions:
- Response time:
response_time { max_ms: 5000 }— Mibo computes(root.end - root.start) / 1e6in milliseconds. - HTTP status:
http_status { expected_status: 200 }— Mibo readshttp.response.status_codefrom the root span, falling back to descendant spans if absent. - Token usage: See Token limit for a test case and its required trace data.
- Span name matching:
node_call { expected_name: "search" }— matches any span whosenamecontains “search” (substring match, case-insensitive). - Tool calls:
tool_call { expected_name: "create_booking" }— matches a child span whosegen_ai.tool.nameis exactlycreate_booking. - Text-based assertions: Any semantic or regex assertion uses
gen_ai.response.textas the default source. Override per-assertion withtarget_nodeto point at a specific span name instead.
Asserting on your own span attributes
Section titled “Asserting on your own span attributes”Beyond the standard semconv keys, you can attach arbitrary attributes to your spans and assert on them directly with json_match and json_schema. Combine target_node (substring match on span name) with field (literal attribute key) to point at any span’s attribute.
For example, if your agent’s orchestrator span carries an attribute like myco.workflow.status with values such as "READY" / "WAITING_FOR_USER", you can gate other assertions on it:
{ "target": "json_match", "target_node": "orchestrator", "field": "myco.workflow.status", "expected_value": "READY", "id": "ready_gate"}And validate the shape of a structured payload on another span:
{ "target": "json_schema", "target_node": "generator", "field": "myco.output_payload", "depends_on": "ready_gate", "schema": { "type": "array", "minItems": 1 }}Notes:
fieldis the literal attribute key, not a dot-path — OTel attribute names commonly contain dots (http.request.method,myco.foo.bar) and are looked up verbatim.- OTel attribute values can’t be dicts or lists-of-dicts. JSON-stringify structured payloads (e.g.
myco.output_payload = JSON.stringify(payload)) —json_schemaparses the string before validating. target_nodeis a substring, case-insensitive match againstspan.name. Pick distinctive names to avoid collisions.- This works with any OTel-instrumented system, not just GenAI semconv — pick a namespace under your own company prefix (e.g.
myco.*) to avoid clashing with future semconv keys.
Trace identity and incremental exports
Section titled “Trace identity and incremental exports”Mibo requires every OTLP span to include a valid, non-zero traceId and spanId. For a request containing one trace, Mibo uses the x-request-id header when you provide one. Otherwise, it uses the first span’s traceId as the trace’s external identifier.
An exporter batch can contain spans from multiple traces. Mibo splits that batch into one trace per traceId and uses each traceId as that trace’s external identifier, even when the request includes x-request-id. The X-Mibo-Trace-Id response header is returned only for a successful single-trace request. Leave x-request-id unset for normal OTLP exports so each trace keeps its own traceId; Mibo merges later batches for that trace by span_id.
How batching affects trace assembly
Section titled “How batching affects trace assembly”This is how OTel exporters work in practice:
SimpleSpanProcessorships each span as it ends — a 10-span trace = 10 POSTsBatchSpanProcessorflushes everyscheduledDelayMillis(5s default) — a long-running trace splits across multiple batches
Either way, Mibo stitches them together on the server, so a single logical trace ends up as a single record in your dashboard.
Timing of passive test execution
Section titled “Timing of passive test execution”Passive evaluation is debounced: Mibo waits ~5 s after the latest batch lands before starting evaluation, with a 60 s cap from the first batch. When at least one test applies, a multi-batch trace is normally evaluated once, after all batches have arrived. Idempotent retries that add no new spans don’t trigger another execution.
You don’t have to change SimpleSpanProcessor / BatchSpanProcessor settings to get the right behavior — the server handles assembly. Only consider tuning the exporter if your trace genuinely takes longer than ~60 s of wall-clock to finish.
Error handling and resilience
Section titled “Error handling and resilience”Response codes
Section titled “Response codes”The Mibo ingestion endpoint returns standard HTTP status codes. Your exporter should handle them as follows:
200 OK— OTLP payload accepted. JSON clients receive{}; protobuf clients receive an empty OTLP response. See OTLP batch responses for partial acceptance.400 OTLP_SERVICE_NAME_REQUIRED—service.nameresource attribute is missing.400 OTLP_NO_SPANS— the OTLP envelope contained zero spans.400 OTLP_ID_INVALID— a span is missing a valid, non-zerotraceIdorspanId, or has an invalid suppliedparentSpanId.400 VALIDATION_ERROR— the OTLP envelope has an invalid shape.401 UNAUTHORIZED— API key is missing, invalid, revoked, or expired.403 EMAIL_NOT_VERIFIED— your email isn’t verified, so Mibo didn’t store or evaluate the trace. Open the verification link, return to Mibo, and click I’ve verified my email in the Verify your email banner. OTLP exporters don’t retry a403. Repeat the source operation to emit a new trace, or manually resend the rejected payload if you retained it.403 OTLP_PLATFORM_NOT_ALLOWED—service.namematched an agent, but the API key isn’t authorized for it.404 OTLP_PLATFORM_NOT_FOUND— no agent in this project has anOTLP service namematchingservice.name. Set it in the dashboard or fixOTEL_SERVICE_NAME.413 PAYLOAD_TOO_LARGE— trace exceeds 30 MB decompressed. Reduce batch size viamaxExportBatchSize.400 OTLP_PROTOBUF_INVALID— the protobuf request could not be decoded. Check that your exporter sends OTLP/HTTP protobuf withContent-Type: application/x-protobuf.500+errors — transient server issue. Most OTel exporters retry automatically with exponential backoff — no action needed. The trace will be retried.
OTLP batch responses
Section titled “OTLP batch responses”One OTLP request can contain more than one traceId. Mibo processes each trace separately. If some traces are accepted and others are rejected, the endpoint still returns 200 and reports the rejected spans in a partial-success response. Accepted traces are stored.
- JSON clients receive
partialSuccess.rejectedSpansandpartialSuccess.errorMessage. - Protobuf clients receive
ExportTraceServiceResponse.partialSuccesswithrejectedSpansanderrorMessage.
If every trace in the batch is rejected, Mibo returns the corresponding error status instead. For predictable trace identity, do not use one x-request-id value for a batch containing multiple traces; Mibo uses each trace’s traceId in that case.
Idempotency
Section titled “Idempotency”OTel’s built-in batching and merging provide natural idempotency:
- Mibo merges spans by the trace identity and
span_idacross multiple POSTs. - If a batch POST times out, resending it is safe — duplicate spans are deduplicated by
span_id. - For a request containing one trace,
x-request-idreplacestraceIdas the trace identity. A multi-trace batch ignores the header and preserves each trace’straceId.
Operational concerns
Section titled “Operational concerns”Payload size limits
Section titled “Payload size limits”The endpoint accepts up to 30 MB decompressed per request. Verbose GenAI instrumentation (full prompts, completions, and tool arguments in attributes) can hit this ceiling with batches of 50–100 spans.
If you see HTTP 413 (Payload Too Large):
- Reduce
maxExportBatchSizein yourBatchSpanProcessorconfig — 10–20 spans per batch is a healthy default - Alternatively, filter verbose attributes before export if your SDK supports it
- Consider sampling to export only a subset of traces to production if payload size is persistent
PII and sensitive data
Section titled “PII and sensitive data”OTel GenAI instrumentation packages (@opentelemetry/instrumentation-openai, traceloop/instrumentation-langchain, and similar) capture request/response payloads by default — including prompts, completions, and tool arguments. If your system handles user PII:
- Review your instrumentation SDK config before sending traces to Mibo — most packages support disabling request/response capture via environment variables or config flags
- Understand what’s captured — prompts, tool arguments, and LLM responses can all contain sensitive user data
- Know the visibility model — traces are encrypted at rest and in transit, but attribute contents are visible in the Mibo dashboard to any team member with access
- Consider sampling strategies — in production, you can sample only a subset of traces to reduce PII exposure while still evaluating real interactions
Testing your setup
Section titled “Testing your setup”Quick sanity check via curl
Section titled “Quick sanity check via curl”Send a minimal OTLP trace to verify your endpoint and API key are configured correctly:
curl -X POST https://api.mibo-ai.com/public/traces \ -H "x-api-key: $MIBO_API_KEY" \ -H "content-type: application/json" \ -d '{ "resourceSpans": [{ "resource": { "attributes": [ { "key": "service.name", "value": { "stringValue": "my-agent" } } ] }, "scopeSpans": [{ "spans": [{ "traceId": "5b8aa5a2d2c872e8321cf37308d69df2", "spanId": "5b8aa5a2d2c872e8", "name": "agent.run", "startTimeUnixNano": "1717000000000000000", "endTimeUnixNano": "1717000001500000000", "attributes": [ { "key": "gen_ai.response.text", "value": { "stringValue": "Hello!" } } ], "status": { "code": 1 } }] }] }] }'Understanding the response
Section titled “Understanding the response”200 OK— The trace was accepted. JSON clients receive{}; protobuf clients receive an empty OTLP response. It will appear in your Mibo dashboard within seconds.- 400 —
service.nameis missing, the envelope has no spans, the OTLP shape is invalid, or a protobuf body could not be decoded - 413 — Payload is too large; reduce batch size (see Payload size limits)
For evaluation timing and retry limits, see Timing of passive test execution.
Troubleshooting
Section titled “Troubleshooting”| Problem | Likely cause | Solution |
|---|---|---|
| 400 OTLP_PROTOBUF_INVALID | The protobuf body could not be decoded | Check that the exporter uses OTLP/HTTP protobuf with Content-Type: application/x-protobuf. |
| 400 VALIDATION_ERROR | Your OTLP shape is malformed or missing required fields | For JSON, check that you’re sending an OTLP/HTTP JSON envelope with resourceSpans. For protobuf, check the exporter and content type. See Error handling. |
| 400 service name missing | The OTLP payload has no service.name resource attribute |
Set OTEL_SERVICE_NAME or OTEL_RESOURCE_ATTRIBUTES=service.name=<YOUR_OTLP_SERVICE_NAME>, matching the target agent’s OTLP service name. |
| 401 UNAUTHORIZED | API key is missing, invalid, revoked, or expired | Check the OTEL_EXPORTER_OTLP_TRACES_HEADERS configuration without printing the secret, then check the dashboard for expiration or revocation. |
| 413 Payload Too Large | Your batch contains too many spans or very large attributes | Reduce maxExportBatchSize to 10–20 spans. Filter verbose attributes before export if possible. See Payload size limits. |
| Traces appear but tests don’t run | Evaluation is pending, or no test is eligible and applicable | Wait for the final OTLP batch, then follow If a trace appears without an evaluated result. Identical retries do not create another evaluation. |
Traces missing gen_ai.* attributes |
Your instrumentation doesn’t follow OTel GenAI conventions | Review your SDK config and enable the relevant instrumentation packages. See Attribute mapping. |