Skip to content
Mibo. We test. You rest.Documentation

Quickstart: OpenLLMetry

In about five minutes, you export an OpenLLMetry-instrumented OpenAI call to Mibo, send one trace, and create a test from it. OpenLLMetry is Traceloop’s open-source OpenTelemetry instrumentation for LLM SDKs and frameworks such as LangChain. Mibo evaluates the traces your code sends. It never calls your Agent. If you already export traces to another backend, keep it and add Mibo with the Collector fan-out.

Before you start

Section titled “Before you start”
  • A Mibo account with a verified email. Mibo rejects traces and AI features until you verify. See Email verification.
  • Python 3.10 or later, or Node.js 22 or later.
  • An OpenAI API key.

1. Set up your Agent in Mibo

Section titled “1. Set up your Agent in Mibo”
  1. Create an account at app.mibo-ai.com, or sign in, and click Set up your Agent.
  2. Enter an Agent Name and choose the Other agent type. Click Continue to API key.
  3. Click Create Agent API key and copy the key. Mibo shows it only once. Click I’ve saved it — continue to traces.
  4. In the Agent navigation, open Set up agent. Set OTLP Service Name to support-agent-prod and click Save Changes.

OpenLLMetry sets service.name from the app name you pass to it. Mibo routes each trace to the Agent whose OTLP Service Name equals that value. A missing name returns 400 OTLP_SERVICE_NAME_REQUIRED, and an unknown one returns 404 OTLP_PLATFORM_NOT_FOUND.

2. Install and send your first trace

Section titled “2. Install and send your first trace”

OpenLLMetry’s api_endpoint and baseUrl options append /v1/traces to the URL, which Mibo does not serve. Pass your own OTLP exporter with the full Mibo URL instead.

Terminal window
pip install traceloop-sdk openai
export MIBO_API_KEY=<YOUR_MIBO_API_KEY>
export OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>

Save this as agent.py:

import json
import os


from openai import OpenAI
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import tool, workflow


Traceloop.init(
    app_name="support-agent-prod",
    exporter=OTLPSpanExporter(
        endpoint="https://api.mibo-ai.com/public/traces",
        headers={"x-api-key": os.environ["MIBO_API_KEY"]},
    ),
)


client = OpenAI()


TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "lookup_order",
            "description": "Look up an order by ID",
            "parameters": {
                "type": "object",
                "properties": {"order_id": {"type": "string"}},
                "required": ["order_id"],
            },
        },
    }
]




@tool(name="lookup_order")
def lookup_order(order_id: str) -> dict:
    return {"order_id": order_id, "status": "shipped", "eta": "Friday"}




@workflow(name="support_agent")
def answer(question: str) -> str:
    messages = [
        {"role": "system", "content": "You are a support agent for an online store."},
        {"role": "user", "content": question},
    ]
    first = client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=TOOLS)
    reply = first.choices[0].message
    if not reply.tool_calls:
        return reply.content or ""
    messages.append(reply.model_dump(exclude_none=True))
    for call in reply.tool_calls:
        result = lookup_order(**json.loads(call.function.arguments))
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
    final = client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=TOOLS)
    return final.choices[0].message.content or ""




print(answer("Where is my order A-1042?"))

Run it:

Terminal window
python agent.py

The script prints the model’s reply, for example Your order A-1042 has shipped and should arrive on Friday. The OpenTelemetry SDK flushes the trace when the process exits.

Open the Agent’s Traces page in Mibo. The trace has a support_agent.workflow root span, two openai.chat or chat gpt-4o-mini spans, and a lookup_order.tool span.

3. Create your first test

Section titled “3. Create your first test”
  1. In the Agent navigation, open Create tests with AI.
  2. Click Use real trace. In Ground generation on a real trace, select your trace and click Use this trace.
  3. Review the generated test cases and click Save. Keep the test enabled, with Passive behavior set to Always run or Run when relevant.
  4. Run the script again. Mibo evaluates each new trace against your saved tests.
  5. Open the test and check Last Result.

What Mibo reads from the trace

Section titled “What Mibo reads from the trace”
Data Attribute Notes
User input gen_ai.input.messages on the LLM span Recorded while content tracing is on
Final response gen_ai.output.messages on the LLM span Recorded while content tracing is on
Tool calls gen_ai.tool.name on the lookup_order.tool span Python only. The span carries the tool name but no gen_ai.tool.call.arguments. In TypeScript, the span has no gen_ai.tool.name, so Mibo treats it as a step named lookup_order.tool.
Token usage gen_ai.usage.input_tokens and gen_ai.usage.output_tokens Always recorded

OpenLLMetry records prompts and completions by default. Setting TRACELOOP_TRACE_CONTENT=false removes them. The trace_content=False option in Python and traceContent: false in TypeScript do the same. Without them, Mibo has no user input, so Run when relevant tests are skipped and the trace shows No usable input. Always run tests still run, but checks on the reply have no text to read.

Prompts and replies can contain personal data. Mibo always redacts secrets and redacts personal data when your project enables it. See Data handling.

For other OpenTelemetry instrumentations, see Capture the content Mibo needs.

Endpoint reference

Section titled “Endpoint reference”
Setting Value
Endpoint POST https://api.mibo-ai.com/public/traces. Set the full URL. Exporters that take a base URL append /v1/traces.
Authentication x-api-key header
Encoding OTLP/HTTP protobuf, as in these examples, or JSON. Mibo does not accept OTLP/gRPC.
Compression gzip is accepted.
Size limit 10 MB per request. See Payload size limits.
Routing service.name must match the Agent’s OTLP Service Name.

What’s next

Section titled “What’s next”