Skip to content

Observability and governance of agentic workflows: MLflow integration

Rationale and architecture

Agentic workflows are non-deterministic, made up of multiple steps and mediated by tools: the same user request can follow different execution paths based on model output, retrieved context, and the availability of external tools. Traditional application logging is insufficient in this context because it records what the service did, but not why the agent decided to do it.

For this reason, every agentic workflow in ALIDA is connected to MLflow, which acts as a single platform for storing execution traces and evaluating agent interactions. MLflow was chosen because it is open source, vendor-independent, and its tracing SDK is fully compatible with OpenTelemetry, including native support for GenAI semantic conventions. This reduces lock-in: the same instrumentation can be exported to a different observability backend without rewriting the agents.

The integration includes two distinct agents built on different technology stacks:

  • Agent A, based on OpenCode;
  • Agent B, based on Agent Development Kit (ADK).

Both write to the same MLflow tracking server, in separate experiment namespaces.

The integration follows a two-level model.

Level 1: trace collection. Both agents send traces to the MLflow tracking server. Instrumentation is applied at the agent-runtime layer rather than in business logic so that observability does not propagate into application code and remains stable between agent versions.

Level 2: model-access governance. For agents where the LLM provider can be configured, inference traffic is also routed through MLflow AI Gateway, which mediates access to underlying LLM providers. The OpenCode-based agent invokes its model providers directly and is observed only through trace collection.

flowchart TD
    OC["Agent (OpenCode)"]
    ADK["Agent (ADK)"]
    GW["MLflow AI Gateway"]
    MLF["Tracking server"]

    OC -- "traces" --> MLF
    ADK -- "traces (OTel)" --> MLF
    ADK -- "inference" --> GW
    GW --> MLF
    GW -. "LLM providers" .-> P(["OpenAI / Anthropic / ..."])

    classDef agent fill:#eef4ff,stroke:#4a6fa5,stroke-width:1px;
    classDef infra fill:#f4f1e8,stroke:#8a7a4a,stroke-width:1px;
    class OC,ADK agent;
    class GW,MLF infra;

Tracing

Each agent invocation produces a trace made up of nested spans that reproduce the workflow's complete execution tree: the incoming request, LLM calls, tool invocations with their results, and the final response.

Traces can be enriched with a session identifier, allowing the platform to group all interactions belonging to the same conversation or task execution. This is important because a single error can rarely be explained by looking at only one isolated trace; a session instead shows how context accumulated, where the agent entered a loop, and at which step the reasoning diverged from expected behavior.

What is captured

For both agents, traces stored in MLflow make it possible to inspect:

  • The execution tree, with latency per span, input, output, and error state.
  • MCP tool use: which Model Context Protocol servers and tools were invoked, with which arguments and which results they returned. This makes the tool layer, normally the least observable part of an agentic system, more explicit and verifiable.
  • Token consumption, broken down by span and aggregated by trace and session, including input, output, and, where applicable, cached tokens.
  • The underlying model, including provider, model identifier, and sampling parameters, so behavior changes can be correlated with model changes.
  • Reasoning content, when the provider exposes it, so the model's intermediate deliberation is retained with the final response instead of being discarded.

OpenCode agent instrumentation

The OpenCode agent is instrumented through MLflow's first-party OpenCode integration, distributed as an OpenCode plugin rather than as application-level code. No tracing logic is written in the agent: the plugin is declared in the OpenCode configuration and reads its settings from the environment.

opencode.json

{
  "plugin": ["@mlflow/opencode"]
}

# .env, loaded automatically by OpenCode at startup
MLFLOW_TRACKING_URI=<tracking-server-uri>
MLFLOW_EXPERIMENT_ID=<experiment-id>

Once the plugin is active, MLflow automatically captures OpenCode conversation traces and records them in the configured experiment. Traces are emitted when the session becomes idle, at the end of each conversational turn; tracing therefore does not add latency to the interactive cycle and does not require the agent to flush explicitly.

Each trace records:

  • user prompts and assistant responses;
  • tool use in the turn, including file operations, bash commands, and code changes, as well as tools exposed by configured MCP servers;
  • the timing and duration of the conversational turn;
  • token usage, split into input, output, and total, recorded in the mlflow.chat.tokenUsage attribute and shown on the trace-details page and overview dashboard along with its associated cost.

The plugin links session and user metadata without further configuration: every OpenCode session is tagged with mlflow.trace.session, and the system user starting the invocation is tagged with mlflow.trace.user. These tags make it possible to group and filter traces by session in the MLflow interface and through the search API, and make the OpenCode agent comparable with the ADK agent on the same platform.

Model access remains under the agent's direct control: provider credentials and model selection are configured in OpenCode, while MLflow observes the resulting calls without mediating them. This has two operational consequences. First, token usage is recorded only when the underlying provider reports it, because the plugin does not intercept the request path. Second, tracing can be disabled simply by removing the plugin from opencode.json, without changing the agent's behavior.

ADK agent instrumentation and AI Gateway

The ADK-based agent supports the OpenTelemetry export path provided by MLflow, but the integration can go further: inference requests are sent to an endpoint exposed by MLflow AI Gateway instead of the provider's native endpoint. The gateway provides a unified interface to multiple LLM providers behind a single secure endpoint and centralizes provider API keys, with request and response logging.

This choice has three direct advantages.

Dynamic model replacement. Endpoints can be added, removed, or reconfigured dynamically without restarting the server or interrupting running applications. In practice, the model used by the ADK agent can be replaced, or traffic can be split between models for A/B tests, without redeploying the agent or changing its code. Automatic fallback chains also protect the workflow from provider outages.

Tracking without integration effort. Since every request passes through the gateway, use, token consumption, latency, and cost are centrally recorded regardless of the agent framework. Observability no longer depends on adopting or correctly configuring an SDK in the agent: every endpoint client is observable by construction. This significantly reduces the cost of onboarding new agents onto the platform.

Guardrails. The gateway makes it possible to apply content policies at the endpoint level through LLM-based judges, which can block or sanitize requests and responses. The policy is thus expressed once, at the infrastructure boundary, and applied consistently to every endpoint consumer rather than being reimplemented, and potentially allowed to diverge, in each agent. Budget limits and spending alerts can be configured at the same level.

Evaluation and monitoring

In addition to storing traces, MLflow is used to evaluate the quality of agent interactions. Collected traces can become the evaluation dataset: real production sessions are evaluated with LLM-as-a-judge metrics through built-in judges for common dimensions such as hallucination or relevance, or through custom scorers that encode domain criteria defined for ALIDA use cases.

This closes the loop between observability and improvement. A change to a prompt, tool, or model is not only observed in production, but measured against a stable set of recorded interactions so regressions can be detected before they reach users.