MLflow

MLflow is an open-source AI engineering platform that covers both LLM applications and traditional machine learning. For LLMs and agents, it records detailed traces of each step using OpenTelemetry, runs evaluations with built-in metrics and LLM judges or custom scorers, versions and optimizes prompts, and offers an AI gateway that routes requests to multiple model providers through an OpenAI-compatible interface. An agent server provides a way to host agents as API endpoints with built-in tracing. For classic machine learning, MLflow provides experiment tracking for parameters and metrics, a model registry for managing model versions and tools for deploying models to batch or real-time serving.

The project is aimed at data scientists, ML engineers and teams building AI agents who want a vendor-neutral way to observe, test and manage their systems. The website lists support for Python, TypeScript and JavaScript, Java and R, along with integrations for frameworks such as LangChain, LlamaIndex, DSPy and PyTorch and for providers including OpenAI, Anthropic and Amazon Bedrock. None of these integrations were tested for this listing.

MLflow's source code is published on GitHub under the Apache-2.0 license, and the project states that it is backed by the Linux Foundation. The open-source software can be self-hosted without license fees, although running a tracking server, storage and model serving brings infrastructure costs. Managed MLflow services are offered by third parties such as Databricks; their prices were not reviewed here.

Because MLflow is self-hosted software, data handling depends on how and where an organization deploys it. Traces and evaluation records may contain prompts and outputs from production systems, so access controls and retention settings should be planned before connecting live applications.