Agent framework integrations

EvalGate accepts quality evidence at three boundaries: an SDK wrapper around a runtime call, an OpenTelemetry/OpenInference trace export, or an authenticated MCP/API call. Pick the narrowest boundary your application already supports.

Hosted trace, MCP, and API calls require an EvalGate account, an organization, and a scoped API key. Your EvalGate key does not pay for model inference; model-backed workflows use a provider credential owned and billed by your organization.

Official coding-agent Skills

Browse the official EvalGate Skills repository for the portable evaluation decision framework and setup, gate, trace, repository, and MCP workflows. Preview the collection with npx skills add evalgate/skills --list, then install the complete collection with npx skills add evalgate/skills or a single self-contained Skill with npx skills add evalgate/skills --skill evaluate-ai-change. Skills do not configure MCP servers. The collection is experimental. Skills supply instructions; they do not issue credentials or replace release-policy enforcement. See the coding-agent guide and agent discovery guide.

Support matrix

Runtime	Supported boundary	Package or endpoint	Current limitation
LangChain	TypeScript structural tracing wrapper and offline recipe	`traceLangChainAgent` from `@evalgate/sdk`	Wraps structural `invoke` or `call` methods; install and configure LangChain separately
CrewAI	TypeScript structural tracing wrapper	`traceCrewAI` from `@evalgate/sdk`	Wraps `kickoff`; install and configure CrewAI separately
AutoGen	TypeScript structural tracing wrapper	`traceAutoGen` from `@evalgate/sdk`	Wraps `initiate_chat`; install and configure AutoGen separately
Vercel AI SDK	Language Model v3-compatible model wrapper	`@evalgate/sdk/integrations/vercel-ai`	Trace collection is fail-open and does not replace provider policy
LiteLLM	OpenAI-compatible Model Gateway backend	Customer-configured LiteLLM endpoint	Gateway routing path, not a dedicated framework tracing wrapper
LlamaIndex	OpenTelemetry/OpenInference trace export	OTLP/HTTP	No first-class EvalGate wrapper is advertised today
MCP clients	Streamable HTTP	`/api/mcp` and `/api/mcp/docs`	Product tools use API-key scopes; the public docs server has no tenant context
DSPy and other Python runtimes	OpenTelemetry/OpenInference or explicit Python SDK trace calls	OTLP/HTTP and `evalgate-sdk`	No dedicated DSPy adapter is advertised today

The framework packages remain your dependencies. EvalGate’s structural wrappers do not install them or take over their model-provider configuration.

CrewAI

Wrap the crew object after creating an authenticated client and workflow tracer:

import {
  AIEvalClient,
  createWorkflowTracer,
  traceCrewAI,
} from "@evalgate/sdk";

const evalgate = AIEvalClient.init();
const tracer = createWorkflowTracer(evalgate);
const tracedCrew = traceCrewAI(crew, tracer, { crewName: "ResearchCrew" });

const result = await tracedCrew.kickoff({ topic: "release evidence" });

The wrapper records workflow and agent-span completion or failure around the existing kickoff call. It does not change the crew’s tools or provider keys.

AutoGen

Use the corresponding conversation wrapper when your runtime exposes initiate_chat:

import {
  AIEvalClient,
  createWorkflowTracer,
  traceAutoGen,
} from "@evalgate/sdk";

const evalgate = AIEvalClient.init();
const tracer = createWorkflowTracer(evalgate);
const tracedConversation = traceAutoGen(conversation, tracer, {
  conversationName: "ReleaseReview",
});

await tracedConversation.initiate_chat(reviewer, { message: "Review this change" });

Vercel AI SDK

The Vercel integration wraps a Language Model v3-compatible object without adding the ai package as an EvalGate runtime dependency:

import { traceVercelAIModel } from "@evalgate/sdk/integrations/vercel-ai";
import { generateText } from "ai";

const tracedModel = traceVercelAIModel(model, evalgate, {
  captureInput: true,
  captureOutput: true,
  environment: "staging",
});

const result = await generateText({
  model: tracedModel,
  prompt: "Summarize the release evidence.",
});

Trace-delivery failures do not mask the model result. If trace persistence is a release requirement, verify the trace separately before promoting a baseline.

MCP product and documentation servers

Use the public server cards to inspect the tools before connecting:

Product card: https://www.evalgate.com/.well-known/mcp/server-card.json
Documentation card: https://www.evalgate.com/.well-known/mcp/docs-server-card.json
Product transport: https://www.evalgate.com/api/mcp
Documentation transport: https://www.evalgate.com/api/mcp/docs

The documentation server is public, read-only, and limited to the reviewed published-doc catalog. Product tools are filtered by the scopes on the organization API key. See MCP integration and agent authentication.

DSPy and other OpenTelemetry-compatible runtimes

EvalGate does not currently ship a first-class DSPy wrapper. Use an existing OpenTelemetry or OpenInference instrumentation path to export trace evidence, or call the Python SDK explicitly at the application boundary. Keep model calls and tool side effects in your application; EvalGate receives the trace and evaluation evidence you choose to send. See TypeScript SDK: OTLP/OpenInference and trace setup. Do not label a generic trace export as a dedicated framework integration in architecture or procurement documents.

Verify the integration

Before relying on a framework path for a release gate:

Send one successful and one failing run from a non-production environment.
Confirm workflow, agent, model, tool, and error fields preserve the evidence needed by your assertions.
Turn one reviewed failure into a permanent evaluation case.
Run the gate against the exact commit and baseline used for the release.
Confirm provider, budget, parser, and missing-trace failures remain explicit.

Get Started

Core Concepts

Guides

SDK Reference

Platform

Agent framework integrations

Agent framework integrations

Official coding-agent Skills

Support matrix

CrewAI

AutoGen

Vercel AI SDK

MCP product and documentation servers

DSPy and other OpenTelemetry-compatible runtimes

Verify the integration

​Agent framework integrations

​Official coding-agent Skills

​Support matrix

​CrewAI

​AutoGen

​Vercel AI SDK

​MCP product and documentation servers

​DSPy and other OpenTelemetry-compatible runtimes

​Verify the integration

Agent framework integrations

Official coding-agent Skills

Support matrix

CrewAI

AutoGen

Vercel AI SDK

MCP product and documentation servers

DSPy and other OpenTelemetry-compatible runtimes

Verify the integration