EvalGate

API Reference

Complete REST API documentation for programmatic access to the platform

Authentication

All API requests require authentication using a bearer token. Get your API key from the Developer Dashboard.

curl https://evalgate.com/api/evaluations \
  -H "Authorization: Bearer YOUR_API_KEY"

Base URL: https://evalgate.com

Evaluations API

Create, run, and manage evaluations programmatically

GET/api/evaluations

List all evaluations

Example Request

curl https://evalgate.com/api/evaluations \
  -H "Authorization: Bearer YOUR_API_KEY"

Example Response

{
  "evaluations": [
    {
      "id": "eval_123",
      "name": "Chatbot Accuracy Test",
      "status": "completed",
      "overall": 0.87,
      "created_at": "2024-01-15T10:30:00Z"
    }
  ],
  "total": 1,
  "page": 1
}

POST/api/evaluations

Create new evaluation

Example Request

curl https://evalgate.com/api/evaluations \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Evaluation",
    "datasetId": "dataset_123",
    "metrics": ["factuality", "toxicity"]
  }'

Example Response

{
  "id": "eval_456",
  "name": "My Evaluation",
  "status": "pending",
  "created_at": "2024-01-15T10:35:00Z"
}

GET/api/evaluations/{id}

Get evaluation details

Example Request

curl https://evalgate.com/api/evaluations/eval_123 \
  -H "Authorization: Bearer YOUR_API_KEY"

Example Response

{
  "id": "eval_123",
  "name": "Chatbot Accuracy Test",
  "status": "completed",
  "overall": 0.87,
  "metrics": {
    "factuality": 0.90,
    "toxicity": 0.02
  },
  "items": 10,
  "passed": 8,
  "failed": 2
}

POST/api/evaluations/{id}/runs

Start evaluation run

Example Request

curl https://evalgate.com/api/evaluations/eval_123/runs \
  -X POST \
  -H "Authorization: Bearer YOUR_API_KEY"

Example Response

{
  "run_id": "run_789",
  "status": "running",
  "started_at": "2024-01-15T10:40:00Z"
}

Traces API

Send trace data from your LLM applications

GET/api/traces

List traces

POST/api/traces

Create trace

GET/api/traces/{id}

Get trace details

POST/api/traces/{id}/spans

Add span to trace

LLM Judge API

Configure and run LLM-based evaluations

GET/api/llm-judge/configs

List judge configs

POST/api/llm-judge/evaluate

Evaluate output

GET/api/llm-judge/results

Get evaluation results

POST/api/llm-judge/alignment

Check judge alignment

Annotations API

Create and manage human annotation tasks

GET/api/annotations/tasks

List annotation tasks

POST/api/annotations/tasks

Create annotation task

GET/api/annotations/tasks/{id}

Get task details

POST/api/annotations/tasks/{id}/items

Submit annotation

Rate Limits

• Free tier: 100 requests per hour

• Pro tier: 1,000 requests per hour

• Enterprise: Custom limits

Error Codes

• 400 Bad Request - Invalid parameters

• 401 Unauthorized - Invalid API key

• 429 Too Munknown Requests - Rate limit exceeded

• 500 Internal Server Error - Contact support

Evaluations API

Create, run, and manage evaluations programmatically

GET/api/evaluations

List all evaluations

Example Request

curl https://evalgate.com/api/evaluations \
  -H "Authorization: Bearer YOUR_API_KEY"

Example Response

{
  "evaluations": [
    {
      "id": "eval_123",
      "name": "Chatbot Accuracy Test",
      "status": "completed",
      "overall": 0.87,
      "created_at": "2024-01-15T10:30:00Z"
    }
  ],
  "total": 1,
  "page": 1
}

POST/api/evaluations

Create new evaluation

Example Request

curl https://evalgate.com/api/evaluations \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Evaluation",
    "datasetId": "dataset_123",
    "metrics": ["factuality", "toxicity"]
  }'

Example Response

{
  "id": "eval_456",
  "name": "My Evaluation",
  "status": "pending",
  "created_at": "2024-01-15T10:35:00Z"
}

GET/api/evaluations/{id}

Get evaluation details

Example Request

curl https://evalgate.com/api/evaluations/eval_123 \
  -H "Authorization: Bearer YOUR_API_KEY"

Example Response

{
  "id": "eval_123",
  "name": "Chatbot Accuracy Test",
  "status": "completed",
  "overall": 0.87,
  "metrics": {
    "factuality": 0.90,
    "toxicity": 0.02
  },
  "items": 10,
  "passed": 8,
  "failed": 2
}

POST/api/evaluations/{id}/runs

Start evaluation run

Example Request

curl https://evalgate.com/api/evaluations/eval_123/runs \
  -X POST \
  -H "Authorization: Bearer YOUR_API_KEY"

Example Response

{
  "run_id": "run_789",
  "status": "running",
  "started_at": "2024-01-15T10:40:00Z"
}