MCP Server

Run evaluations from Claude Desktop, Cursor, or Claude Code.

The PeerLM MCP (Model Context Protocol) server lets you run evaluations directly from Claude Desktop, Cursor, Claude Code, or any MCP-compatible client. Available on Pro and Enterprise plans.

Setup

Add PeerLM to your MCP client configuration. No installation needed — it runs via npx.

Claude Desktop

Edit ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "peerlm": {
      "command": "npx",
      "args": ["-y", "@peerlm/mcp"],
      "env": { "PEERLM_API_KEY": "plm_live_..." }
    }
  }
}

Cursor

Add to .cursor/mcp.json in your project root:

{
  "mcpServers": {
    "peerlm": {
      "command": "npx",
      "args": ["-y", "@peerlm/mcp"],
      "env": { "PEERLM_API_KEY": "plm_live_..." }
    }
  }
}

Claude Code

Add to your Claude Code MCP settings or .mcp.json:

{
  "mcpServers": {
    "peerlm": {
      "command": "npx",
      "args": ["-y", "@peerlm/mcp"],
      "env": { "PEERLM_API_KEY": "plm_live_..." }
    }
  }
}

Available Tools

The MCP server provides 15 tools:

Eval Suite Tools

• list_suites — list all evaluation suites
• get_suite — get full configuration for a suite
• create_suite — create an evaluation suite with models, prompts, and criteria
• run_eval — trigger an evaluation run
• get_results — check run status and view results

Monitor Tools

• list_workloads — list all monitors (workloads)
• create_workload — create a new monitor
• ingest_logs — send production log records to a monitor
• list_audits — list comparison evals for a monitor
• get_audit — get detailed audit results and recommendations
• start_audit — start a comparison eval against candidate models

Resource Tools

• list_models — browse available models (filter by provider or tier)
• get_usage — check plan, credit balance, and usage
• create_system_prompt — create a system prompt in your library
• create_test_prompt — create a test prompt in your library

Example Workflow

Ask your AI assistant something like: "Compare Claude Sonnet, GPT-5.2, and Gemini 2.5 for extracting structured data from medical notes."

The assistant will:

1. Use list_models to find the exact model IDs
2. Use create_system_prompt to define the system prompt
3. Use create_test_prompt to create sample tasks
4. Use create_suite to wire everything together
5. Use run_eval to start the evaluation
6. Use get_results to retrieve the leaderboard

Requirements

• API key — generate one from Settings → API Keys with read-write scope
• Plan — Pro or Enterprise (Free plan does not include API access)
• Node.js — version 20 or later (for npx)