mcp evals
npxStandardA Node.js package and GitHub Action for evaluating MCP (Model Context Protocol) tool implementations using LLM-based scoring. This helps ensure your MCP server's tools are working correctly and performing well.
125110Updated 8 months ago
Overview
A Node.js package and GitHub Action for evaluating MCP (Model Context Protocol) tool implementations using LLM-based scoring. This helps ensure your MCP server's tools are working correctly and performing well.
Installation
1Install the package
npx
npx -y mcp-evals2Add to Claude
Add this configuration to your claude_desktop_config.json:
claude_desktop_config.json
{
"mcpServers": {
"mclenhard-mcp-evals-github": {
"command": "npx",
"args": [
"-y",
"npx -y mcp-evals"
]
}
}
}3Verify installation
Restart Claude Desktop, then ask:
"What tools do you have available from mcp evals?"
Prerequisites
- Node.js 18 or higher
- npm or npx available
- Claude Desktop or Claude API access
- API key for mcp evals
Configuration
API Key Required
This server requires an API key from mcp evals. Add it to your environment or config.
| Variable | Required | Description |
|---|---|---|
| MCP_EVALS_API_KEY | Yes | Your mcp evals API key |
Usage Examples
List Resources
"What resources are available in mcp evals?"
Claude will query available resources and return a list of what you can access.
Get Details
"Show me details about [specific item] in mcp evals"
Claude will fetch and display detailed information about the requested item.
Create New
"Create a new [item] in mcp evals with [details]"
Claude will use the appropriate tool to create the resource and confirm success.
Works with mcp evals
Related skills
Browse all skillsNeed help with MCP integration?
We build custom MCP integrations for B2B companies. From simple connections to complex multi-tool setups.