mcp bench
pipStandardMCP-Bench evaluates LLMs' tool-use capabilities through complex real-world tasks. It exposes benchmarking tools for AI agents, connects to various MCP servers, and provides metrics for task performance. Developers use it to assess and improve LLM agents' ability to use tools effectively in practical scenarios.
450490Updated 4 months ago
Overview
MCP-Bench evaluates LLMs' tool-use capabilities through complex real-world tasks. It exposes benchmarking tools for AI agents, connects to various MCP servers, and provides metrics for task performance. Developers use it to assess and improve LLM agents' ability to use tools effectively in practical scenarios.
Installation
1Install the package
pip
pip install mcp-bench2Add to Claude
Add this configuration to your claude_desktop_config.json:
claude_desktop_config.json
{
"mcpServers": {
"accenture-mcp-bench-github": {
"command": "uvx",
"args": [
"pip install mcp-bench"
]
}
}
}3Verify installation
Restart Claude Desktop, then ask:
"What tools do you have available from mcp bench?"
Prerequisites
- Python 3.10 or higher
- pip or uv package manager
- Claude Desktop or Claude API access
- API key for mcp bench
Configuration
API Key Required
This server requires an API key from mcp bench. Add it to your environment or config.
| Variable | Required | Description |
|---|---|---|
| MCP_BENCH_API_KEY | Yes | Your mcp bench API key |
Usage Examples
List Resources
"What resources are available in mcp bench?"
Claude will query available resources and return a list of what you can access.
Get Details
"Show me details about [specific item] in mcp bench"
Claude will fetch and display detailed information about the requested item.
Create New
"Create a new [item] in mcp bench with [details]"
Claude will use the appropriate tool to create the resource and confirm success.
Works with mcp bench
Related skills
Browse all skillsNeed help with MCP integration?
We build custom MCP integrations for B2B companies. From simple connections to complex multi-tool setups.