Upskill generates and evaluates agent skills for code agents like Claude Code, Open Code, and OpenAI Codex. It benefits operations teams by creating skills with teacher models and optimizing them with student models. It connects to code agents and workflows requiring automated skill evaluation.
git clone https://github.com/huggingface/upskill.gitUPskill automates the creation and evaluation of agent skills for code agents through a two-tier model approach. Teacher models (expensive/slow) generate high-quality skills that student models (cheap/fast) can use to perform complex tasks reliably. The tool generates skills from task descriptions or existing agent traces, creates synthetic test cases, and benchmarks multiple models against defined evaluation criteria. It supports local execution and remote evaluation on Hugging Face Jobs, making it ideal for operations teams building optimized skill libraries for their agent workflows.
["Identify the task to automate: Use this skill to generate agent skills for repetitive or error-prone tasks like code refactoring, test generation, or documentation updates. For example, automate the creation of unit tests for new API endpoints.","Define inputs and outputs: Specify the inputs (e.g., repository URL, target branches) and desired outputs (e.g., Swagger JSON, Markdown files) for the skill. Include edge cases and performance requirements.","Generate the skill: Use the prompt template to create the initial skill draft. Replace [AGENT_TYPE], [SPECIFIC_TASK], [EDGE_CASE_1], [EDGE_CASE_2], [TEACHER_MODEL], and [STUDENT_MODEL] with your specific details.","Validate and optimize: Run the skill through the teacher model to ensure correctness and completeness. Then, use the student model to refine performance, reduce errors, and improve benchmarks.","Integrate and test: Deploy the skill in your code agent (e.g., Claude Code, Open Code) and test it on real-world examples. Monitor performance metrics and adjust as needed."]
Create reusable skills for code agents from natural language task descriptions
Optimize expensive teacher model outputs for cheaper student model inference
Benchmark multiple models against generated skills to identify performance trade-offs
Improve existing skills by adding examples and refining based on evaluation results
No install command available. Check the GitHub repository for manual installation instructions.
git clone https://github.com/huggingface/upskillCopy the install command above and run it in your terminal.
Launch Claude Code, Cursor, or your preferred AI coding agent.
Use the prompt template or examples below to test the skill.
Adapt the skill to your specific use case and workflow.
Generate an agent skill for [AGENT_TYPE] that automates [SPECIFIC_TASK]. The skill should include: 1) A clear description of what it does, 2) Input/output examples, 3) Error handling for edge cases like [EDGE_CASE_1] or [EDGE_CASE_2], and 4) Performance benchmarks with metrics like accuracy or speed. Validate the skill using [TEACHER_MODEL] and optimize it with [STUDENT_MODEL].
```json
{
"skill_name": "Automated_API_Documentation_Generator",
"description": "This skill automates the generation of API documentation by parsing codebases, extracting endpoints, and formatting them into Markdown files compatible with tools like Swagger or Postman.",
"input_example": {
"repository_url": "https://github.com/example/backend-service",
"target_branches": ["main", "develop"],
"output_format": "swagger"
},
"output_example": {
"documentation": "Generated Swagger UI-compatible JSON file with 12 endpoints, including request/response schemas and example payloads.",
"coverage_report": "95% of endpoints documented (1 missing due to deprecated endpoint).",
"execution_time": "4.2 seconds"
},
"error_handling": {
"missing_repository": "Returns error: 'Repository not found or inaccessible.'",
"unsupported_language": "Returns error: 'Language not supported. Currently supports Python, JavaScript, and Java.'",
"invalid_format": "Returns error: 'Unsupported output format. Choose from: swagger, postman, or markdown.'"
},
"performance_benchmarks": {
"accuracy": "98% endpoint coverage across 50 tested repositories.",
"speed": "Average execution time: 3.8 seconds for repositories under 100k lines of code.",
"scalability": "Handles repositories up to 500k lines of code without significant degradation."
}
}
```
**Validation & Optimization:**
The skill was initially generated using **Claude Code** as the teacher model, which provided a baseline implementation. The student model (**Open Code**) then refined the skill by:
- Reducing false positives in endpoint detection by 15%.
- Improving error messages for unsupported languages.
- Adding support for Java annotations in endpoint parsing.
The optimized skill was tested on a dataset of 200 repositories, achieving a 97% success rate in generating valid documentation. The student model also suggested adding a caching mechanism to reduce execution time for repeated runs on the same repository.Pioneering accessible, high-performance AI models
AI assistant built for thoughtful, nuanced conversation
Get more done every day with Microsoft Teams – powered by AI
Automate your spreadsheet tasks with AI power
Agentic AI Workflow platform
Connected workspace for docs, wikis, and projects
Take a free 3-minute scan and get personalized AI skill recommendations.
Take free scan