# Contributing > How to contribute tasks to SkillsBench — the first benchmark that tests whether agent skills can improve agent performance. Docs index for agents: https://www.benchflow.ai/docs/llms.txt SkillsBench is the first benchmark that tests whether agent skills can improve agent performance, and how good agents are at using skills. [Skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills) was first introduced by Anthropic on Oct 16, 2025, and became an [open standard](https://agentskills.io/) on Dec 18, 2025. Our goal is to build the best, broadest, and highest-quality benchmark for measuring the performance of skill-enabled agents, and to make it the most widely adopted in the field. We aim to design tasks that require skill composition (3+ skills) hard enough so that SOTA performances are lower than 39%. SkillsBench evaluates: 1. How well skills improve agent efficacy vs no skills 2. How well agents can compose multiple skills together 3. Whether agents can identify correct skills among distractors This addresses a gap: nobody measures agent performance on common daily tasks (office docs, git, data processing) despite these being 99% of real use cases. # How to Get Involved ## Getting Access 1. Join the [BenchFlow Discord](https://discord.gg/G9dg3EfSva) server (#skillsbench channel) or [add Xiangyi's WeChat](https://github.com/benchflow-ai/skillsbench/blob/main/docs/wechat-qr.jpg) (please add note: SkillsBench + Background) - Introduce yourself in the channel 2. Provide your name, email, affiliation on the [SkillsBench Workspace](https://docs.google.com/spreadsheets/d/1BJpSxIt4DYedVQ26eOa9Put4TgPBv9295wB2bBkHfA8/edit?gid=1867352925#gid=1867352925) - Subscribe to meetings: [Weekly Sync](https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=NmYzM2Y5NDc3NDg5NGUyYjhiZmQ4OGEwZmZlMjA0MTBfMjAyNjAxMDZUMDEwMDAwWiB4aWFuZ3lpQGJlbmNoZmxvdy5haQ&tmsrc=xiangyi%40benchflow.ai&scp=ALL), [ICML Sprint](https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=NjE4YjMzNDc0MTVjNDc5NGJmNzAyZDMyNzA0MDYwZjJfMjAyNjAxMDlUMDEwMDAwWiB4aWFuZ3lpQGJlbmNoZmxvdy5haQ&tmsrc=xiangyi%40benchflow.ai&scp=ALL) 3. (Optional) [Schedule a quick call](https://cal.com/xiangyi/skillsbench) with Xiangyi Li to answer questions and brainstorm ideas ## Getting Started 1. Read through the [CONTRIBUTING.md](https://github.com/benchflow-ai/skillsbench/blob/main/CONTRIBUTING.md) on GitHub for basic context and orientation - The project adopts agent-native development. While we require instruction.md, task.toml, and task ideas to be written by humans, it's okay to use AI-assisted programming for other tasks. 2. Join meetings - Weekly sync on Monday 5PM PT / 8PM ET / 9AM GMT+8 # Contributing See the [CONTRIBUTING.md](https://github.com/benchflow-ai/skillsbench/blob/main/CONTRIBUTING.md) and [PR template](https://github.com/benchflow-ai/skillsbench/blob/main/.github/PULL_REQUEST_TEMPLATE.md) on GitHub. ## Task Requirements - BenchFlow task format with oracle solution at 100% pass rate - Test composability: tasks requiring 3-6 skills together - Limit distractor skills to <10 ## Workflow 1. Design the skill 2. Run with local claude code / codex / goose / gemini cli 3. Run agent without skills, then with skills 4. When working, add distractor skills # What Tasks We Want ## Priority Skill Categories **High priority** (daily use, unmeasured): - Office suite: pptx, google docs, excel - Version control: git, github - Collaboration: slack, notion **Subject matter expertise:** - Balance of payments, logistics, bio, finance ## Task Types to Create 1. **Single skill baseline** - e.g., "create a spreadsheet summarizing this data" 2. **Two skills composed** - e.g., "pull git history and generate report document" 3. **Three+ skills composed** - e.g., "fetch data from API, analyze in spreadsheet, create presentation" 4. **Skills with distractors** - correct skills among irrelevant ones 5. **Novel skill application** - can agent apply unfamiliar skill from reading it For each task, document: - Which skills are required vs distractor - Expected pass rate without skills vs with skills - Verification criteria # FAQ ## Contributing **Q: What kind of tasks are we looking for?** See the [`skillsbench` SKILL.md](https://github.com/benchflow-ai/skillsbench/blob/main/.claude/skills/skillsbench/SKILL.md) and the repo [CONTRIBUTING.md](https://github.com/benchflow-ai/skillsbench/blob/main/CONTRIBUTING.md) for the task classification philosophy. **Q: How do I qualify for authorship?** 3 high-quality tasks merged to main = automatic authorship **Q: What if I contribute fewer tasks but help with other work?** We absolutely consider other contributions: - Engineering work (infrastructure, tooling, CI/CD) - Running experiments - Paper writing We are very flexible. If you're interested in helping, please reach out! ## Skills Source **Q: Do we use existing skills or contribute new skills?** Both are okay! You can find useful skills at: - [skillsmp.com](https://skillsmp.com/) - [smithery.ai/skills](https://smithery.ai/skills) - [claude-scientific-skills](https://github.com/K-Dense-AI/claude-scientific-skills) For more details, visit the [Google Docs Quick Start](https://docs.google.com/document/d/17f_qDeYPaNQRVDIFIr5topEUMd4_hv1RboVGGLGgdLc/edit). # Task Format Tasks follow the [BenchFlow task format](https://www.benchflow.ai/docs/benchflow/task-authoring.md): ``` task-name/ ├── instruction.md # REQUIRED - Task description ├── task.toml # REQUIRED - Metadata, timeouts, required/distractor skills ├── environment/ │ ├── Dockerfile # REQUIRED - Container with dependencies │ └── skills/ # OPTIONAL - Skills available to agent │ └── skill-name/ │ ├── SKILL.md # REQUIRED (per skill) │ ├── scripts/ # OPTIONAL │ ├── references/ # OPTIONAL │ └── assets/ # OPTIONAL ├── solution/ │ └── solve.sh # REQUIRED - Oracle solution (must pass 100%) └── tests/ ├── test.sh # REQUIRED - Runs pytest └── test_outputs.py # REQUIRED - Writes reward to /logs/verifier/reward.txt ``` ## instruction.md style Direct, terminal-bench style. No "Objective:" or "Available Skills:" sections: ``` Build a sales report from the spreadsheet data. 1. Load sales data from /app/data/sales.csv 2. Calculate total revenue by region 3. Generate /app/output/report.xlsx with summary sheet 4. Create /app/output/chart.png showing revenue breakdown ``` Style traits: - Conversational - "I am trying to...", "Help!", "Could you help me..." - Context-rich - Often starts with WHY or a scenario - Numbered lists for sequential steps - Explicit about output format and file paths - No unnecessary sections # Resources ## Skills Documentation - [Anthropic Skills Docs](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview) - [Anthropic Skills Repo](https://github.com/anthropics/skills) - [OpenAI Skills Repo](https://github.com/openai/skills/tree/main/skills) ## BenchFlow Runtime SkillsBench tasks run on the [BenchFlow runtime](https://www.benchflow.ai/docs/benchflow.md): - [BenchFlow Repo](https://github.com/benchflow-ai/benchflow) - [BenchFlow docs](https://www.benchflow.ai/docs/benchflow.md) Key commands: ```bash benchflow run skillsbench --agent # run all tasks benchflow run skillsbench/ # run a single task ``` Supported agents: see [Getting started](https://www.benchflow.ai/docs/benchflow/getting-started.md). The runtime ships with verified ACP support for Claude Code, Codex, Gemini CLI, OpenCode, OpenHands, OpenClaw, and Pi. # Coworking Xiangyi works out of [Founders, Inc.](https://f.inc/) at [2 Marina Blvd, San Francisco](https://share.google/7oQr4XWnOuCl5rigs). Feel free to drop by if you are in the Bay. We can also host coworking sessions on a given work day.