SkillsBench investigates whether Agent Skills—reusable packages of procedural instructions, scripts, and reference materials—help AI agents complete tasks, and how their effectiveness varies across models and agent harnesses. Unlike benchmarks that report only overall performance, SkillsBench isolates the contribution of Skills by comparing the same agents on the same tasks with and without them. The benchmark comprises 87 containerized tasks across eight domains, including software engineering, natural science, cybersecurity, finance, office work, and industrial systems. The tasks were selected from 400 submissions by 142 contributors. Each task includes a human-written instruction, a fixed environment, a reference solution, and a deterministic verifier. Skills were curated separately from task-specific solutions, and task instructions do not tell agents which Skills to use. This design aims to ensure that Skills provide reusable expertise rather than hidden answers. The authors evaluate 18 model–harness configurations across four terminal-agent harnesses. Each configuration is tested under matched no-Skills and curated-Skills conditions. Each task-condition cell targets three trials; scores are calculated by averaging pass rates across trials and then across tasks. The study also examines domain differences, Skill design and invocation, and a separate condition in which agents generate their own Skills. Curated Skills raise the average task-macro pass rate from 33.9% to 50.5%, a gain of 16.6 percentage points, or a 25.5% normalized gain. All 18 configurations improve, although the size of the lift varies substantially, from 4.1 to 25.7 points. OpenHands with GPT-5.5 achieves the highest pass rate with Skills, at 67.3%, while OpenHands with GLM 5.1 records the largest absolute gain, at 25.7 points. Harness choice also matters. Gemini 3.1 Pro reaches 60.8% with Skills in Gemini CLI, compared with 52.8% in OpenHands. Claude Opus 4.7 reaches 61.2% in Claude Code and 53.1% in OpenHands. The authors conclude that Skill efficacy depends on the entire model–harness system, not just on the Skill itself. Benefits vary by domain. Natural Science shows the largest average increase, at 28.8 points, followed by Media and Content Production at 24.1 points and Cybersecurity at 18.9 points. Mathematics and Operations Research gain 9.7 points, while Software Engineering gains 11.6. Skills appear especially valuable for tasks involving specialized procedures, data conventions, or precise output requirements. Skills can also reduce performance. Thirteen tasks show negative average deltas, often because a Skill recommends an unnecessarily complex workflow, displaces a better default approach, or leads the agent toward a brittle method it cannot debug. Design analyses find that tasks with one to three Skills gain more than those with four or more, and that compact or standard-length documentation outperforms comprehensive prose. Self-generated Skills underperform the no-Skills baseline in all three tested configurations, by 8.1 to 11.5 points. By contrast, curated Skills improve those same configurations by 18.2 to 24.8 points. Overall, the findings suggest that well-curated, focused Skills can substantially improve agent performance and sometimes help smaller models approach the unaugmented results of stronger models. However, the benchmark represents an optimistic setting: it uses high-quality curated Skills, containerized terminal tasks, a limited set of models and harnesses, and deterministic verifiers. The authors also note that additional context may contribute to the gains, so stronger length-matched controls are needed to distinguish procedural guidance from the effect of simply providing more text.
- SkillsBench measures the impact of reusable Agent Skills by comparing the same agents on the same tasks with and without Skills, isolating their contribution rather than reporting only overall performance.
- The benchmark covers 87 containerized tasks across eight domains, selected from 400 submissions by 142 contributors. Tasks include human-written instructions, fixed environments, reference solutions, and deterministic verifiers.
- Skills were curated separately from task solutions, and task instructions do not tell agents which Skills to use. This is intended to test reusable expertise rather than access to hidden answers.
- The evaluation tested 18 model–harness configurations across four terminal-agent harnesses under matched conditions. Each task-condition cell targeted three trials, with scores averaged across trials and tasks.
- Curated Skills improved average task-macro pass rates from 33.9% to 50.5%—a 16.6-point increase, or a 25.5% normalized gain. All 18 configurations improved, though gains ranged from 4.1 to 25.7 points.
- Results depend on the model–harness combination. OpenHands with GPT-5.5 achieved the highest Skills-enabled pass rate at 67.3%; OpenHands with GLM 5.1 had the largest gain at 25.7 points. The same model also performed differently across harnesses.
- Benefits varied by domain: Natural Science gained the most on average (28.8 points), followed by Media and Content Production (24.1) and Cybersecurity (18.9). Skills helped less in Mathematics and Operations Research (9.7) and Software Engineering (11.6).
- Skills can sometimes hurt performance. Thirteen tasks had negative average changes, often because Skills encouraged overly complex workflows, displaced better default approaches, or suggested brittle methods.
- Focused Skills performed best: Tasks with one to three Skills gained more than those with four or more, and compact or standard-length documentation outperformed comprehensive prose.
- Self-generated Skills underperformed: In all three tested configurations, they fell 8.1–11.5 points below the no-Skills baseline, while curated Skills improved those same configurations by 18.2–24.8 points.
- The findings are promising but have limitations. The benchmark uses high-quality curated Skills, terminal-based tasks, a limited set of models and harnesses, and deterministic verifiers.
Summary1. SkillsBench checks whether reusable computer instructions help AI agents solve tasks.
2. It tested 87 tasks in eight areas using many different AI setups.
3. With carefully chosen Skills, agents solved more tasks: their average score rose from about 34% to about 51%.
4. Skills helped most in science, but sometimes made agents do worse, especially when there were too many instructions.
5. The results look helpful, but they may not apply to every kind of AI or task.
Definitions- Agent: A computer program that can try to complete tasks.
- Skill: An instruction an agent can reuse to help with a task.
- Benchmark: A set of tests used to compare how well something works.
- Domain: A subject area, such as science or mathematics.
- Curated: Carefully chosen and prepared.
Do AI Agents Benefit from “Skills”? A Benchmark Puts Reusable Instructions to the Test
AI agents are often given more than a prompt. They may also have access to “Skills”: reusable packages of instructions, scripts, and reference materials intended to help them carry out particular kinds of work. But do these add-ons actually improve performance, or do they simply add more text for an agent to process?
SkillsBench takes a direct approach to answering that question. Instead of reporting only how well agents perform overall, it compares the same model-and-agent setups on the same tasks with and without curated Skills. The result is a detailed look at when reusable expertise helps, when it does not, and why the surrounding agent system matters.
A benchmark designed to isolate the effect of Skills
SkillsBench contains 87 containerized tasks across eight domains, including software engineering, natural science, cybersecurity, finance, office work, and industrial systems. The tasks were selected from 400 submissions by 142 contributors. Each one comes with a human-written instruction, a fixed environment, a reference solution, and a deterministic verifier to assess whether the task was completed.
A key design choice separates Skills from answers. Skills were curated independently of task-specific solutions, and task instructions do not tell agents which Skills to use. This is intended to make Skills function as reusable expertise rather than as hidden task solutions.
The researchers evaluated 18 model–harness configurations across four terminal-agent harnesses. Each configuration was tested under matched conditions: once without Skills and once with curated Skills. The benchmark targeted three trials for each task-condition combination, averaged pass rates across trials, and then averaged scores across tasks. The study also explored domain differences, Skill design and invocation, and a separate setup in which agents generated their own Skills.
Curated Skills improve average performance
Across the benchmark, curated Skills raised the average task-macro pass rate from 33.9% to 50.5%. That is a gain of 16.6 percentage points, or a 25.5% normalized improvement. All 18 tested configurations improved, though the size of the lift ranged widely, from 4.1 to 25.7 percentage points.
The strongest result with Skills came from OpenHands with GPT-5.5, which reached a 67.3% pass rate. OpenHands with GLM 5.1 had the largest absolute improvement, gaining 25.7 points. The spread suggests that Skills can help broadly, but they do not provide a uniform boost.
The harness is part of the result
The same model can perform differently depending on the agent harness it uses. Gemini 3.1 Pro reached 60.8% with Skills in Gemini CLI, compared with 52.8% in OpenHands. Claude Opus 4.7 achieved 61.2% in Claude Code and 53.1% in OpenHands.
In other words, Skill effectiveness is not a property of the instructions alone. It depends on the whole model–harness system: how the model interprets guidance, how the harness presents it, and how the agent carries out the work.
Some domains benefit more than others
Skills produced their largest average improvement in Natural Science, at 28.8 percentage points. Media and Content Production followed at 24.1 points, while Cybersecurity gained 18.9. Mathematics and Operations Research improved by 9.7 points, and Software Engineering by 11.6.
This pattern suggests that Skills are especially useful when success depends on specialized procedures, unfamiliar data conventions, or precise output requirements. A compact guide to a particular workflow may offer more value in such situations than in tasks where an agent can rely on general-purpose problem-solving.
More guidance is not always better
SkillsBench also documents cases where added guidance hurts. Thirteen tasks had negative average changes. In some, a Skill encouraged an unnecessarily complicated workflow. In others, it displaced a better default approach or pushed the agent toward a brittle method that it could not debug.
The design analyses point toward focus and restraint. Tasks with one to three Skills gained more than tasks with four or more. Compact or standard-length documentation also outperformed comprehensive prose. Together, these findings challenge the idea that an agent benefits simply from having more instructions available.
Self-generated Skills underperform
The study’s self-generated Skills condition offers a striking contrast. In all three tested configurations, agents that created their own Skills performed 8.1 to 11.5 points below the no-Skills baseline. On those same configurations, curated Skills improved performance by 18.2 to 24.8 points.
The result highlights a difference between having access to useful procedures and being able to create reliable reusable guidance from scratch. For now, careful curation appears to matter more than asking an agent to build its own playbook.
Promising results, with important caveats
SkillsBench presents a strong case that well-designed, focused Skills can improve agent performance. In some cases, they may even help a smaller model approach the unaugmented results of a stronger one. But the findings come from an optimistic setting: high-quality curated Skills, containerized terminal tasks, a limited group of models and harnesses, and deterministic verifiers.
There is also an open question about what causes the gains. Skills provide procedural guidance, but they also provide additional context. The authors note that stronger length-matched controls are needed to separate the effect of useful instructions from the effect of simply giving an agent more text.
For developers building agent systems, the practical message is clear: reusable Skills can be valuable, but quality, scope, and integration matter. A small, well-targeted guide may help more than a comprehensive manual—and a poorly matched Skill can steer an otherwise capable agent in the wrong direction.
