Find tools › Open-source projects › Test & monitor AI apps

SkillEvaluator

Open-source project · by NVIDIA

Tests custom instructions and tools for AI helpers to make sure they work safely and improve results.

Some setupWell known3,485 installs a month

What it costs

Free and open source; you may pay for the AI model it uses.

Use it to

  • Validate agent skill formatting and security before publishing
  • Detect duplicate or overlapping guidance across skill libraries
  • Benchmark live agent performance with and without a skill

How to start

  1. Install the tool with pip install skillevaluator.

  2. Run basic static checks without any model keys using skillevaluator validate ./my-skill --tiers 1.

  3. Configure model keys and a Docker sandbox to measure actual agent task performance gains.

Good to know. Full live agent evaluation requires a running container sandbox such as Docker and model provider credentials that can incur API usage costs.

Before you install

  • Found automaticallyNot reviewed by a person yet. Check it before installing.
  • License: Apache-2.0You are allowed to use it under these terms.
  • Last updated 2026-11-18

More detail

SkillEvaluator is an evaluation framework that tests AI agent skills for quality, security compliance, duplication, and measurable performance impacts.

Skills given to AI coding agents can contain syntax errors, security flaws, or contradictory instructions that degrade task success. Reach for this when building, auditing, or adding continuous integration checks for agent skills.

Similar tools

Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-11-15. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map