---
slug: "agent-auditor"
source_type: "readme"
source_url: "https://cdn.jsdelivr.net/gh/basher83/agent-auditor@main/README.md"
repo: "https://github.com/basher83/agent-auditor"
source_file: "README.md"
branch: "main"
---
# Agent Auditor

Audits Claude Code skills, agents, and components for compliance and effectiveness.

**Status**: v1.0-ready. The skill auditor is production-ready for immediate use.

## Overview

Validates Claude Code skills against official Anthropic specifications.
Ensures skills trigger reliably through auto-invocation.

## Structure

```text
agent-auditor/
├── docs/                    # Documentation
│   ├── research/            # Research and analysis
│   └── guides/              # User guides
├── src/                     # Source code
│   ├── skill_auditor/       # Python SDK
│   └── tests/              # Test suite
├── agents/                  # Claude Code agent definitions
├── commands/                # Claude Code command definitions
└── MIGRATION.md            # Migration checklist
```

## What's Included

### Research & Documentation

- Architecture decisions and design rationale
- Problem analysis and root cause investigations
- Implementation documentation and plans
- Testing results and validation reports
- Agent evolution history
- Development notes and observations

### Source Code

- Python SDK for deterministic skill auditing
- Metrics extraction and validation logic
- Test suite with comprehensive coverage
- CLI application

### Agent Definitions

- Skill-auditor agents (v3-v6)
- Progression from non-deterministic to deterministic approaches

### Commands

- Commands for skill validation and auditing

## Quick Start

### Installation

```bash
# Install from source
uv pip install -e .
```

### Usage

Three modes available:

**Fast Mode (Default)** - Deterministic validation

```bash
# Using module
uv run python -m skill_auditor.cli /path/to/skill/directory

# Or using console script
skill-auditor /path/to/skill/directory
```

**Explain Mode** - AI-powered analysis and fix suggestions

```bash
# Add --explain flag
uv run python -m skill_auditor.cli /path/to/skill/directory --explain

# Or with console script
skill-auditor /path/to/skill/directory --explain
```

**Hybrid Mode** - JSON output for agent integration

```bash
# Machine-readable JSON with metrics and evidence
skill-auditor /path/to/skill/directory --hybrid
```

### Examples

**Fast Mode** (instant, free):

```bash
$ uv run python -m skill_auditor.cli .claude/skills/my-skill

🔍 Auditing skill: .claude/skills/my-skill
============================================================

📊 Extracting metrics...
✅ Extracted 16 metrics
   - Line count: 450
   - YAML delimiters: 2
   - Forbidden files: 0

📊 Running deterministic validation...

🟢 READY
```

**Explain Mode** (~2-3s, ~$0.004):

```bash
$ uv run python -m skill_auditor.cli .claude/skills/my-skill --explain

🔍 Auditing skill: .claude/skills/my-skill
============================================================

📊 Extracting metrics...
✅ Extracted 16 metrics

🤖 Using Claude for detailed analysis...

# Skill Audit Report: my-skill

**Status:** 🔴 BLOCKED

[Detailed explanations and fix suggestions from Claude...]
```

**Hybrid Mode** (instant, machine-readable):

```bash
$ skill-auditor .claude/skills/my-skill --hybrid

{
  "skill_name": "my-skill",
  "status": "READY",
  "summary": {"blockers_count": 0, "warnings_count": 0},
  "blockers": [],
  "warnings": [],
  "metrics": {...},
  "evidence": {"description_lines": [...], "description_text": "..."}
}
```

## Key Concepts

- **Deterministic Auditing**: Python extracts metrics consistently across runs
- **Three Modes**: Fast validation (default), AI explanations (--explain), or JSON output (--hybrid)
- **Effectiveness Validation**: Checks both compliance and auto-invocation potential
- **Progressive Disclosure**: Validates skills follow correct information architecture

## Architecture

Two-tier validation:

```mermaid
flowchart TD
    A[SKILL.md] --> B[Python: Extract Metrics]
    B --> C{Deterministic<br/>Validation}
    C --> D[Calculate Checks<br/>B1-B5, W1-W2]
    D --> E{Mode?}

    E -->|Fast Mode<br/>default| F[Format Results]
    F --> G[Terminal Output<br/>READY / BLOCKED]

    E -->|--explain| H[Build Context]
    H --> I[Claude Analysis]
    I --> J[Semantic Evaluation]
    J --> K[Detailed Report<br/>+ Fix Suggestions]

    style C fill:#e1f5ff
    style D fill:#e1f5ff
    style F fill:#e1f5ff
    style I fill:#fff4e1
    style J fill:#fff4e1
    style G fill:#d4edda
    style K fill:#d4edda
```

**1. Deterministic Layer (Python)** - Always runs

- Extracts metrics from SKILL.md (see `rules.py` for source of truth)
- Validates official requirements:
  - B1: No forbidden files (README*, INSTALL*, CHANGELOG*, QUICK*)
  - B2: Valid YAML frontmatter (name + description)
  - B3: Valid name format (hyphen-case, max 40 chars)
  - B4: Description length (max 1024 chars)
  - B5: No angle brackets in description
- Checks soft recommendations:
  - W1: SKILL.md under 500 lines
  - W2: No unexpected frontmatter properties
- Fast, free, reproducible

**2. Semantic Layer (Claude)** - Optional `--explain` flag

- Explains violations in context
- Suggests specific fixes
- Adds recommendations
- Costs ~$0.004 per audit, 2-3 seconds