pi-ast-sgrep

内容来源：README.md（说明文档） · 原始地址 · 查看安装指南

原始内容

<p align="center">
  <img src=".github/assets/banner.svg" width="100%" alt="ast-sgrep">
</p>

# ast-sgrep

**Hybrid code search that understands intent** -- not only text or syntax.

**v1.3.2** · 8 languages · lexical + AST graph + **semantic symbol search** (on by default, no API key)

> **ast-grep finds shapes. ripgrep finds strings. ast-sgrep finds intent.**

---

## Install

For Pi, install the native package directly:

```bash
pi install npm:pi-ast-sgrep
```

It immediately adds `asgrep_search`, `asgrep_index`, `asgrep_status`, four `/asgrep-*` commands, and the `ast-sgrep` skill. The first search lazily creates `.asgrep/`; no Rust toolchain, PATH setup, MCP adapter, credential, or runtime download is required. See the [complete Pi package guide](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/pi-package.md) for supported hosts, data/privacy behavior, updates, rollback, troubleshooting, and uninstall.

For the standalone CLI, after `1.3.2` is published to crates.io, install with:

```bash
cargo install ast-sgrep-cli --version 1.3.2 --locked
```

Until publication, build from source:

```bash
git clone https://github.com/AdityaVG13/ast-sgrep
cd ast-sgrep
cargo build --release -p ast-sgrep-cli
./target/release/asgrep --help
```

Standalone binaries: `asgrep` and `ast-sgrep` (aliases).

On Unix, the CLI runs commands through the process supervisor. `ASGREP_CPU_LIMIT_PERCENT`
sets the worker process runnable wall-time fraction in each 10 ms SIGSTOP/CONT cycle; it is not
a machine-wide or one-core CPU percentage, and multi-threaded work may consume several cores
while runnable. On Windows, commands run directly: search, indexing, cancellation, and path
handling are supported, but the duty cycle is not enforced.

---

## Easy start (agents)

Paste into your agent:

```text
Clone https://github.com/AdityaVG13/ast-sgrep, cd into it, run `cargo build --release -p ast-sgrep-cli`.
Register target/release/asgrep-mcp as a stdio MCP server named "ast-sgrep" (build with: cargo build --release -p ast-sgrep-mcp).
Verify: run ./target/release/asgrep index . then search for defs: of a symbol in this repo.
```

---

## Why this exists

Most code search is either **fast text** (ripgrep) or **pattern matching** (ast-grep). Neither answers questions like *"where does credential renewal happen?"* when the words in your question do not appear in the code.

**ast-sgrep** builds a **persistent index**: symbols, caller/callee edges, imports, lexical FTS, and **symbol-level semantic vectors** enriched with call-graph context. Query in natural language or with graph prefixes; get ranked hits with excerpts for humans or agents.

**No API key required.** Offline semantic search works out of the box. Cloud, Ollama, and optional neural embeddings are upgrades.

| You need... | ast-sgrep gives you... |
|-------------|------------------------|
| Where is X defined? | `defs:` + ranked hybrid hits |
| Who calls this? | `callers:` + call hierarchy (LSP) |
| How does auth refresh work? | NL → symbols + anchors + semantic similarity |
| "credential renewal" (no token overlap) | Semantic hit on `auth_refresh` |
| Structured JSON for an agent | `--json --format agent` |
| Structural rewrite / codemod | `pattern:` (ast-grep when available) |

[Full comparison →](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/comparison.md)

---

## Where it fits

ast-sgrep **complements** ripgrep and ast-grep; it does not replace them.

| Tool | Role |
|------|------|
| **[ripgrep](https://github.com/BurntSushi/ripgrep)** | Fast scan of any file. No index. |
| **[ast-grep](https://github.com/ast-grep/ast-grep)** | Structural patterns and codemods |
| **ast-sgrep** | Persistent navigation + intent: NL, defs/callers/graph, semantic hits, agent JSON |

---

## Quick start

Index is incremental and lives under the project root at `.asgrep/`.

```bash
cargo build --release -p ast-sgrep-cli
./target/release/asgrep index .
./target/release/asgrep 'defs:auth_refresh' . --limit 3
./target/release/asgrep semantic 'credential renewal' . --limit 3
./target/release/asgrep chain 'auth_refresh' . --limit 3
```

Unprefixed queries run **hybrid** retrieval. See the [query grammar](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/QUERY_GRAMMAR.md).

[Getting started →](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/getting-started.md) · [Architecture →](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/ARCHITECTURE.md) · [Docs index →](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/README.md)

---

## What "semantic" means here

ast-sgrep embeds **symbol chunks** (function/method/type with name, kind, callers, callees, excerpt), expanded with code-domain concept groups (auth ↔ credential ↔ token, refresh ↔ renewal, …).

```text
Query: "credential renewal"
  → semantic pass ranks auth_refresh (zero token overlap)
```

Provider chain: **Cloud** (if key) → **Ollama** (if configured) → **neural** (optional feature) → **local semantic** (always available). Large repos may use a persisted IVF-ANN sidecar (`.asgrep/semantic.ivf`).

[Semantic layer →](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/semantic-search.md)

---

## Benchmarks (honest summary)

These are **checked-in run summaries**, not portable guarantees. Hardware, corpus, cache state, and flags all matter.

| Recorded comparison | Published result | Evidence |
|---------------------|------------------|----------|
| Warm lexical suite vs ripgrep | Strong on recorded cases | [speed.md](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/benchmarks/results/speed.md) |
| Structural workloads vs ast-grep | Large speedups in recorded cases | [speed.md](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/benchmarks/results/speed.md) |
| Cross-tool bake-off | Mixed; inspect every row | [bakeoff.md](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/benchmarks/results/bakeoff.md) |
| Known regressions | Published without suppression | [losses.md](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/benchmarks/results/losses.md) |

Canonical table: [head-to-head.md](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/benchmarks/results/head-to-head.md). Index: [benchmarks/README.md](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/benchmarks/README.md). Methodology: [docs/benchmarks.md](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/benchmarks.md).

**Quality snapshot (self corpus, 18 labeled gold queries):** hybrid MRR **0.712**, Recall@k **0.889**, nDCG@k **0.751**. The canonical row and its reproduction status are in [baselines.md](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/benchmarks/results/baselines.md#retrieval-quality--self-corpus-18-gold-queries). On some foreign corpora the offline embedder currently adds little over lexical + AST; a stronger local model is on the roadmap.
**Historical quality snapshot (self corpus, labeled gold):** hybrid MRR ≈ 0.75, Recall@k ≈ 0.94 (see docs/benchmarks for commands). These are point estimates from an historical evaluation whose sample size and confidence intervals were not recorded; they are not product guarantees and must not be used to claim statistically significant improvements. On some foreign corpora the offline embedder currently adds little over lexical + AST; a stronger local model is on the roadmap.

---

## Interfaces

| Interface | Build | Use case |
|-----------|-------|----------|
| **CLI** | `cargo build --release -p ast-sgrep-cli` | Terminal, scripts |
| **MCP** | `cargo build --release -p ast-sgrep-mcp` | AI agents (stdio) |
| **LSP** | `cargo build --release -p ast-sgrep-lsp` | Editor navigation |
| **Library** | `ast-sgrep-core` | Embed search in Rust tools |
| **JSON plugins** | `--format agent\|github\|gitlab\|agent-capsule` | Agents / CI |

---

## Documentation

| Doc | Contents |
|-----|----------|
| [docs/README.md](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/README.md) | Full documentation index |
| [Getting started](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/getting-started.md) | Install, index, queries, flags |
| [Pi package guide](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/pi-package.md) | Pi install, tools, data, security, updates, rollback, uninstall |
| [Architecture](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/ARCHITECTURE.md) | Index schema, search pipeline, crates |
| [Query grammar](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/QUERY_GRAMMAR.md) | Prefixes and composition |
| [Semantic search](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/semantic-search.md) | Chunks, providers, IVF-ANN |
| [Benchmarks](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/benchmarks.md) | Methodology, reproduction, losses |
| [Comparison](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/comparison.md) | vs ripgrep / ast-grep |
| [MCP](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/mcp.md) · [Use cases](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/use-cases.md) · [Releasing](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/docs/RELEASING.md) | Agents, LSP, release checklist |

---

## Workspace layout

| Path | Role |
|------|------|
| `crates/ast-sgrep-core` | Index, SQLite store, hybrid search |
| `crates/ast-sgrep-cli` | `asgrep` / `ast-sgrep` CLI + supervisor |
| `crates/ast-sgrep-lang` | Tree-sitter extraction (8 languages) |
| `crates/ast-sgrep-embed` | Embedding backends + optional rerank |
| `crates/ast-sgrep-lsp` | Language server |
| `crates/ast-sgrep-mcp` | MCP server |
| `crates/ast-sgrep-plugins` | Output formats |
| `crates/ast-sgrep-testkit` | Shared test fixtures |
| `benchmarks/` | Published results (`results/`) and studies (`studies/`) |
| `docs/` | User and architecture docs |
| `tests/fixtures/` | Sample corpora for tests |

---

## Project status and verification

**v1.3.2.** Hybrid search, semantic layer, LSP, MCP, agent JSON, and IVF path are in place.

GitHub Actions workflows are **manual-only** (`workflow_dispatch`) to control Actions minutes. Local quality bar for contributors:

```bash
cargo check --workspace -j1
cargo test -p ast-sgrep-core --test parity -j1 -- --test-threads=1
cargo build --release -p ast-sgrep-cli -j1
./target/release/asgrep --help
```

See [CONTRIBUTING.md](https://github.com/AdityaVG13/ast-sgrep/blob/HEAD/CONTRIBUTING.md). Optional full-workspace tests and CI jobs remain available when you intentionally run them.

---

## License

MIT. See [LICENSE](https://github.com/AdityaVG13/ast-sgrep/tree/HEAD/LICENSE).

ast-sgrep

Hybrid code search that understands intent -- not only text or syntax.

v1.3.2 · 8 languages · lexical + AST graph + semantic symbol search (on by default, no API key)

ast-grep finds shapes. ripgrep finds strings. ast-sgrep finds intent.

Install

For Pi, install the native package directly:

pi install npm:pi-ast-sgrep

It immediately adds asgrep_search, asgrep_index, asgrep_status, four /asgrep-* commands, and the ast-sgrep skill. The first search lazily creates .asgrep/; no Rust toolchain, PATH setup, MCP adapter, credential, or runtime download is required. See the complete Pi package guide for supported hosts, data/privacy behavior, updates, rollback, troubleshooting, and uninstall.

For the standalone CLI, after 1.3.2 is published to crates.io, install with:

cargo install ast-sgrep-cli --version 1.3.2 --locked

Until publication, build from source:

git clone https://github.com/AdityaVG13/ast-sgrep
cd ast-sgrep
cargo build --release -p ast-sgrep-cli
./target/release/asgrep --help

Standalone binaries: asgrep and ast-sgrep (aliases).

On Unix, the CLI runs commands through the process supervisor. ASGREP_CPU_LIMIT_PERCENT sets the worker process runnable wall-time fraction in each 10 ms SIGSTOP/CONT cycle; it is not a machine-wide or one-core CPU percentage, and multi-threaded work may consume several cores while runnable. On Windows, commands run directly: search, indexing, cancellation, and path handling are supported, but the duty cycle is not enforced.

Easy start (agents)

Paste into your agent:

Clone https://github.com/AdityaVG13/ast-sgrep, cd into it, run `cargo build --release -p ast-sgrep-cli`.
Register target/release/asgrep-mcp as a stdio MCP server named "ast-sgrep" (build with: cargo build --release -p ast-sgrep-mcp).
Verify: run ./target/release/asgrep index . then search for defs: of a symbol in this repo.

Why this exists

Most code search is either fast text (ripgrep) or pattern matching (ast-grep). Neither answers questions like "where does credential renewal happen?" when the words in your question do not appear in the code.

ast-sgrep builds a persistent index: symbols, caller/callee edges, imports, lexical FTS, and symbol-level semantic vectors enriched with call-graph context. Query in natural language or with graph prefixes; get ranked hits with excerpts for humans or agents.

No API key required. Offline semantic search works out of the box. Cloud, Ollama, and optional neural embeddings are upgrades.

You need...	ast-sgrep gives you...
Where is X defined?	`defs:` + ranked hybrid hits
Who calls this?	`callers:` + call hierarchy (LSP)
How does auth refresh work?	NL → symbols + anchors + semantic similarity
"credential renewal" (no token overlap)	Semantic hit on `auth_refresh`
Structured JSON for an agent	`--json --format agent`
Structural rewrite / codemod	`pattern:` (ast-grep when available)

Full comparison →

Where it fits

ast-sgrep complements ripgrep and ast-grep; it does not replace them.

Tool	Role
ripgrep	Fast scan of any file. No index.
ast-grep	Structural patterns and codemods
ast-sgrep	Persistent navigation + intent: NL, defs/callers/graph, semantic hits, agent JSON

Quick start

Index is incremental and lives under the project root at .asgrep/.

cargo build --release -p ast-sgrep-cli
./target/release/asgrep index .
./target/release/asgrep 'defs:auth_refresh' . --limit 3
./target/release/asgrep semantic 'credential renewal' . --limit 3
./target/release/asgrep chain 'auth_refresh' . --limit 3

Unprefixed queries run hybrid retrieval. See the query grammar.

Getting started → · Architecture → · Docs index →

What "semantic" means here

ast-sgrep embeds symbol chunks (function/method/type with name, kind, callers, callees, excerpt), expanded with code-domain concept groups (auth ↔ credential ↔ token, refresh ↔ renewal, …).

Query: "credential renewal"
  → semantic pass ranks auth_refresh (zero token overlap)

Provider chain: Cloud (if key) → Ollama (if configured) → neural (optional feature) → local semantic (always available). Large repos may use a persisted IVF-ANN sidecar (.asgrep/semantic.ivf).

Semantic layer →

Benchmarks (honest summary)

These are checked-in run summaries, not portable guarantees. Hardware, corpus, cache state, and flags all matter.

Recorded comparison	Published result	Evidence
Warm lexical suite vs ripgrep	Strong on recorded cases	speed.md
Structural workloads vs ast-grep	Large speedups in recorded cases	speed.md
Cross-tool bake-off	Mixed; inspect every row	bakeoff.md
Known regressions	Published without suppression	losses.md

Canonical table: head-to-head.md. Index: benchmarks/README.md. Methodology: docs/benchmarks.md.

Quality snapshot (self corpus, 18 labeled gold queries): hybrid MRR 0.712, Recall@k 0.889, nDCG@k 0.751. The canonical row and its reproduction status are in baselines.md. On some foreign corpora the offline embedder currently adds little over lexical + AST; a stronger local model is on the roadmap. Historical quality snapshot (self corpus, labeled gold): hybrid MRR ≈ 0.75, Recall@k ≈ 0.94 (see docs/benchmarks for commands). These are point estimates from an historical evaluation whose sample size and confidence intervals were not recorded; they are not product guarantees and must not be used to claim statistically significant improvements. On some foreign corpora the offline embedder currently adds little over lexical + AST; a stronger local model is on the roadmap.

Interfaces

Interface	Build	Use case
CLI	`cargo build --release -p ast-sgrep-cli`	Terminal, scripts
MCP	`cargo build --release -p ast-sgrep-mcp`	AI agents (stdio)
LSP	`cargo build --release -p ast-sgrep-lsp`	Editor navigation
Library	`ast-sgrep-core`	Embed search in Rust tools
JSON plugins	`--format agent\|github\|gitlab\|agent-capsule`	Agents / CI

Documentation

Doc	Contents
docs/README.md	Full documentation index
Getting started	Install, index, queries, flags
Pi package guide	Pi install, tools, data, security, updates, rollback, uninstall
Architecture	Index schema, search pipeline, crates
Query grammar	Prefixes and composition
Semantic search	Chunks, providers, IVF-ANN
Benchmarks	Methodology, reproduction, losses
Comparison	vs ripgrep / ast-grep
MCP · Use cases · Releasing	Agents, LSP, release checklist

Workspace layout

Path	Role
`crates/ast-sgrep-core`	Index, SQLite store, hybrid search
`crates/ast-sgrep-cli`	`asgrep` / `ast-sgrep` CLI + supervisor
`crates/ast-sgrep-lang`	Tree-sitter extraction (8 languages)
`crates/ast-sgrep-embed`	Embedding backends + optional rerank
`crates/ast-sgrep-lsp`	Language server
`crates/ast-sgrep-mcp`	MCP server
`crates/ast-sgrep-plugins`	Output formats
`crates/ast-sgrep-testkit`	Shared test fixtures
`benchmarks/`	Published results (`results/`) and studies (`studies/`)
`docs/`	User and architecture docs
`tests/fixtures/`	Sample corpora for tests

Project status and verification

v1.3.2. Hybrid search, semantic layer, LSP, MCP, agent JSON, and IVF path are in place.

GitHub Actions workflows are manual-only (workflow_dispatch) to control Actions minutes. Local quality bar for contributors:

cargo check --workspace -j1
cargo test -p ast-sgrep-core --test parity -j1 -- --test-threads=1
cargo build --release -p ast-sgrep-cli -j1
./target/release/asgrep --help

See CONTRIBUTING.md. Optional full-workspace tests and CI jobs remain available when you intentionally run them.

License

MIT. See LICENSE.