loki-adityamiskin

内容来源：README.md（说明文档） · 原始地址 · 查看安装指南

原始内容

# Loki

Loki is a terminal-first security research pipeline for source repositories. It
combines deterministic scanning with coding agents to find, investigate,
prioritize, verify, and prepare fixes for vulnerabilities.

It runs locally by default. There is no web application or server to deploy.
Pi provides the interactive terminal UI, while the CLI remains the complete
automation surface.

## How it works

Loki is a resumable MapReduce-style pipeline:

```mermaid
flowchart LR
    Repo["Source repository"] --> Plan["1. Plan<br/>Threat model + selectors"]
    Plan --> Scan["2. Scan<br/>Deterministic candidates"]
    Scan --> Process["3. Process · Map<br/>Parallel agent investigation"]
    Process --> Reduce["4. Reduce<br/>Dedupe + attack chains + priority"]
    Reduce --> Verify["5. Verify<br/>Static or runtime evidence"]
    Verify --> Fix["6. Fix<br/>Agent-ready remediation bundle"]
```

1. **Plan** learns the shape and trust boundaries of the target repository. It
   writes a threat model and selectors that focus the scan.
2. **Scan** applies deterministic matchers to selected source files. This is
   fast, local, and requires no AI provider.
3. **Process** fans candidates out to coding-agent workers. Each worker reads
   the relevant code, follows data flow, and records structured findings.
4. **Reduce** reasons across all findings, removes duplication, connects
   multi-step attack chains, and assigns global priority.
5. **Verify** gathers stronger evidence for the most important findings using
   offline checks, static agents, or an isolated runtime.
6. **Fix** packages the finding, evidence, affected code, constraints, and
   validation guidance so a coding agent or engineer can remediate it safely.

Every stage checkpoints its work under `.loki/data/<project-id>/`. If a run is
interrupted, running it again resumes from the saved state instead of repeating
completed work.

### The core data flow

```mermaid
flowchart TB
    Pi["Pi extension<br/>custom TUI"] --> Bridge["Constrained CLI bridge"]
    Shell["CLI / scripts / CI"] --> CLI
    Bridge --> CLI["@adityamiskin/loki<br/>commands + orchestration"]

    CLI --> Core["@loki/core<br/>schemas · config · durable records · plugins"]
    CLI --> Scanner["@loki/scanner<br/>selectors · matchers · candidates"]
    CLI --> Processor["@loki/processor<br/>agents · reduce · verify · fix"]

    Scanner --> State[(".loki/data/&lt;project-id&gt;<br/>shared project state")]
    Processor <--> State
    Core <--> State
    State --> Pi
    State --> Outputs["Reports · exports · fix bundles"]
```

The unit of work is a source file represented by a durable `FileRecord`.
Scanning adds candidates to it; processing adds findings and analysis history;
triage, revalidation, enrichment, and verification add more evidence. The CLI,
Pi UI, reports, and exports all read the same records—there is no separate
database or web backend.

## Use Loki in Pi

`@adityamiskin/loki-pi` adds a native interface to Pi:

- `/loki` opens the custom dashboard
- overview and finding views read the current Loki workspace
- severity filters and full finding details make review interactive
- scans show live progress
- Loki tools get purpose-built terminal rendering

Try it from this checkout:

```bash
bun install
pi -e ./packages/loki-pi
```

Open Pi in a repository containing a `.loki` workspace, then run:

```text
/loki
```

The extension currently exposes project status, findings, and deterministic
scanning. Paid agent stages remain available through the CLI.

## Use Loki from the CLI

From the root of the repository you want to inspect:

```bash
bunx @adityamiskin/loki init
cd .loki
bun install
```

`init` registers the surrounding repository and creates the Loki workspace.
Before the first serious scan, have a coding agent read the generated
`SETUP.md` and fill in `INFO.md` with concise, project-specific context.

Run the full primary pipeline:

```bash
bun run loki swarm --project-id <project-id> --root ..
```

Or run it stage by stage:

```bash
bun run loki plan    --project-id <project-id> --root ..
bun run loki scan    --project-id <project-id> --root ..
bun run loki process --project-id <project-id> --root ..
bun run loki reduce  --project-id <project-id>
bun run loki verify  --project-id <project-id>
bun run loki fix     --project-id <project-id>
```

Most planning, scanning, reduction, verification, and fix commands have free
offline modes. `process` and any `--agent` mode use a coding-agent backend and
can consume substantial tokens, so begin with a narrow scope.

See [Getting Started](https://github.com/adityamiskin/loki/blob/HEAD/docs/getting-started.md) for the full first-run workflow.
For exact flags, run `bun run loki --help` or
`bun run loki <command> --help` inside the workspace.

## Packages

| Package | Role |
| --- | --- |
| `@loki/core` | Schemas, configuration, durable records, and plugin contracts |
| `@loki/scanner` | Deterministic selectors, matchers, and candidate discovery |
| `@loki/processor` | Agent investigation, reduction, verification, and fix generation |
| `@adityamiskin/loki` | Public CLI, orchestration, and distributable workspace |
| `@adityamiskin/loki-pi` | Pi extension and custom terminal UI |

The packages communicate through typed records from `@loki/core`; interface
packages do not duplicate the scanning or processing engine.

## Commands

| Command | What it does |
| --- | --- |
| `init` | Create a self-contained `.loki/` workspace |
| `init-project` | Register another target repository |
| `plan` | Build a threat model and deterministic selectors |
| `scan` | Generate candidates with matchers; no AI required |
| `process` | Investigate candidates in parallel with coding agents |
| `reduce` | Deduplicate, connect, and prioritize findings |
| `verify` | Collect offline, static-agent, or runtime evidence |
| `fix` | Generate a remediation bundle |
| `swarm` | Run the primary pipeline end to end |
| `revalidate` | Revisit existing findings after deeper review or code changes |
| `triage` | Classify findings by priority |
| `enrich` | Add repository history and ownership context |
| `report` | Produce a project report |
| `export` | Export findings as JSON or Markdown |
| `metrics` | Summarize project and finding metrics |
| `status` | Show saved project state |
| `sandbox` | Run a command through an optional remote sandbox |
| `sandbox-all` | Run a distributed stage through optional remote sandboxes |

## Agents and execution

Loki supports Codex, Claude, and Pi as agent backends. Depending on the chosen
backend, it can use a locally authenticated agent session or explicit provider
credentials and base URLs.

Local execution is the default. For untrusted repositories or larger parallel
runs, use isolation. Vercel Sandbox is one supported remote execution adapter,
and Vercel AI Gateway is one optional provider route; neither is required by
Loki's architecture.

- [Models and agent backends](https://github.com/adityamiskin/loki/blob/HEAD/docs/models.md)
- [Optional Vercel integration](https://github.com/adityamiskin/loki/blob/HEAD/docs/vercel-setup.md)

## Security model

Treat agent-backed Loki stages like a coding agent with shell access to their
execution environment. Deterministic scanning is local code inspection, but
agent and runtime-verification modes may read files, run tools, and execute
project commands.

Use local execution for repositories you trust. Run untrusted, vendored, or
prompt-injection-prone code in an isolated environment with limited credentials
and network access.

## Configuration and state

Loki discovers `loki.config.ts` from the `.loki` workspace. Configuration
controls projects, selectors, matchers, agent backends, models, concurrency,
budgets, verification policy, and optional plugins.

Project state lives in:

```text
.loki/data/<project-id>/
├── project.json
├── INFO.md
├── THREAT_MODEL.md
├── selectors.json
├── files/                 # FileRecords: candidates, findings, evidence
├── runs/                  # resumable run metadata
├── reduce-report.json
├── verify-report.json
├── fix-bundle.md
└── reports/
```

Exact files appear as their stages run. See [Data Layout](https://github.com/adityamiskin/loki/blob/HEAD/docs/data-layout.md)
and [Configuration](https://github.com/adityamiskin/loki/blob/HEAD/docs/configuration.md) for the schemas and options.

## Development

Requires Bun 1.3+ and Node.js 22+.

```bash
git clone https://github.com/adityamiskin/loki.git
cd loki
bun install
bun run validate
```

Useful commands:

```bash
bun run test
bun run typecheck:loki
bun run lint
bun run bundle
```

## Documentation

- [Getting Started](https://github.com/adityamiskin/loki/blob/HEAD/docs/getting-started.md)
- [Architecture](https://github.com/adityamiskin/loki/blob/HEAD/docs/architecture.md)
- [Configuration](https://github.com/adityamiskin/loki/blob/HEAD/docs/configuration.md)
- [Writing Matchers](https://github.com/adityamiskin/loki/blob/HEAD/docs/writing-matchers.md)
- [Plugins](https://github.com/adityamiskin/loki/blob/HEAD/docs/plugins.md)
- [Models](https://github.com/adityamiskin/loki/blob/HEAD/docs/models.md)
- [FAQ](https://github.com/adityamiskin/loki/blob/HEAD/docs/faq.md)
- [Roadmap](https://github.com/adityamiskin/loki/blob/HEAD/ROADMAP.md)

## Origin and license

Loki is an independently maintained evolution of
[DeepSec](https://github.com/vercel-labs/deepsec). Original attribution and
modification notices are retained in [NOTICE](https://github.com/adityamiskin/loki/tree/HEAD/NOTICE).

Licensed under [Apache-2.0](https://github.com/adityamiskin/loki/tree/HEAD/LICENSE).

Loki

Loki is a terminal-first security research pipeline for source repositories. It combines deterministic scanning with coding agents to find, investigate, prioritize, verify, and prepare fixes for vulnerabilities.

It runs locally by default. There is no web application or server to deploy. Pi provides the interactive terminal UI, while the CLI remains the complete automation surface.

How it works

Loki is a resumable MapReduce-style pipeline:

flowchart LR
    Repo["Source repository"] --> Plan["1. Plan<br/>Threat model + selectors"]
    Plan --> Scan["2. Scan<br/>Deterministic candidates"]
    Scan --> Process["3. Process · Map<br/>Parallel agent investigation"]
    Process --> Reduce["4. Reduce<br/>Dedupe + attack chains + priority"]
    Reduce --> Verify["5. Verify<br/>Static or runtime evidence"]
    Verify --> Fix["6. Fix<br/>Agent-ready remediation bundle"]

Plan learns the shape and trust boundaries of the target repository. It writes a threat model and selectors that focus the scan.
Scan applies deterministic matchers to selected source files. This is fast, local, and requires no AI provider.
Process fans candidates out to coding-agent workers. Each worker reads the relevant code, follows data flow, and records structured findings.
Reduce reasons across all findings, removes duplication, connects multi-step attack chains, and assigns global priority.
Verify gathers stronger evidence for the most important findings using offline checks, static agents, or an isolated runtime.
Fix packages the finding, evidence, affected code, constraints, and validation guidance so a coding agent or engineer can remediate it safely.

Every stage checkpoints its work under .loki/data/<project-id>/. If a run is interrupted, running it again resumes from the saved state instead of repeating completed work.

The core data flow

flowchart TB
    Pi["Pi extension<br/>custom TUI"] --> Bridge["Constrained CLI bridge"]
    Shell["CLI / scripts / CI"] --> CLI
    Bridge --> CLI["@adityamiskin/loki<br/>commands + orchestration"]

    CLI --> Core["@loki/core<br/>schemas · config · durable records · plugins"]
    CLI --> Scanner["@loki/scanner<br/>selectors · matchers · candidates"]
    CLI --> Processor["@loki/processor<br/>agents · reduce · verify · fix"]

    Scanner --> State[(".loki/data/&lt;project-id&gt;<br/>shared project state")]
    Processor <--> State
    Core <--> State
    State --> Pi
    State --> Outputs["Reports · exports · fix bundles"]

The unit of work is a source file represented by a durable FileRecord. Scanning adds candidates to it; processing adds findings and analysis history; triage, revalidation, enrichment, and verification add more evidence. The CLI, Pi UI, reports, and exports all read the same records—there is no separate database or web backend.

Use Loki in Pi

@adityamiskin/loki-pi adds a native interface to Pi:

/loki opens the custom dashboard
overview and finding views read the current Loki workspace
severity filters and full finding details make review interactive
scans show live progress
Loki tools get purpose-built terminal rendering

Try it from this checkout:

bun install
pi -e ./packages/loki-pi

Open Pi in a repository containing a .loki workspace, then run:

/loki

The extension currently exposes project status, findings, and deterministic scanning. Paid agent stages remain available through the CLI.

Use Loki from the CLI

From the root of the repository you want to inspect:

bunx @adityamiskin/loki init
cd .loki
bun install

init registers the surrounding repository and creates the Loki workspace. Before the first serious scan, have a coding agent read the generated SETUP.md and fill in INFO.md with concise, project-specific context.

Run the full primary pipeline:

bun run loki swarm --project-id <project-id> --root ..

Or run it stage by stage:

bun run loki plan    --project-id <project-id> --root ..
bun run loki scan    --project-id <project-id> --root ..
bun run loki process --project-id <project-id> --root ..
bun run loki reduce  --project-id <project-id>
bun run loki verify  --project-id <project-id>
bun run loki fix     --project-id <project-id>

Most planning, scanning, reduction, verification, and fix commands have free offline modes. process and any --agent mode use a coding-agent backend and can consume substantial tokens, so begin with a narrow scope.

See Getting Started for the full first-run workflow. For exact flags, run bun run loki --help or bun run loki <command> --help inside the workspace.

Packages

Package	Role
`@loki/core`	Schemas, configuration, durable records, and plugin contracts
`@loki/scanner`	Deterministic selectors, matchers, and candidate discovery
`@loki/processor`	Agent investigation, reduction, verification, and fix generation
`@adityamiskin/loki`	Public CLI, orchestration, and distributable workspace
`@adityamiskin/loki-pi`	Pi extension and custom terminal UI

The packages communicate through typed records from @loki/core; interface packages do not duplicate the scanning or processing engine.

Commands

Command	What it does
`init`	Create a self-contained `.loki/` workspace
`init-project`	Register another target repository
`plan`	Build a threat model and deterministic selectors
`scan`	Generate candidates with matchers; no AI required
`process`	Investigate candidates in parallel with coding agents
`reduce`	Deduplicate, connect, and prioritize findings
`verify`	Collect offline, static-agent, or runtime evidence
`fix`	Generate a remediation bundle
`swarm`	Run the primary pipeline end to end
`revalidate`	Revisit existing findings after deeper review or code changes
`triage`	Classify findings by priority
`enrich`	Add repository history and ownership context
`report`	Produce a project report
`export`	Export findings as JSON or Markdown
`metrics`	Summarize project and finding metrics
`status`	Show saved project state
`sandbox`	Run a command through an optional remote sandbox
`sandbox-all`	Run a distributed stage through optional remote sandboxes

Agents and execution

Loki supports Codex, Claude, and Pi as agent backends. Depending on the chosen backend, it can use a locally authenticated agent session or explicit provider credentials and base URLs.

Local execution is the default. For untrusted repositories or larger parallel runs, use isolation. Vercel Sandbox is one supported remote execution adapter, and Vercel AI Gateway is one optional provider route; neither is required by Loki's architecture.

Security model

Treat agent-backed Loki stages like a coding agent with shell access to their execution environment. Deterministic scanning is local code inspection, but agent and runtime-verification modes may read files, run tools, and execute project commands.

Use local execution for repositories you trust. Run untrusted, vendored, or prompt-injection-prone code in an isolated environment with limited credentials and network access.

Configuration and state

Loki discovers loki.config.ts from the .loki workspace. Configuration controls projects, selectors, matchers, agent backends, models, concurrency, budgets, verification policy, and optional plugins.

Project state lives in:

.loki/data/<project-id>/
├── project.json
├── INFO.md
├── THREAT_MODEL.md
├── selectors.json
├── files/                 # FileRecords: candidates, findings, evidence
├── runs/                  # resumable run metadata
├── reduce-report.json
├── verify-report.json
├── fix-bundle.md
└── reports/

Exact files appear as their stages run. See Data Layout and Configuration for the schemas and options.

Development

Requires Bun 1.3+ and Node.js 22+.

git clone https://github.com/adityamiskin/loki.git
cd loki
bun install
bun run validate

Useful commands:

bun run test
bun run typecheck:loki
bun run lint
bun run bundle

Documentation

Origin and license

Loki is an independently maintained evolution of DeepSec. Original attribution and modification notices are retained in NOTICE.

Licensed under Apache-2.0.