browser-pilot

内容来源：README.md（说明文档） · 原始地址 · 查看安装指南

原始内容

# browser-pilot

Automation-first browser control over Chrome DevTools Protocol for AI agents. Version 0.1.0
supports Node.js, Bun, and Cloudflare Workers with zero production dependencies.

> Companion package: [Flightplan](https://github.com/svilupp/flightplan) provides simple,
> reusable, cheap automation on top of browser-pilot. It will be released immediately after
> browser-pilot. Start with `bunx flightplan --help` for the higher-level interface.

## Install

```bash
bun add browser-pilot
# or
npm install browser-pilot
```

## CLI quick start

On Chrome 144+, enable remote debugging at `chrome://inspect/#remote-debugging`, then connect:

```bash
bp connect --name dev
bp exec -s dev '{"action":"goto","url":"https://example.com"}'
bp snapshot -i -s dev
bp exec -s dev '[{"action":"click","selector":"ref:e4"}]'
```

Use `bp --help` to route a task by job and `bp --version` to check the CLI build.

| Job | Commands |
| --- | --- |
| Inspect page state | `snapshot`, `page`, `forms`, `text`, `targets`, `diagnose` |
| Review structured state | `review`, `delta` |
| Act in the browser | `exec`, `run` |
| Capture a human workflow | `record` |
| Analyze behavior over time | `trace` |
| Exercise voice and media | `audio` |
| Change browser conditions | `env` |

For multiple local Chrome profiles, use `--channel` or `--user-data-dir`. Use
`bp connect --new-tab --page-url <url>` to start from a fresh tab. New tabs stay in the background
unless `--foreground` is passed.

## Known issue

Native `select` can emit a synthetic untrusted `change` event before the final event. Apps that
reject untrusted events may observe an intermediate rejected event; the final selection is correct.

## Library quick start

Hosted browser:

```typescript
import { connect } from 'browser-pilot';

const browser = await connect({
  provider: 'browser-use',
  apiKey: process.env.BROWSER_USE_API_KEY,
});

const page = await browser.page();
await page.batch([
  { action: 'goto', url: 'https://example.com/login' },
  { action: 'fill', selector: ['#email', 'input[type=email]'], value: 'user@example.com' },
  { action: 'submit', selector: 'form' },
  { action: 'assertUrl', expect: '/dashboard' },
]);

await browser.close();
```

Local Chrome from TypeScript:

```typescript
import { connect, getBrowserWebSocketUrl } from 'browser-pilot';

const browser = await connect({
  provider: 'generic',
  wsUrl: await getBrowserWebSocketUrl(),
});
```

## The 0.1.0 interface

- **Target-safe control:** each page keeps its own CDP session. Target selection, popups, workers,
  and cross-origin iframes are explicit and isolated.
- **Inspect before acting:** snapshots produce reusable `ref:` selectors. `page`, `forms`, `text`,
  `review`, `delta`, `resolveAll`, and `diagnose` expose progressively richer reads.
- **Verified actions:** every action can use readiness waits, assertions, outcome conditions,
  failure hints, bounded retries, and target provenance.
- **Reusable evidence:** `record` and `trace` share one version 2 artifact model. `audio` and
  `env` cover voice pipelines and browser-state failure modes.

## Record and replay

Capture a human workflow with `bp record`:

```bash
bp connect --name demo
bp record -s demo --profile automation -f ./artifacts/demo.recording.json
# perform the flow manually, then stop with Ctrl+C
bp record summary ./artifacts/demo.recording.json
bp record inspect ./artifacts/demo.recording.json
bp record derive ./artifacts/demo.recording.json -o ./artifacts/demo.workflow.json
bp run ./artifacts/demo.workflow.json -s demo
```

`record` captures an existing named session. Start with `record summary` or `record inspect`, not
the raw artifact. `record derive` produces browser-pilot workflow JSON for `bp run`; Flightplan is
the companion interface for simple reusable automation.

For proof while replaying known steps:

```bash
bp connect --name validation --record
bp exec -s validation -f ./artifacts/demo.workflow.json
```

## Iframes and tabs

The action interface uses `switchFrame` and `switchToMain`:

```json
[
  {"action":"switchFrame","selector":"iframe#payment"},
  {"action":"fill","selector":"#card-number","value":"4242424242424242"},
  {"action":"switchToMain"}
]
```

The library method is `page.switchToFrame(selector)`. Cross-origin frames support `click`, `fill`,
`type`, `focus`, `press`, `shortcut`, `text`, `waitFor`, and `evaluate`. Unsupported actions fail
with a clear error. Cross-origin support requires Chrome site isolation (`--site-per-process`).

## Providers

Use [Browser Use](https://browser-use.com) when local Chrome is unavailable:

```bash
BROWSER_USE_API_KEY=bu_... bp connect --provider browser-use
```

See [Providers](https://github.com/svilupp/browser-pilot/blob/HEAD/docs/providers.md) for BrowserBase, Browserless, local Chrome, and direct CDP
connections.

## Documentation

- [Getting Started](https://github.com/svilupp/browser-pilot/blob/HEAD/docs/getting-started.md)
- [CLI Guide](https://github.com/svilupp/browser-pilot/blob/HEAD/docs/cli.md)
- [Automation workflows](https://github.com/svilupp/browser-pilot/blob/HEAD/docs/guides/automation-workflows.md)
- [Action recording](https://github.com/svilupp/browser-pilot/blob/HEAD/docs/guides/action-recording.md)
- [Trace workflows](https://github.com/svilupp/browser-pilot/blob/HEAD/docs/guides/trace-workflows.md)
- [Page API](https://github.com/svilupp/browser-pilot/blob/HEAD/docs/api/page.md)
- [Browser API](https://github.com/svilupp/browser-pilot/blob/HEAD/docs/api/browser.md)
- [Types](https://github.com/svilupp/browser-pilot/blob/HEAD/docs/api/types.md)
- [LLM contract](https://github.com/svilupp/browser-pilot/blob/HEAD/docs/llms.txt)

browser-pilot

Automation-first browser control over Chrome DevTools Protocol for AI agents. Version 0.1.0 supports Node.js, Bun, and Cloudflare Workers with zero production dependencies.

Companion package: Flightplan provides simple, reusable, cheap automation on top of browser-pilot. It will be released immediately after browser-pilot. Start with bunx flightplan --help for the higher-level interface.

Install

bun add browser-pilot
# or
npm install browser-pilot

CLI quick start

On Chrome 144+, enable remote debugging at chrome://inspect/#remote-debugging, then connect:

bp connect --name dev
bp exec -s dev '{"action":"goto","url":"https://example.com"}'
bp snapshot -i -s dev
bp exec -s dev '[{"action":"click","selector":"ref:e4"}]'

Use bp --help to route a task by job and bp --version to check the CLI build.

Job	Commands
Inspect page state	`snapshot`, `page`, `forms`, `text`, `targets`, `diagnose`
Review structured state	`review`, `delta`
Act in the browser	`exec`, `run`
Capture a human workflow	`record`
Analyze behavior over time	`trace`
Exercise voice and media	`audio`
Change browser conditions	`env`

For multiple local Chrome profiles, use --channel or --user-data-dir. Use bp connect --new-tab --page-url <url> to start from a fresh tab. New tabs stay in the background unless --foreground is passed.

Known issue

Native select can emit a synthetic untrusted change event before the final event. Apps that reject untrusted events may observe an intermediate rejected event; the final selection is correct.

Library quick start

Hosted browser:

import { connect } from 'browser-pilot';

const browser = await connect({
  provider: 'browser-use',
  apiKey: process.env.BROWSER_USE_API_KEY,
});

const page = await browser.page();
await page.batch([
  { action: 'goto', url: 'https://example.com/login' },
  { action: 'fill', selector: ['#email', 'input[type=email]'], value: 'user@example.com' },
  { action: 'submit', selector: 'form' },
  { action: 'assertUrl', expect: '/dashboard' },
]);

await browser.close();

Local Chrome from TypeScript:

import { connect, getBrowserWebSocketUrl } from 'browser-pilot';

const browser = await connect({
  provider: 'generic',
  wsUrl: await getBrowserWebSocketUrl(),
});

The 0.1.0 interface

Target-safe control: each page keeps its own CDP session. Target selection, popups, workers, and cross-origin iframes are explicit and isolated.
Inspect before acting: snapshots produce reusable ref: selectors. page, forms, text, review, delta, resolveAll, and diagnose expose progressively richer reads.
Verified actions: every action can use readiness waits, assertions, outcome conditions, failure hints, bounded retries, and target provenance.
Reusable evidence: record and trace share one version 2 artifact model. audio and env cover voice pipelines and browser-state failure modes.

Record and replay

Capture a human workflow with bp record:

bp connect --name demo
bp record -s demo --profile automation -f ./artifacts/demo.recording.json
# perform the flow manually, then stop with Ctrl+C
bp record summary ./artifacts/demo.recording.json
bp record inspect ./artifacts/demo.recording.json
bp record derive ./artifacts/demo.recording.json -o ./artifacts/demo.workflow.json
bp run ./artifacts/demo.workflow.json -s demo

record captures an existing named session. Start with record summary or record inspect, not the raw artifact. record derive produces browser-pilot workflow JSON for bp run; Flightplan is the companion interface for simple reusable automation.

For proof while replaying known steps:

bp connect --name validation --record
bp exec -s validation -f ./artifacts/demo.workflow.json

Iframes and tabs

The action interface uses switchFrame and switchToMain:

[
  {"action":"switchFrame","selector":"iframe#payment"},
  {"action":"fill","selector":"#card-number","value":"4242424242424242"},
  {"action":"switchToMain"}
]

The library method is page.switchToFrame(selector). Cross-origin frames support click, fill, type, focus, press, shortcut, text, waitFor, and evaluate. Unsupported actions fail with a clear error. Cross-origin support requires Chrome site isolation (--site-per-process).

Providers

Use Browser Use when local Chrome is unavailable:

BROWSER_USE_API_KEY=bu_... bp connect --provider browser-use

See Providers for BrowserBase, Browserless, local Chrome, and direct CDP connections.