---
slug: "j6e-pi-md-web-surfer"
source_type: "readme"
source_url: "https://cdn.jsdelivr.net/gh/j6e/pi-md-web-surfer@main/README.md"
repo: "https://github.com/j6e/pi-md-web-surfer"
source_file: "README.md"
branch: "main"
---
# pi-md-web-surfer

A [pi](https://github.com/nichochar/pi-coding-agent) extension that adds `fetch_markdown` and `web_search` tools — fetch web pages as clean markdown and search the web.

Inspired by the [skill](https://github.com/ruliana/pi-fetch-markdown) of the same name.

## Tools

### `fetch_markdown`

Fetch any web page as clean, token-efficient markdown.

Three strategies, tried in order:

1. **Content Negotiation** — Sends `Accept: text/markdown` header. Sites using Cloudflare's [Markdown for Agents](https://blog.cloudflare.com/markdown-for-agents/) return markdown directly (~80% fewer tokens than HTML).
2. **Direct text/plain** — For raw `.md` files, `text/plain` responses, etc. The body is already clean text, so we use it directly.
3. **Jina Reader** (fallback) — For sites that return HTML, routes through `r.jina.ai` which renders the page and extracts clean markdown.

**Usage:**
```
fetch_markdown(url, info?)
```

### `web_search`

Search the web and return results as clean markdown via Jina Search.

**Requires a Jina API key.** Get one free at [jina.ai/?newKey](https://jina.ai/?newKey), then configure it with `/md-web-surfer-config` or the `JINA_API_KEY` environment variable.

**Usage:**
```
web_search(query, num?, info?, retainImages?, withLinksSummary?)
```

- `query` — Search query
- `num` — Number of results (1–20, default 5)
- `info` — Include metadata (method, tokens)
- `retainImages` — Image retention mode: `"none"` (default), `"all"`, or `"alt_text"` (keep alt text with auto-generation)
- `withLinksSummary` — Include a summary section for hyperlinks (default: false)

## Configuration

### `/md-web-surfer-config`

Interactive command to set your Jina API key. Writes to `~/.config/pi-md-web-surfer/config.json` (chmod 0600).

```
/md-web-surfer-config        # prompt for key
/md-web-surfer-config --show # show current key status (masked)
```

### API key resolution

First match wins:

1. `JINA_API_KEY` environment variable
2. `jinaApiKey` field in `~/.config/pi-md-web-surfer/config.json`

## Installation

### Quick test (one session)

```bash
pi -e ./src/index.ts
```

### Persistent (auto-discovered)

```bash
ln -s $(pwd) ~/.pi/agent/extensions/pi-md-web-surfer
```

Then run `pi` normally. The tools appear alongside `read`, `write`, `edit`, etc.

### As a pi package

```bash
pi install npm:@j6e/pi-md-web-surfer
```

## Development

```bash
npm install
npm test           # run tests once
npm run test:watch # TDD mode
npm run check      # typecheck
```

Smoke test against real URLs:

```bash
node --experimental-strip-types scripts/smoke-test.ts
```

## Requirements

- Node.js 22+ (for `--experimental-strip-types`)
- pi-coding-agent 0.74+

## License

MIT