---
slug: "henyo-pi-web"
source_type: "readme"
source_url: "https://cdn.jsdelivr.net/gh/henyojess/henyo-pi-web@master/README.md"
repo: "https://github.com/henyojess/henyo-pi-web"
source_file: "README.md"
branch: "master"
---
# henyo-pi-web

Web search and content extraction tools for [Pi](https://github.com/earendil-works/pi).

**Henyo** means "genius" in Filipino — because Pi is sharp, and so are you.

## Install

**Ask your Pi agent:**

```
Install the henyo-pi-web npm package
```

Or run directly:

```bash
pi install npm:henyo-pi-web
```

> `pi install` automatically resolves npm dependencies (`defuddle`, `jsdom`) and registers two native tools: `henyo_search` and `henyo_fetch`.

> **Breaking change (v2.0):** Config keys changed from `web-search`/`web-fetch` to `henyo-search`/`henyo-fetch`. Update your `~/.pi/settings.json` accordingly. Cache directory changed from `.pi/tools-cache/web_search`/`web_fetch` to `.pi/tools-cache/henyo_search`/`henyo_fetch` — old cache files will be ignored.

## Tools

### `henyo_search`

Search the web using DuckDuckGo, Stack Overflow, npm, GitHub, Wikipedia, or a custom SearXNG instance. Jina is available via config but requires an API key. Context-aware routing (coding vs general), BM25 ranking, domain diversification, and provider-level result counts. Results cached 30 min.

**Parameters:**

- `query` (string) — Search query
- `max` (integer, default 10) — Max results (1–50)
- `context` (string, default "auto") — `"coding"`, `"general"`, or `"auto"`
- `noCache` (boolean, default false) — Skip cache

**Features:**

- Auto-detects coding vs general queries using multi-token pattern matching
- Runs providers sequentially by priority group, deduplicates results
- Applies corpus-level BM25 ranking within each priority group
- Diversifies results by domain (default 2 per domain)
- Reports per-provider status (ok/error/timeout) with result counts
- Supports partial results on abort

**TUI Rendering:**

- Collapsed/expanded views — header shows context, count, and per-provider breakdown (`duckduckgo:5 stackoverflow:3`)
- Collapse/expand hints with keybindings
- Partial results show "Processing…" while fetching
- Post-slice result count (accurate after dedup/rank/diversify/max)

### `henyo_fetch`

Extract clean readable content from any URL. Uses Defuddle-first extraction with Jina quality-check fallback. Handles Cloudflare protection, SPAs, GitHub raw files, JSON, plain text, and binary content detection (PDF, images, archives). Includes SSRF protection. Cached 1 hour.

**Parameters:**

- `url` (string) — URL to fetch
- `timeout` (integer, default 15000) — Timeout in ms (1000–60000)
- `noCache` (boolean, default false) — Skip cache
- `headers` (object, optional) — Custom HTTP headers, e.g. `{ "Authorization": "Bearer token" }`

**Features:**

- Content-type aware: handles HTML, JSON, plain text, and binary content
- Smart truncation with configurable heading/content thresholds
- Oversized content returns metadata only (URL, title, source, cache path)
- Politeness delay between requests (configurable min/max)
- Retry with exponential backoff
- `cached` flag on cached results
- Error categories in `details` (`errorCategory`: ssrf, invalid-url, timeout, not-found, forbidden, server-error, network, unknown)

**TUI Features:**

- **Source badges** — color-coded `[defuddle]`, `[jina]`, `[github]`, etc.
- **Size labels** — human-readable sizes (`12.3 KB`, `1.45 MB`)
- **Status indicators** — `[cached]`, `[truncated]`, `[oversized]` badges
- **Error cards** — categorized errors with actionable messages
- **Oversized content card** — structured metadata with guidance (reduce threshold, check cache, fresh fetch)
- **Collapsible content** — press expand key to view full content, collapse to return to header

## Configuration

Optional settings go in `~/.pi/settings.json`:

```json
{
  "henyo-fetch": {
    "jinaEnabled": true,
    "min-delay": 1000,
    "max-delay": 3000,
    "cache-max-files": 100,
    "heading-threshold": 40000,
    "content-threshold": 32000,
    "jina-timeout": 30000,
    "max-response-size": 10485760
  },
  "henyo-search": {
    "default-context": "general",
    "contexts": {
      "coding": {
        "duckduckgo": { "priority": 1 },
        "stackoverflow": { "priority": 1 },
        "npm": { "priority": 1 },
        "github": { "priority": 1 },
        "ranking": true
      },
      "general": {
        "duckduckgo": { "priority": 1 },
        "wikipedia": { "priority": 1 },
        "ranking": true
      }
    }
  }
}
```

**henyo-search config options:**

| Option | Type | Description |
|--------|------|-------------|
| `default-context` | string | Default context when `context="auto"` (default: `"general"`) |
| `contexts` | object | Per-context provider chains with priorities |
| `contexts.<name>.ranking` | boolean | Enable BM25 ranking per context (default: `true`) |
| `contexts.<name>.searxng` | object | Custom SearXNG instance — requires `url` field |
| `contexts.<name>.searxng.priority: 0` | number | SearXNG-only mode (replaces default chain) |
| `api-key` | string | StackOverflow API key (optional) |
| `rate-limit-cooldowns` | object | Per-provider cooldown in seconds |
| `max-per-domain` | number | Max results per domain |

> **Note:** Jina is available as a provider but requires an API key. Add it manually via `contexts.general.jina` config if you have a Jina API key.

**henyo-fetch config options:**

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `jinaEnabled` | boolean | `true` | Enable Jina Reader fallback |
| `min-delay` / `max-delay` | number | 1000 / 3000 | Politeness delay range (ms) |
| `cache-max-files` | number | 100 | Max cached files per directory |
| `heading-threshold` | number | 40000 | Heading size for smart truncation |
| `content-threshold` | number | 32000 | Max content size; oversize returns metadata only |
| `jina-timeout` | number | 30000 | Jina fallback timeout (ms) |
| `max-response-size` | number | 10485760 | Max response body size (bytes) |

## Requirements

- Node.js (ESM modules)
- Internet access

## License

MIT