🇨🇳 中文

Pi Web Suite

Web search, URL fetching, PDF extraction, and content extraction tools for the Pi coding agent.

A Pi extension that brings web_search, fetch_content, and get_search_content tools with cascading search providers and smart content extraction. Based on @juicesharp/rpiv-web-tools.

Install

pi install npm:pi-web-suite

Once installed and pi restarted, you get:

• web_search — search the web via cascading providers
• fetch_content — fetch URLs, GitHub repos, PDFs, and extract content
• get_search_content — retrieve past search results by ID
• /web-search-config — configure API keys interactively

Features

• Zero-config search — Exa MCP endpoint works without any API key
• Smart cascading — Exa → AnySearch → Tavily, automatic provider selection
• Content extraction — Readability + linkedom local extraction, Jina / defuddle proxy fallback
• GitHub repos — API-based file and directory access
• PDF extraction — Text extraction via unpdf
• SSRF protection — DNS resolution + IP range checks blocking private network requests

Quick Start

// Search the web
web_search({ query: "TypeScript best practices 2025" })

// Fetch a page
fetch_content({ url: "https://docs.example.com/guide" })

// Fetch a GitHub repo
fetch_content({ url: "https://github.com/owner/repo" })

// Fetch a PDF
fetch_content({ url: "https://example.com/doc.pdf" })

// Retrieve past search results
get_search_content({ searchId: "sr_..." })

Tools

web_search

Search the web. Automatically selects the first available provider (Exa → AnySearch → Tavily).

Parameter	Type	Required	Description
query	string	yes	Search query
numResults	number	no	Results per query (default 5, max 20)
recencyFilter	string	no	day, week, month, year

Returns search results with titles, URLs, snippets, and an optional answer. Each call returns a searchId for later retrieval via get_search_content.

fetch_content

Fetch a URL and return its content as readable Markdown. Auto-detects and handles GitHub repos, PDFs, and regular web pages.

Parameter	Type	Required	Description
url	string	yes	Supports http/https, GitHub links, PDF links

Fetch chain:

• GitHub URL → GitHub API (file contents / directory tree)
• PDF → unpdf text extraction
• HTML → Readability + linkedom local extraction → Jina Reader proxy → defuddle proxy

get_search_content

Retrieve stored content from a previous web_search call.

Parameter	Type	Required	Description
searchId	string	yes	ID from web_search response details

Configuration

Run /web-search-config to configure API keys interactively, or edit ~/.pi/web-search.json directly:

{
  "exaApiKey": "exa-...",
  "anySearchApiKey": "...",
  "tavilyApiKey": "tvly-..."
}

Environment variables take precedence over config: EXA_API_KEY, ANYSEARCH_API_KEY, TAVILY_API_KEY.

Provider notes:

• Exa: Direct API with key, MCP endpoint (zero-config) without
• AnySearch: Bearer auth with key, anonymous without (1000 req/day)
• Tavily: Requires API key

Project Structure

pi-web-suite/
├── index.ts      # Main entry (3 tools, 1 command)
├── types.ts      # Shared types
├── search.ts     # Search cascade (Exa / AnySearch / Tavily)
├── fetch.ts      # URL fetch router (GitHub / PDF / HTML)
├── extract.ts    # HTML content extraction (Readability → Jina → defuddle)
├── config.ts     # API key management
├── ssrf.ts       # SSRF protection
├── storage.ts    # Search result cache
├── test/         # Tests
├── README.md     # This file (English)
└── README-zh.md  # Chinese translation

License

MIT — based on @juicesharp/rpiv-web-tools (MIT).