--- slug: "free-web-search-js-x-2" source_type: "clawhub" source_url: "https://clawhub.ai/skills/free-web-search-js" repo: "" source_file: "description" --- --- name: free-web-search-js description: Web search & content fetching via Playwright + HTTP — zero API keys. Searches Bing CN (domestic) or DDG (international), auto-fetches top page contents. For real-time search, fact-checking, news, tutorials, documentation lookup. version: 29.1.0 license: MIT author: free-web-search-js trigger_keywords: - search - find - look up - 搜索 - 查一下 - 找一下 tools: - name: search description: Search + auto-fetch; Bing CN via Playwright, DDG via HTTP script: scripts/search.js parameters: query: type: string description: "Search query" required: true max: type: integer description: "Max results 1-30, default 10" required: false region: type: string description: "Region: auto/cn/intl, defaults to auto (IP detection)" required: false - name: fetch description: Fetch page content by URL, HTTP first with headed fallback script: scripts/fetch.js parameters: urls: type: string description: "URLs to fetch, space-separated" required: true max-len: type: integer description: "Max chars per page, default 12000" required: false --- # free-web-search-js Pure search — no rewriting, no filtering, noise handled by the AI. Uses real browser automation (Playwright) for Bing CN anti-bot, with HTTP fallback for international queries. Zero API keys required. ## Quick start ```bash # Search (auto-fetches top 3 page contents) node scripts/search.js "silver price" node scripts/search.js "how to deploy docker" --max=5 node scripts/search.js "xxx" --region=cn node scripts/search.js "xxx" --fetch=5 # fetch top 5 node scripts/search.js "xxx" --no-fetch # search only # Fetch specific URLs node scripts/fetch.js "https://example.com/page1" "https://example.com/page2" ``` ## Architecture ``` Domestic (CN IP): Playwright headless → Bing CN → home page for cookie → search box submit → 0 results? fallback to DDG HTML → auto-fetch top N page contents International: Pure HTTP → DDG HTML parse → fetch fails? fallback to Playwright request → DDG empty? fallback to Bing → auto-fetch top N page contents ``` ## Install All dependencies are installed upfront by the setup script. **The skill never runs `npm install` or `pip install` at runtime.** This avoids audit flags and supply-chain risks. ### Prerequisites | Dependency | Purpose | Size | |---|---|---| | Node.js >= 18 | Runtime | — | | cheerio | HTML parsing | small | | commander | CLI arg parsing | small | | iconv-lite | GBK charset conversion | small | | playwright | Browser automation | ~50 MB | | Chromium (Playwright) | Browser engine for Bing search + headed fetch | ~150 MB | ### One-command setup ```bash # Linux / macOS bash scripts/setup.sh # Windows powershell -File scripts/setup.ps1 ``` The scripts auto-detect region and use `npmmirror.com` mirrors inside China for faster downloads. ### Manual install ```bash cd skills/free-web-search-js npm install npx playwright install chromium # ~150 MB # Verify node scripts/check-env.js ``` If you already have Chrome/Chromium installed: ```bash export CHROMIUM_EXECUTABLE_PATH="/path/to/chrome" node scripts/check-env.js ``` `check-env.js` will confirm whether the provided executable is usable. ### Environment variables | Variable | Purpose | |---|---| | `CHROMIUM_EXECUTABLE_PATH` | Path to system Chrome/Chromium, skips 150 MB download | | `PLAYWRIGHT_DOWNLOAD_HOST` | Mirror for Chromium download (auto-set by setup script) | | `BROWSER_DAEMON_IDLE_MS` | Daemon idle timeout in ms (default 600000 = 10 min) | ## Search engines | Engine | Method | Region | Notes | |---|---|---|---| | Bing CN | Playwright search box submit | Domestic | Visits home page first for cookie, then types into search box | | DDG | Playwright search box submit | International | Full browser automation | ### Strategy | Region | Search | Fetch | |---|---|---| | Domestic | Bing CN, fallback DDG | Auto-fetch top 3 | | International | DDG HTML, fallback Playwright request; DDG empty→Bing | Auto-fetch top 3 | ### IP detection On each search, three probes run in parallel — first responder wins: | Probe | Service | Logic | |---|---|---| | 1 | `myip.ipip.net` / `cip.cc` | Reachable from China | | 2 | `ipinfo.io` / `ipapi.co` | International | | 3 | `cn.bing.com` | Reachable → likely CN | | Fallback | — | Defaults to domestic | Override with `--region=cn` or `--region=intl` if your proxy causes misdetection. ## Fetch mode After search, auto-fetches the top N results (default 3). | Tier | Method | Speed | Notes | |---|---|---|---| | 1 | Lightweight HTTP + cheerio | Fast | No browser launch | | 2 | Playwright headed | Slower | Full JS rendering | Tier 1 enhancements: - JSON API response detection with structured content extraction - JSON-LD (`