--- slug: "llm-wiki-mduongvandinh" source_type: "readme" source_url: "https://cdn.jsdelivr.net/gh/mduongvandinh/llm-wiki@main/README.md" repo: "https://github.com/mduongvandinh/llm-wiki" source_file: "README.md" branch: "main" --- # LLM Wiki **A personal knowledge base that builds and maintains itself.** Drop articles, notes, PDFs, or Reddit threads into a folder. Your AI reads everything, builds a structured wiki with cross-references, detects contradictions between sources, and keeps it all up to date โ€” automatically. You just read and ask questions. > Inspired by [Andrej Karpathy's LLM Wiki pattern](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f). Extended with auto-discovery, Reddit scanning, GitHub tracking, and ready-made variants for common use cases. --- ## Why not just use RAG? | | Traditional RAG | LLM Wiki | |---|---|---| | Knowledge | Re-retrieved fresh every query | Compiled once, maintained continuously | | Cross-references | None | Built and maintained automatically | | Contradictions | Not detected | Flagged immediately on ingest | | Compounding | Flat | Grows richer over time | | Infrastructure | Vector DB + embeddings | Just folders + markdown | --- ## Quick Start ### Option A โ€” Start with a ready-made variant (recommended) Don't configure from scratch. Pick a variant that matches your use case โ€” config, schema, and demo data are pre-tuned: ```bash git clone https://github.com/mduongvandinh/llm-wiki.git my-wiki cd my-wiki # Install skill (Claude Code only โ€” other tools skip this) cp -r skills/llm-wiki ~/.claude/skills/llm-wiki # Pick your variant: /llm-wiki setup book-companion # ๐Ÿ“š Reading companion /llm-wiki setup competitive-intel # ๐ŸŽฏ Track competitors /llm-wiki setup job-search # ๐Ÿ’ผ Job search research ``` ### Option B โ€” Start from scratch ```bash git clone https://github.com/mduongvandinh/llm-wiki.git my-wiki cd my-wiki cp config.example.yaml config.yaml # Edit config.yaml: set your topics, keywords, feeds ``` Then drop content into `raw/articles/` and run your first cycle. --- ## Variants Pre-configured templates for specific use cases. One command setup, includes demo data. | | Variant | Use Case | Key Command | |---|---------|----------|-------------| | ๐Ÿ“š | [book-companion](https://github.com/mduongvandinh/llm-wiki/blob/HEAD/variants/book-companion/README.md) | Build a companion wiki as you read โ€” characters, timeline, factions, lore auto-extracted | `book-summary` ยท [demo](https://github.com/mduongvandinh/llm-wiki/tree/HEAD/demo/book-companion/) | | ๐ŸŽฏ | [competitive-intel](https://github.com/mduongvandinh/llm-wiki/blob/HEAD/variants/competitive-intel/README.md) | Track competitors, detect pricing/feature changes, generate battlecards | `competitive-brief "Linear"` | | ๐Ÿ’ผ | [job-search](https://github.com/mduongvandinh/llm-wiki/blob/HEAD/variants/job-search/README.md) | Research target companies, prep interview 1-pagers | `interview-prep "Stripe"` | โ†’ [See all variants โ†’](https://github.com/mduongvandinh/llm-wiki/blob/HEAD/variants/README.md) --- ## Using with Your AI Tool LLM Wiki works with any AI coding assistant. The wiki is just folders and markdown โ€” the AI reads `CLAUDE.md` (or `AGENTS.md`) to understand the schema and rules, then operates on files. ### Claude Code The primary supported tool. Has a slash command skill for the best experience. **Install the skill:** ```bash cp -r skills/llm-wiki ~/.claude/skills/llm-wiki ``` **Commands:** ```bash /llm-wiki setup book-companion # Set up a variant /llm-wiki run # Full cycle: discover โ†’ ingest โ†’ lint /llm-wiki ingest # Process new files in raw/ /llm-wiki query "What is X?" # Ask questions against the wiki /llm-wiki discover # Auto-find new sources from the web /llm-wiki lint # Health check: contradictions, orphans, gaps /llm-wiki status # Overview of wiki state /llm-wiki digest # Daily brief: new sources, insights, changes /llm-wiki book-summary # Structured book summary (book-companion variant) /llm-wiki competitive-brief "Name" # Battlecard for a competitor /llm-wiki interview-prep "Company" # Interview prep 1-pager ``` **Auto-run:** ```bash /loop 1h /llm-wiki run # Runs every hour in current session /loop 2h /llm-wiki run # Every 2 hours ``` --- ### Cursor Cursor reads `AGENTS.md` from the project root automatically. No installation needed. **Open the project in Cursor, then chat:** ``` Follow AGENTS.md. Run full cycle: discover new sources, ingest into wiki, then lint. ``` ``` Follow AGENTS.md. Ingest everything new in raw/ and update the wiki. ``` ``` Follow AGENTS.md. Query the wiki: what do we know about [topic]? ``` ``` Follow AGENTS.md. Run competitive-brief for "Linear" and save to outputs/. ``` --- ### GitHub Copilot (VS Code) Copilot reads `.github/copilot-instructions.md` automatically when you open the project. No installation needed. **Open project in VS Code, then use Copilot Chat:** ``` Read CLAUDE.md then run discover, ingest, and lint for this wiki. ``` ``` Based on the wiki schema in CLAUDE.md, ingest the new files in raw/articles/. ``` ``` Query the wiki: summarize everything we know about [topic]. ``` > **Note:** Copilot Chat doesn't support background loops. Run manually or use system scheduler. --- ### Windsurf (Codeium) Windsurf reads `AGENTS.md` or `.windsurfrules` from the project root. **Chat in Windsurf:** ``` Read AGENTS.md. Run the full llm-wiki cycle: discover โ†’ ingest โ†’ lint. ``` ``` Read AGENTS.md. I've added new files to raw/articles/. Please ingest them into the wiki. ``` --- ### Gemini CLI Gemini CLI reads `GEMINI.md` from the project root (or falls back to `AGENTS.md`). **Create `GEMINI.md`** (copy from `AGENTS.md`): ```bash cp AGENTS.md GEMINI.md ``` **Then run:** ```bash gemini "Read GEMINI.md and run full cycle: discover, ingest, lint" gemini "Read GEMINI.md and query the wiki: what do we know about [topic]?" ``` --- ### Codex CLI (OpenAI) Codex reads `AGENTS.md` from the project root. ```bash codex "Read AGENTS.md and run full cycle: discover โ†’ ingest โ†’ lint" codex "Read AGENTS.md and ingest new files in raw/" codex "Read AGENTS.md and query: summarize the wiki on [topic]" ``` --- ### Aider Aider works best with explicit file references. ```bash aider --message "Read CLAUDE.md to understand the schema. Run ingest on all new files in raw/articles/. Update wiki/ accordingly." CLAUDE.md raw/articles/*.md ``` --- ### Continue.dev Add `CLAUDE.md` to Continue's context, then chat: ``` @CLAUDE.md Ingest the new articles in raw/ and update the wiki with entities, concepts, and cross-references. ``` --- ### Tool Comparison | Tool | Config File | Skill/Slash Command | Auto-loop | Best For | |------|------------|-------------------|-----------|----------| | Claude Code | `CLAUDE.md` | โœ… `/llm-wiki` | โœ… `/loop` | Full experience, all features | | Cursor | `AGENTS.md` | โŒ chat only | โŒ | IDE-integrated workflow | | GitHub Copilot | `.github/copilot-instructions.md` | โŒ chat only | โŒ | VS Code users | | Windsurf | `AGENTS.md` | โŒ chat only | โŒ | VS Code alternative | | Gemini CLI | `GEMINI.md` | โŒ chat only | via cron | Google ecosystem | | Codex CLI | `AGENTS.md` | โŒ chat only | via cron | OpenAI preference | | Aider | `CLAUDE.md` | โŒ explicit | via cron | Terminal-only | > All tools use the same `raw/`, `wiki/`, `outputs/` folder structure. The AI config file tells each tool how to operate โ€” same content, different filenames. --- ## Auto-Run Without Claude Code For tools without `/loop`, use your system scheduler. **Windows โ€” Task Scheduler** (run once as Admin): ```powershell powershell -ExecutionPolicy Bypass -File "scripts/setup-scheduler.ps1" ``` **macOS / Linux โ€” crontab:** ```bash # Claude Code (headless) 0 */2 * * * cd /path/to/my-wiki && claude --print "/llm-wiki run" >> outputs/auto-run.log 2>&1 # Codex 0 */2 * * * cd /path/to/my-wiki && codex "Read AGENTS.md and run full cycle" >> outputs/auto-run.log 2>&1 # Gemini CLI 0 */2 * * * cd /path/to/my-wiki && gemini "Read GEMINI.md and run full cycle" >> outputs/auto-run.log 2>&1 ``` --- ## How It Works ``` โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ YOU (read & ask) โ”‚ โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค โ”‚ wiki/ โ”‚ outputs/ โ”‚ wiki-viewer.html โ”‚ โ”‚ (Obsidian) โ”‚ (reports) โ”‚ (browser view) โ”‚ โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค โ”‚ LLM (writes & maintains) โ”‚ โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค โ”‚ discover โ”‚ ingest โ”‚ query โ”‚ lint โ”‚ โ”‚ (find) โ”‚ (process)โ”‚ (answer) โ”‚ (health check) โ”‚ โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค โ”‚ raw/ โ”‚ config.yaml โ”‚ CLAUDE.md โ”‚ โ”‚ (sources) โ”‚ (settings) โ”‚ (schema/rules) โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ ``` **Main flow:** `discover` โ†’ `raw/` โ†’ `ingest` โ†’ `wiki/` โ†’ `query` โ†’ `outputs/` **Self-improvement loop:** `lint` finds gaps โ†’ `discover` finds sources โ†’ `ingest` fills them โ†’ wiki grows richer ### Core Workflows **`discover`** โ€” Searches the web for new sources based on your topics in `config.yaml`. Can scan Reddit for pain points, track GitHub trending repos, follow RSS feeds, and snowball from references found in existing sources. **`ingest`** โ€” Reads each new file in `raw/`. Extracts entities, concepts, and key claims. Creates wiki pages. Adds cross-references. Flags contradictions with existing content. One source typically creates 5โ€“15 wiki pages. **`query`** โ€” Reads `wiki/INDEX.md`, finds relevant pages, synthesizes an answer with citations. Valuable answers are automatically saved as synthesis pages. **`lint`** โ€” Scans the entire wiki for: contradictions, orphan pages, broken links, stale claims, and knowledge gaps. Updates `.discoveries/gaps.json` so `discover` knows what to find next. --- ## Adding Sources ### Drop a file manually Create a markdown file in `raw/articles/` with frontmatter, then run `ingest`: ```markdown --- title: "Article Title" url: "https://example.com/article" author: "Author Name" discovered: 2026-04-14 topic: "your topic" --- Article content here... ``` For **book notes** (book-companion variant), add `chapter: N`: ```markdown --- title: "Dune Chapter 5 Notes" chapter: 5 book: "Dune" --- ``` For **competitor data** (competitive-intel variant), add `competitor: "Name"`: ```markdown --- title: "Linear โ€” Pricing Page" competitor: "Linear" --- ``` ### Obsidian Web Clipper 1. Install [Obsidian Web Clipper](https://obsidian.md/clipper) browser extension 2. See an interesting article โ†’ click clip โ†’ save to `raw/articles/` 3. Run ingest ### Auto-discovery Configure `config.yaml` with topics, Reddit subreddits, GitHub orgs, and RSS feeds โ€” the `discover` command handles the rest. --- ## Viewing Your Wiki **Obsidian** (recommended): `File โ†’ Open vault โ†’ select your wiki folder` โ†’ Graph View shows connections **Browser**: Open `wiki-viewer.html` โ€” mobile-first dark theme dashboard with graph view **Any markdown editor**: All wiki pages are plain `.md` files --- ## Folder Structure ``` my-wiki/ โ”œโ”€โ”€ CLAUDE.md โ† Schema + rules for Claude Code โ”œโ”€โ”€ AGENTS.md โ† Schema + rules for Cursor, Codex, Windsurf, Aider โ”œโ”€โ”€ GEMINI.md โ† Schema + rules for Gemini CLI (copy of AGENTS.md) โ”œโ”€โ”€ .github/ โ”‚ โ””โ”€โ”€ copilot-instructions.md โ† Schema for GitHub Copilot โ”œโ”€โ”€ config.yaml โ† Your topics, feeds, schedule โ”œโ”€โ”€ config.example.yaml โ† Template to start from โ”œโ”€โ”€ wiki-viewer.html โ† Browser-based wiki viewer โ”‚ โ”œโ”€โ”€ variants/ โ† Ready-made use case templates โ”‚ โ”œโ”€โ”€ book-companion/ โ† ๐Ÿ“š Reading companion โ”‚ โ”œโ”€โ”€ competitive-intel/ โ† ๐ŸŽฏ Competitor tracking โ”‚ โ””โ”€โ”€ job-search/ โ† ๐Ÿ’ผ Job search research โ”‚ โ”œโ”€โ”€ raw/ โ† YOUR SOURCES (LLM never modifies these) โ”‚ โ”œโ”€โ”€ articles/ โ† Web articles, blog posts, papers โ”‚ โ”œโ”€โ”€ reddit/ โ† Reddit threads, pain points โ”‚ โ”œโ”€โ”€ notes/ โ† Personal notes โ”‚ โ””โ”€โ”€ media/ โ† Screenshots, diagrams โ”‚ โ”œโ”€โ”€ wiki/ โ† LLM-maintained wiki (don't edit manually) โ”‚ โ”œโ”€โ”€ INDEX.md โ† Master catalog of all wiki pages โ”‚ โ† LOG.md โ† Activity log โ”‚ โ”œโ”€โ”€ entities/ โ† People, orgs, tools, projects โ”‚ โ”œโ”€โ”€ concepts/ โ† Ideas, patterns, methodologies โ”‚ โ”œโ”€โ”€ sources/ โ† Summary of each raw source โ”‚ โ””โ”€โ”€ syntheses/ โ† Analysis, comparisons, rankings โ”‚ โ”œโ”€โ”€ outputs/ โ† Query results, reports, battlecards, briefs โ”œโ”€โ”€ .discoveries/ โ† Auto-discovery metadata โ”‚ โ”œโ”€โ”€ feeds.json โ† Sources being tracked โ”‚ โ”œโ”€โ”€ gaps.json โ† Knowledge gaps to fill โ”‚ โ””โ”€โ”€ history.json โ† Processed sources (dedup) โ”‚ โ””โ”€โ”€ skills/llm-wiki/SKILL.md โ† Claude Code skill definition ``` --- ## Configuration All behavior is controlled by `config.yaml`. Copy `config.example.yaml` to get started. Key sections: ```yaml wiki: name: "My Wiki" language: "en" # Any language โ€” wiki content will be in this language topics: - name: "Topic Name" keywords: ["keyword1", "keyword2"] priority: high feeds: reddit: enabled: true subreddits_pain_points: - sub: "r/SaaS" hackernews: enabled: true min_score: 50 github_trending: enabled: true languages: ["python", "typescript"] schedule: run: loop_interval: "1h" # For /loop in Claude Code ``` โ†’ See `config.example.yaml` for the full reference with all options. --- ## Commands Reference | Command | Description | |---------|-------------| | `setup ` | Initialize wiki from a variant template (book-companion, competitive-intel, job-search) | | `init "Topic"` | Add a new topic to an existing wiki | | `ingest` | Process all new files in `raw/` | | `query "..."` | Answer a question using the wiki; saves valuable answers | | `discover` | Auto-find new sources from the web based on topics | | `run` | Full cycle: discover โ†’ ingest โ†’ lint | | `lint` | Health check: contradictions, orphans, gaps, stale content | | `status` | Show wiki stats and health overview | | `digest` | Daily brief: new sources, top insights, detected changes | | `pain-rank` | Rank pain points by business opportunity (Reddit data) | | `book-summary` | Structured book summary: cast, timeline, themes, mysteries | | `competitive-brief "Name"` | Battlecard 1-pager for a specific competitor | | `interview-prep "Company"` | Interview prep 1-pager for a target company | For tools without slash commands, use the equivalent natural language prompt: ``` Read CLAUDE.md (or AGENTS.md). Then [describe what you want]. ``` --- ## FAQ **What AI model do I need?** Any capable model with tool use (web search, file read/write). Claude Sonnet/Opus recommended. Works with GPT-4o, Gemini 1.5 Pro, and others via their respective CLI tools. **How much does it cost to run?** A typical `/llm-wiki run` cycle (discover + ingest 5 sources + lint) costs $0.10โ€“0.40 depending on wiki size and source length. Running every 2 hours = ~$50/month. **My wiki is getting large. How do I query it efficiently?** The skill reads `wiki/INDEX.md` first to find relevant pages, then reads only those. Even large wikis (500+ pages) query efficiently because you never load everything at once. **Can I use this for a team?** Yes โ€” put the wiki folder in a shared Git repo. Each team member runs ingest on their own sources; the wiki is a shared output. Conflicts are resolved by `lint`. **Does it hallucinate?** The query and ingest commands only use information from your `raw/` sources. They cannot add claims not supported by your sources. If something is uncertain, the wiki page will say so. **Can I write in a language other than English?** Yes. Set `wiki.language` in `config.yaml`. The LLM will write all wiki content in that language while keeping file names and frontmatter in English. --- ## Credits - Original pattern: [Andrej Karpathy โ€” LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) - Inspiration: [Nick Spisak โ€” How to Build Your Second Brain](https://x.com/NickSpisak_/status/2040448463540830705) - Built for: [Claude Code](https://claude.ai/code) (Anthropic) + compatible with all major AI coding assistants - Wiki viewer: [Obsidian](https://obsidian.md/) ## License MIT โ€” use it, fork it, build variants, share back.