mapreduce-skill (Codex skills)

This repository contains a paired set of skills for OpenAI Codex CLI:

• reduce-orchestrator: a MapReduce (+ Verify) orchestrator that coordinates many workers and synthesizes results.
• map-worker: the worker contract used for all Map and Verify tasks (writes narrative reports to report_path).

These skills are designed to run as a pair. reduce-orchestrator depends on map-worker and will not work correctly if map-worker is not installed.

What these skills do

reduce-orchestrator

The orchestrator enforces a loop:

1. PLAN: write a run contract + deterministic plan.json
2. MAP: spawn multiple workers (must use map-worker) to do the actual work
3. REDUCE: summarize worker reports into consolidated findings + open questions
4. VERIFY (mandatory): spawn parallel verify workers to confirm key claims / contradictions
5. DECIDE: either iterate (schedule more work) or finish (write final.json/final.md)

All state is persisted under .rlm/ so the orchestrator can rehydrate run state deterministically rather than relying on chat history.

map-worker

The worker skill provides a strict invocation contract:

• do one task,
• keep chat output minimal,
• write a narrative report to report_path (required),
• include contract compliance notes when a run contract is provided.

Warning: token usage can be huge

This workflow can consume a very large number of tokens because it intentionally:

• runs many parallel workers,
• runs an additional Verify phase (parallel again),
• and often repeats across multiple iterations.

If you are running with strong models and high reasoning effort, costs can balloon quickly. Treat this skill as a “batch research / batch debugging” workflow, not a casual helper.

Practical safety tips:

• Keep plan.json granular (micro/meso tasks) and cap max_workers / max_iterations.
• Use smaller compute tiers for exploration; upgrade only the tasks that truly need it.
• Keep Verify focused (top-K issues + contradictions). Avoid “verify everything”.
• Prefer evidence by file paths and short quotes instead of re-summarizing huge texts.

Acknowledgements / inspiration

This skill was inspired by (and is grateful to):

• RLM: https://alexzhang13.github.io/blog/2025/rlm/
• AI MapReduce: https://danielsada.tech/blog/ai-map-reduce/

Install

1. Clone this repo.

2. Copy both skills into your Codex skills directory:

for skill in map-worker reduce-orchestrator; do
  mkdir -p "$CODEX_HOME/skills/$skill"
  rsync -a --delete "$skill/" "$CODEX_HOME/skills/$skill/"
done

If CODEX_HOME is not set, it is typically ~/.codex.

How it works (high level)

Persistent run state (.rlm/)

The orchestrator uses a deterministic filesystem layout to keep runs reproducible and concurrency-safe:

• .rlm/runs/<run_id>/ for the active run (plan, reports, final output, etc.)
• .rlm/archive/<archive_id>/ for immutable snapshots of completed runs
• .rlm/locks/ for per-run and cleanup locks

This enables:

• State rehydration (reload from disk instead of chat)
• Concurrency safety (avoid two orchestrators clobbering the same run)
• Retention / TTL cleanup (clean old archives safely)

“Orchestrator doesn’t do the work”

The orchestrator is a planner/dispatcher/integrator. It should not do substantive domain work itself.

All real work is delegated to map-worker tasks, whose outputs are written as narrative reports to deterministic report_path locations. Reduction and verification consume those report files as the source of truth.

Deferred opportunities (don’t lose good ideas)

When you run many workers in parallel, they often surface valuable improvements that are out of scope for the current run (refactors, cleanup, follow-up experiments, better diagnostics, etc.). This system is designed to capture those opportunities instead of losing them in chat scrollback.

reduce-orchestrator supports a “Deferred opportunities” mechanism:

• workers record out-of-scope ideas under a dedicated Deferred opportunities section in their reports,
• the orchestrator aggregates them into .rlm/runs/<run_id>/deferred.json using rlm_collect_deferred.py,
• and final outputs are expected to reference those deferred items so you can schedule follow-up runs intentionally.

Task sizing: granularity and compute_tier

The orchestrator expects each worker task to be defined along two axes:

• granularity: how large the task unit is (controls parallelism and scope risk)
• compute_tier: how strong the model/effort should be (controls reasoning capacity and cost)

Recommended granularity vocabulary:

• micro: one narrow outcome; ideally 1 file / 1 function / 1 command; trivially reviewable
• meso: a small cohesive change across a few files or a small subsystem
• macro: cross-cutting / large-context work (generally avoid assigning macro directly; split into micro/meso + a join worker)

Recommended planning rule:

• If you think you need macro, you usually want 3–8 micro/meso workers + 1 join instead.

Typical compute_tier usage:

• Start with lower tiers for broad exploration, then upgrade only the tasks that truly need deeper synthesis.
• If results are consistently low-quality, either split the task (reduce granularity) or upgrade compute_tier.

Current compute_tier model mapping (defaults)

These are the current defaults used by rlm_run_ready.py when it spawns workers:

• standard → gpt-5.1-codex-mini with reasoning effort medium
• standard-plus → gpt-5.2-codex with reasoning effort medium
• heavy → gpt-5.2-codex with reasoning effort xhigh

Changing the model settings (when needed)

You can and should change model settings when it improves quality/cost:

• Per task: set compute_tier on that task in .rlm/runs/<run_id>/plan.json.
• Per run default: pass --default-compute-tier to rlm_run_ready.py (used when a task does not specify compute_tier).
• Change the mapping itself: edit _COMPUTE_MAP in reduce-orchestrator/scripts/rlm_run_ready.py and re-install/re-package the skill.

Mandatory Verify phase

Even when Map reports look good, Verify is required. It focuses on:

• top-K claims by impact,
• contradictions,
• weakly-supported statements.

This is intentionally conservative: it trades extra cost for fewer “hallucinated conclusions”.

Quickstart (typical run sequence)

The authoritative workflow and command templates live in reduce-orchestrator/SKILL.md. The worker contract lives in map-worker/SKILL.md.

At a high level:

1. Initialize a run (locks + TTL cleanup) with rlm_admin.py
2. Write .rlm/runs/<run_id>/plan.json and .rlm/runs/<run_id>/artifacts/context/run_contract.md
3. Spawn READY map tasks (with inflight cap) using rlm_run_ready.py
4. Monitor progress via report freshness with rlm_watch_reports.py
5. Build a deterministic inventory with rlm_reduce.py
6. Reduce + decide next steps
7. Spawn verify tasks
8. Write final.json/final.md, archive, and cleanup using rlm_admin.py

Repo layout

• reduce-orchestrator/: the Codex skill folder (orchestrator)
• map-worker/: the Codex skill folder (worker)
• dist/reduce-orchestrator.skill: packaged .skill file (zip)
• dist/map-worker.skill: packaged .skill file (zip)