--- slug: "python-x-7" source_type: "clawhub" source_url: "https://clawhub.ai/skills/py" repo: "" source_file: "description" --- --- name: py slug: py version: 1.0.4 changelog: "Full coverage pass: deeper guides, situation-named files, and per-user configuration" description: >- Writes, debugs, and reviews Python code ā€” runtime traps, packaging, typing, async, tests, performance. Use when Python raises or misbehaves: ModuleNotFoundError, circular imports, AttributeError on None, UnboundLocalError, UnicodeDecodeError, mutable default arguments, `is` vs `==`, float rounding, naive vs aware datetimes, or a wrong answer with no exception; when pip, uv, poetry, venv, pyproject or a lockfile fight over dependencies, or a package installs but will not import; when threads, asyncio, multiprocessing or the GIL hang, deadlock, or leak memory; when pytest passes but should not, mocks patch the wrong module, or async tests never run; when mypy or pyright errors need clearing; when a script is slow, eats RAM, or gets OOM-killed; when subprocess calls hang or logs never appear; or when upgrading Python breaks the build. Not for library-specific problems ā€” pandas, numpy, django, fastapi and flask have their own skills. homepage: https://clawic.com/skills/py metadata: clawdbot: emoji: šŸ requires: bins: - python3 os: - linux - darwin - win32 displayName: Python configPaths: - ~/Clawic/data/py/ --- User preferences live in `~/Clawic/data/py/config.yaml` (see Configuration); nothing else is stored on the user's machine. If you have data at an old location (`~/py/` or `~/clawic/py/`), move it to `~/Clawic/data/py/`. ## When To Use - Writing or reviewing Python ā€” scan Core Rules and Output Gates before committing - Debugging wrong results without exceptions: state leaking across calls, aliased mutations, silent coercion - Any traceback whose cause is not obvious in ten seconds, plus hangs, memory growth, and segfaults - Environment and dependency work: venvs, lockfiles, editable installs, publishing, version upgrades - Choosing a concurrency model, a data model, or a typing strategy, and living with the consequences - Hardening a script into a tool: arguments, exit codes, logging, timeouts, retries, untrusted input - Not for library-specific issues ā€” `pandas`, `numpy`, `django`, `fastapi`, `flask` have their own skills ## Quick Reference | Situation | Play | |-----------|------| | Wrong value, no exception | `repr()` at input, midpoint, output; the first wrong one is the bug ā€” suspect aliasing, coercion, stale default ā†’ `debugging.md` | | A traceback you have not seen before | Message ā†’ cause table, then the matching chain ā†’ `debugging.md` | | Installed it, still `ModuleNotFoundError` | Wrong interpreter: check `sys.executable`, then use `python -m pip` ā†’ `packaging.md` | | venvs, lockfiles, uv vs pip vs poetry, publishing | Applications pin everything with hashes; libraries declare ranges ā†’ `packaging.md` | | `cannot import name X from partially initialized module` | Circular import; switch to `import a` + `a.f()` before restructuring ā†’ `imports.md` | | `is` vs `==`, float rounding, money, NaN, string surgery | `is` only for `None`/`True`/`False`/sentinels; `Decimal` from strings ā†’ `types.md` | | Aliasing, copies, ordering, membership cost | `[[]]*3` shares one list; `x in list` is O(n) ā†’ `collections.md` | | State leaks across calls: defaults, closures, decorators, generators | Defaults evaluate once, at `def` time ā†’ `functions.md` | | Class design: shared attributes, hashability, MRO, `__slots__` | `__eq__` without `__hash__` makes instances unhashable ā†’ `classes.md` | | dict vs dataclass vs NamedTuple vs pydantic; validating input | Validate once at the boundary, plain objects inside ā†’ `data-modeling.md` | | Slow, hanging, or racy: GIL, threads, asyncio, multiprocessing | Pure-Python CPU ā†’ processes; I/O ā†’ threads or asyncio ā†’ `concurrency.md` | | Too slow, or too much memory | Profile first, then fix the top line or nothing ā†’ `performance.md` | | Tests pass but should not; mock patches nothing; async tests skipped | Patch where the name is USED; `autospec=True` ā†’ `testing.md` | | mypy or pyright errors, annotating a legacy codebase | Freeze a baseline, then ratchet one package at a time ā†’ `type-checking.md` | | Formatting, import order, lint rules, pre-commit, style review comments | One formatter owns layout; `ruff check` for bugs; same versions in hook and CI ā†’ `linting.md` | | `UnicodeDecodeError`, CSV and JSON traps, atomic writes, temp files | Declare `encoding="utf-8"`; write temp then `os.replace` ā†’ `files.md` | | Timezones, DST, parsing dates, measuring elapsed time | Aware datetimes everywhere; `time.monotonic()` for durations ā†’ `datetime.md` | | Calling git/ffmpeg/anything external, hanging or failing silently | `run([...], check=True, timeout=ā€¦)`, never `shell=True` with interpolation ā†’ `subprocess.md` | | Writing a script or CLI: arguments, exit codes, pipes, signals | `sys.exit(main())`; stdout for data, stderr for logs ā†’ `cli.md` | | Nothing appears in the logs, or deciding what to log | Four gates: basicConfig no-op, logger level, handler level, propagation ā†’ `logging.md` | | Exception design, chaining, retries, timeouts | `raise X from exc`; exponential backoff with jitter ā†’ `errors.md` | | Calling an HTTP API: sessions, status codes, redirects, streaming a big body | One client per process; `raise_for_status()` before `.json()` ā†’ `http.md` | | A local database, cache, or queue in a file; `database is locked` | WAL plus an explicit busy timeout; one connection per thread ā†’ `sqlite.md` | | Untrusted input: pickle, eval, SQL, paths, archives, secrets | Unpickling executes code; parameterize SQL; contain paths ā†’ `security.md` | | "Is there something in the stdlib for this?" | Counter, deque, bisect, itertools, pathlib, secrets, sqlite3 ā†’ `stdlib.md` | | Notebook works for you and nobody else; results change between runs | Restart and run all; `%pip` not `!pip`; strip outputs before committing ā†’ `notebooks.md` | | Upgrading Python, choosing a version, a module that vanished | Run the suite with `-W error::DeprecationWarning` on the OLD version first ā†’ `versions.md` | | Anything else | Core Rules below, then reproduce with `python -I -c ''` and re-add one thing at a time | Each file above is one sub-job and is self-contained: read SKILL.md by default, open exactly one guide when the situation matches. ## Core Rules 1. No mutable defaults: `def f(xs=None)` then `if xs is None: xs = []`. Never `xs = xs or []` ā€” a caller passing an empty list to be filled gets a fresh list instead; their reference stays empty. 2. `is` only for `None`, `True`, `False`, and sentinel objects; `==` for everything else. Interning makes `is` on ints and strings pass in tests and fail in production (`types.md`). 3. Never mutate the collection you iterate ā€” dicts raise `RuntimeError`, lists silently skip elements. Iterate a copy (`for x in list(xs)`) or collect changes and apply after. 4. Match concurrency to workload: pure-Python CPU ā†’ `multiprocessing`; I/O ā†’ threads or asyncio; threads never speed up pure-Python CPU work, because the GIL serializes bytecode (`concurrency.md`). 5. `except Exception:`, never bare `except:` ā€” bare also catches `KeyboardInterrupt` and `SystemExit`, making the process unkillable. Re-raise with bare `raise`, keeping the original traceback (`errors.md`). 6. Files, locks, sockets, connections: always `with`. CPython's refcounting closes leaked handles by accident; exceptions and other interpreters (PyPy) expose the leak. 7. Money and exact decimals: `decimal.Decimal('1.10')` from strings ā€” `Decimal(1.1)` imports the float error it was meant to avoid. Float comparisons via `math.isclose` (`types.md`). 8. Declare `encoding='utf-8'` at every I/O boundary ā€” the default follows the platform locale until UTF-8 becomes the default (PEP 686, `python >=3.15`), so the code breaks first on Windows and in lean containers (`files.md`). 9. Guard entry points with `if __name__ == "__main__":` ā€” top-level code runs on every import and again in every `multiprocessing` spawn child (`imports.md`). ## Version Floors The syntax and stdlib floors that shape everyday code. Individual guides carry additional floors ā€” mostly changed defaults and new keyword arguments ā€” inline in this same `python >=X.Y` form, next to the instruction each one gates. Support windows, removals, and the upgrade procedure: `versions.md`. | Feature | Needs | |---|---| | Guaranteed dict insertion order, dataclasses, `breakpoint()`, `from __future__ import annotations`, `sys.stdout.reconfigure` | python >=3.7 | | Walrus `:=`, positional-only `/`, `functools.cached_property`, `TypedDict`/`Protocol`/`Literal`/`Final`, `basicConfig(force=)`, self-documenting `f"{value=}"` | python >=3.8 | | Builtin generics `list[int]`, `d1 \| d2` (dict merge), `zoneinfo`, `removeprefix`/`removesuffix`, `functools.cache`, `asyncio.to_thread`, `importlib.resources.files`, `Path.is_relative_to`, `graphlib`, `argparse.BooleanOptionalAction`, `executor.shutdown(cancel_futures=)` | python >=3.9 | | `X \| Y` unions, `match`, `zip(strict=True)`, dataclass `slots=`/`kw_only=`, `itertools.pairwise`, `ParamSpec`, `EncodingWarning` and `-X warn_default_encoding` | python >=3.10 | | `ExceptionGroup`/`except*`, `asyncio.TaskGroup`, `tomllib`, `Self`, `StrEnum`, `exc.add_note()`, `contextlib.chdir`, full-ISO `fromisoformat`, `NotRequired` | python >=3.11 | | PEP 695 generics (`def f[T]()`), `@override`, `itertools.batched`, `tarfile(filter="data")` | python >=3.12 | | Experimental free-threaded build; PEP 594 module removals (`cgi`, `telnetlib`, ā€¦) | python >=3.13 | | `forkserver` as the Linux multiprocessing default, lazy annotations (PEP 649) | python >=3.14 | | UTF-8 as the default text encoding everywhere (PEP 686), retiring the locale default | python >=3.15 | ## Output Gates Before delivering Python code, check: - No mutable default argument, and no `xs or []` where an empty argument is meaningful - Every text `open`, `subprocess`, and decode boundary declares `encoding="utf-8"`; every file, lock, socket, and connection is acquired in a `with` - Every network, subprocess, lock, and queue call has a timeout - Exceptions: narrowest class caught, chained with `from`, nothing swallowed without a log line or a written reason - Logging through a module logger with lazy `%s` arguments and no secrets - Syntax and stdlib stay within `min_python`; annotations match `type_strictness` - Nothing user-supplied reaches `eval`, `pickle`, a shell string, an SQL string, or a path join without containment - The new test was seen failing without the fix in place ā€” `change_workflow` decides when that run happens, never whether it happened ## Configuration User-dependent variables. Defaults apply until the user states a preference; store them in `~/Clawic/data/py/config.yaml`. Never interview the user ā€” record a preference the moment it is stated. | Variable | Type | Default | Effect | |---|---|---|---| | min_python | 3.9 \| 3.10 \| 3.11 \| 3.12 \| 3.13 \| 3.14 | 3.11 | Gates which Version Floors features may be emitted unguarded, and which fallback appears instead | | package_manager | pip \| uv \| poetry \| pdm \| conda | pip | Chooses the install, lock, and venv commands in every example (`packaging.md`) | | type_strictness | none \| gradual \| strict | gradual | How far annotations go: none skips hints, gradual annotates public boundaries, strict turns on `disallow_untyped_defs` (`type-checking.md`) | | test_runner | pytest \| unittest | pytest | Shape of emitted tests, fixtures, and mocking guidance (`testing.md`) | | target_os | linux \| macos \| windows \| cross | cross | Path, encoding, and multiprocessing start-method assumptions; `cross` flags all three | | src_layout | src \| flat | src | Project scaffolding, editable-install and import advice (`packaging.md`) | | line_length | number (79-120) | 88 | Formatting of emitted code and the `ruff`/`black` config value (`linting.md`) | | change_workflow | test-first \| fix-first | test-first | Order of the work: test-first writes the failing test before the fix, fix-first patches first and backfills the test against the reverted change (`testing.md`) | Preference areas ā€” customizable dimensions; a stated preference gets recorded in config.yaml and applied: - **Tooling**: formatter and linter (ruff vs black+flake8), checker (mypy vs pyright), task runner, notebook vs module workflow ā€” affects every emitted config block (`linting.md`) - **Conventions**: docstring style (Google/NumPy/reST), naming, module granularity, error-message phrasing ā€” affects generated code and review comments - **Platform**: deployment target (container, serverless, desktop, embedded), CPU architecture, private package index, CI provider ā€” affects `packaging.md` and `versions.md` guidance - **Safety posture**: how aggressively to flag `pickle`, `eval`, `shell=True`, missing timeouts and unpinned dependencies ā€” affects `security.md` and `cli.md` - **Dependencies**: stdlib-only constraints, banned or mandated libraries (requests vs httpx, pydantic vs dataclasses), tolerance for new transitive dependencies ā€” affects every recommendation - **Output format**: whole file vs minimal diff, how much explanation, whether tests accompany every change ā€” affects the shape of the answer, never the correctness rules - **Work order**: which gates run before a change is proposed rather than after ā€” checker and suite green first, a plan approved before editing, a `--dry-run` pass on destructive scripts, a profile before any optimization ā€” affects the sequence of every task, never the correctness rules ## Traps | Trap | Why it fails | Do instead | |------|-------------|------------| | Bare `pip install` | Installs into whatever interpreter is first on PATH ā€” the "installed it, still ModuleNotFoundError" loop | `python -m pip install` inside an activated venv (`packaging.md`) | | `except Exception: pass` | Turns a crash into a wrong answer three layers away | Log with `logger.exception`, or write down why this failure is expected (`errors.md`) | | `assert user.is_admin` as a check | `python -O` removes every assert, including that one | Raise explicitly (`security.md`) | | `time.time()` to measure a duration | NTP can step the clock backwards mid-measurement and produce negative elapsed time | `time.monotonic()` (`datetime.md`) | | `datetime.utcnow()` | Returns a NAIVE datetime holding UTC: compares wrong against aware values, deprecated in `python >=3.12` | `datetime.now(timezone.utc)` (`datetime.md`) | | f-strings inside logging calls | Formats even when the level is disabled, and every line becomes a unique string to the aggregator | `log.info("user %s", uid)` (`logging.md`) | | `sys.path.append(...)` to fix an import | Works from one entry point, breaks under pytest, packaging, and any other cwd | Editable install with a `src/` layout (`imports.md`) | | `shell=True` with an interpolated value | Command injection ā€” and `/bin/sh` is not bash | Argument list; `shlex.quote` if a shell is unavoidable (`subprocess.md`) | | `verify=False` to silence an SSL error | Disables peer authentication for every request in the process | Install the CA bundle (`security.md`) | | `pickle` across a process, user, or network boundary | Unpickling executes attacker-controlled code before you can inspect it | JSON/msgpack, or an HMAC-signed payload (`security.md`) | | `raise e` when re-raising | Appends the current line and hides where the exception really came from | Bare `raise` (`errors.md`) | | Type hints treated as validation | Never enforced at runtime: `def f(x: int)` happily takes a string | Checker in CI plus runtime validation at the boundary (`type-checking.md`) | | `x in list` or `list.pop(0)` inside a loop | O(n) per operation makes the loop O(nĀ²) | `set` for membership, `deque` for both ends (`collections.md`) | ## Where Experts Disagree - **asyncio vs threads.** asyncio is not "faster Python": for moderate I/O concurrency, `ThreadPoolExecutor` matches it with far less ceremony. Boundary: high connection counts AND an async-native stack end to end ā†’ asyncio; one blocking driver anywhere in the chain forfeits the benefit (`concurrency.md`). - **How much typing.** The strict school annotates everything and treats `Any` as a defect; the pragmatic school types boundaries and lets internals be inferred. Boundary: libraries and cross-team APIs ā†’ annotate fully; single-owner scripts and glue ā†’ boundaries only. Either way, one checker in CI (`type-checking.md`). - **Packaging tooling.** `venv` + `pip` exists on every machine and surprises nobody; `uv`/`poetry`/`hatch` are faster and manage more. Boundary: a project with a real dependency problem, many contributors, or several interpreters justifies the tool; one that installs three libraries does not (`packaging.md`). - **EAFP vs LBYL.** Python's culture prefers `try/except` over pre-checking, and it is genuinely more correct under concurrency ā€” the file can disappear between `exists()` and `open()`. Boundary: LBYL when the operation is expensive or irreversible, EAFP when the failure is cheap and the race is real. ## Related Skills More Clawic skills, get them at https://clawic.com/skills/py (install if the user confirms): - `pandas` ā€” DataFrame-specific traps and idioms - `fastapi` ā€” async web APIs in Python - `django` ā€” Django ORM and framework patterns - `profiling` ā€” when the question is "why is it slow", measure first - `debugging` ā€” language-agnostic fault isolation (bisection, hypothesis discipline, minimal repro); the Python-specific version of that job is this skill's `debugging.md` ## Feedback - If useful, star it: https://clawic.com/skills/py - Latest version: https://clawic.com/skills/py Part of [Clawic](https://clawic.com), the verified skill library. Get this skill: https://clawic.com/skills/py.