~cytrogen/gstack

gstack/test/skill-e2e-learnings.test.ts -rw-r--r-- 5.4 KiB
81159512 — Garry Tan 8 days ago
feat: recursive self-improvement — operational learning + full skill wiring (v0.13.8.0) (#647)

* refactor: remove dead contributor mode, replace with operational self-improvement slot

Contributor mode never fired in 18 days of heavy use (required manual opt-in
via gstack-config, gated behind _CONTRIB=true, wrote disconnected markdown).

Removes: generateContributorMode(), _CONTRIB bash var, 2 E2E tests, touchfile
entry, doc references. Cleans up skip-lists in plan-ceo-review, autoplan,
review resolver, and document-release templates.

The operational self-improvement system (next commit) replaces this slot with
automatic learning capture that requires no opt-in.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: operational self-improvement — every skill learns from failures

Adds universal operational learning capture to the preamble completion protocol.
At the end of every skill session, the agent reflects on CLI failures, wrong
approaches, and project quirks, logging them as type "operational" to the
learnings JSONL. Future sessions surface these automatically.

- generateCompletionStatus(ctx) now includes operational capture section
- Preamble bash shows top 3 learnings inline when count > 5
- New "operational" type in generateLearningsLog alongside pattern/pitfall/etc
- Updated unit tests + operational seed entry in learnings E2E

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: wire learnings into all insight-producing skills

Adds LEARNINGS_SEARCH and/or LEARNINGS_LOG to 10 skill templates that
produce reusable insights but were previously disconnected from the
learning system:

- office-hours, plan-ceo-review, plan-eng-review: add LOG (had SEARCH)
- plan-design-review: add both SEARCH + LOG (had neither)
- design-review, design-consultation, cso, qa, qa-only: add both
- retro: add SEARCH (had LOG)

13 skills now fully participate in the learning loop (read + write).
Every review, QA, investigation, and design session both consults prior
learnings and contributes new ones.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add operational-learning E2E test (gate-tier)

Validates the write path: agent encounters a CLI failure, logs an
operational learning to JSONL via gstack-learnings-log. Replaces the
removed contributor-mode E2E test.

Setup: temp git repo, copy bin scripts, set GSTACK_HOME.
Prompt: simulated npm test failure needing --experimental-vm-modules.
Assert: learnings.jsonl exists with type=operational entry.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: learnings-show E2E slug mismatch — seed at computed slug, not hardcoded

The test seeded learnings at projects/test-project/ but gstack-slug computes
the slug from basename(workDir) when no git remote exists. The agent's search
looked at the wrong path and found nothing.

Fix: compute slug the same way gstack-slug does (basename + sanitize) and
seed the learnings there.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.13.8.0)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ae0a9ad1 — Garry Tan 10 days ago
feat: GStack Learns — per-project self-learning infrastructure (v0.13.4.0) (#622)

* feat: learnings + confidence resolvers — cross-skill memory infrastructure

Three new resolvers for the self-learning system:
- LEARNINGS_SEARCH: tells skills to load prior learnings before analysis
- LEARNINGS_LOG: tells skills to capture discoveries after completing work
- CONFIDENCE_CALIBRATION: adds 1-10 confidence scoring to all review findings

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: learnings bin scripts — append-only JSONL read/write

gstack-learnings-log: validates JSON, auto-injects timestamp, appends to
~/.gstack/projects/$SLUG/learnings.jsonl. Append-only (no mutation).

gstack-learnings-search: reads/filters/dedupes learnings with confidence
decay (observed/inferred lose 1pt/30d), cross-project discovery, and
"latest winner" resolution per key+type.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: learnings count in preamble output

Every skill now prints "LEARNINGS: N entries loaded" during preamble,
making the compounding loop visible to the user.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: integrate learnings + confidence into 9 skill templates

Add {{LEARNINGS_SEARCH}}, {{LEARNINGS_LOG}}, and {{CONFIDENCE_CALIBRATION}}
placeholders to review, ship, plan-eng-review, plan-ceo-review, office-hours,
investigate, retro, and cso templates. Regenerated all SKILL.md files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: /learn skill — manage project learnings

New skill for reviewing, searching, pruning, and exporting what gstack
has learned across sessions. Commands: /learn, /learn search, /learn prune,
/learn export, /learn stats, /learn add.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: self-learning roadmap — 5-release design doc

Covers: R1 GStack Learns (v0.14), R2 Review Army (v0.15), R3 Smart Ceremony
(v0.16), R4 /autoship (v0.17), R5 Studio (v0.18). Inspired by Compound
Engineering, adapted to GStack's architecture.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: learnings bin script unit tests — 13 tests, free

Tests gstack-learnings-log (valid/invalid JSON, timestamp injection,
append-only) and gstack-learnings-search (dedup, type/query/limit filters,
confidence decay, user-stated no-decay, malformed JSONL skip).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.13.4.0)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: learnings resolver + bin script edge case tests — 21 new tests, free

Adds gen-skill-docs coverage for LEARNINGS_SEARCH, LEARNINGS_LOG, and
CONFIDENCE_CALIBRATION resolvers. Adds bin script edge cases: timestamp
preservation, special characters, files array, sort order, type grouping,
combined filtering, missing fields, confidence floor at 0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: sync package.json version with VERSION file (0.13.4.0)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: gitignore .factory/ — generated output, not source

Same pattern as .claude/skills/ and .agents/. These SKILL.md files are
generated from .tmpl templates by gen:skill-docs --host factory.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: /learn E2E — seed 3 learnings, verify agent surfaces them

Seeds N+1 query pattern, stale cache pitfall, and rubocop preference
into learnings.jsonl, then runs /learn and checks that at least 2/3
appear in the agent's output. Gate tier, ~$0.25/run.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>