From 3386405c7c9b779779517163046b4c5bcd0f7e89 Mon Sep 17 00:00:00 2001 From: Felix Martin Date: Fri, 5 Jun 2026 19:16:47 -0400 Subject: [PATCH] docs: implementation plan for off-screen AW-backed memory Nine TDD tasks: internal/aw REST client (EnsureBucket/Insert/Recent), internal/memory port (Store + awStore over-fetch filter + nop + fake), KEEL_AW_URL setting, harness.Services.Memory wiring with nop fallback, off-screen write path (proposal_made/action_taken/proposal_dismissed) and read path (## Recently proposed + follow-up prompt rule), plus full and live verification. Derived from the 2026-06-05 design spec. Co-Authored-By: Claude Opus 4.8 --- .../plans/2026-06-05-offscreen-memory.md | 1254 +++++++++++++++++ 1 file changed, 1254 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-05-offscreen-memory.md diff --git a/docs/superpowers/plans/2026-06-05-offscreen-memory.md b/docs/superpowers/plans/2026-06-05-offscreen-memory.md new file mode 100644 index 0000000..b707a50 --- /dev/null +++ b/docs/superpowers/plans/2026-06-05-offscreen-memory.md @@ -0,0 +1,1254 @@ +# Off-screen AW-backed Memory — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Give Keel its first durable cross-run memory by adding an AW REST client and a `memory.Store` port, then wiring the off-screen mode to record its decisions to the `keel.events` bucket and read its recent history back into the next proposal. + +**Architecture:** Two new leaf packages — `internal/aw` (thin ActivityWatch REST client: buckets + events) and `internal/memory` (the `Store` port with an AW-backed adapter, a nop, and a test fake). `harness.Services` gains a `Memory` field; `cmd/keeld` constructs it (bucket ensured at startup, nop fallback if AW is down). Off-screen writes `proposal_made` / `action_taken` / `proposal_dismissed` and reads them back, all best-effort. + +**Tech Stack:** Go 1.26, `net/http` + `encoding/json` (AW REST at `http://localhost:5600/api/0`), `net/http/httptest` for client tests, existing ports-and-adapters style. + +**Spec:** `docs/superpowers/specs/2026-06-05-offscreen-memory-design.md` + +--- + +## File structure + +- `internal/aw/aw.go` (create) — `Event`, `Client`, `New`, `EnsureBucket`, `Insert`, `Recent`. No Keel concepts. +- `internal/aw/aw_test.go` (create) — httptest-backed client tests. +- `internal/memory/memory.go` (create) — `Event`, `Store`, `AWClient`, `awStore`, `nopStore`, `Fake`, constructors. +- `internal/memory/memory_test.go` (create) — `Fake`, `nopStore`, `awStore` mapping/over-fetch tests. +- `internal/settings/settings.go` (modify) — add `AWURL` field + seed. +- `internal/settings/settings_test.go` (modify) — seed test. +- `internal/harness/services.go` (modify) — add `Memory memory.Store`. +- `cmd/keeld/main.go` (modify) — construct the store inside `buildServices`. +- `internal/mode/offscreen/offscreen.go` (modify) — mem/now/proposalID fields, write path, event constants, helpers. +- `internal/mode/offscreen/collect.go` (modify) — `briefHistory`, `relativeAge`, brief section. +- `internal/mode/offscreen/offscreen_test.go` (modify) — write-path + read-path tests. +- `internal/ai/proposer.go` (modify) — add the follow-up rule to the propose prompt. +- `internal/ai/proposer_test.go` (modify) — assert the rule is present. + +Event-kind names (`proposal_made`, …) live in the **offscreen** package — `memory` stays a generic event store. + +--- + +## Task 1: `internal/aw` — Client + EnsureBucket + +**Files:** +- Create: `internal/aw/aw.go` +- Test: `internal/aw/aw_test.go` + +- [ ] **Step 1: Write the failing test** + +```go +package aw + +import ( + "context" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "testing" +) + +func TestEnsureBucketPostsCreate(t *testing.T) { + var gotPath string + var gotBody map[string]string + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotPath = r.URL.Path + b, _ := io.ReadAll(r.Body) + _ = json.Unmarshal(b, &gotBody) + w.WriteHeader(http.StatusOK) + })) + defer srv.Close() + + c := New(srv.URL) + if err := c.EnsureBucket(context.Background(), "keel.events", "keel.event", "keel"); err != nil { + t.Fatalf("EnsureBucket: %v", err) + } + if gotPath != "/api/0/buckets/keel.events" { + t.Fatalf("path = %q", gotPath) + } + if gotBody["type"] != "keel.event" || gotBody["client"] != "keel" { + t.Fatalf("body = %+v", gotBody) + } +} + +func TestEnsureBucketTreatsExistsAsOK(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusNotModified) // AW returns 304 when the bucket exists + })) + defer srv.Close() + if err := New(srv.URL).EnsureBucket(context.Background(), "keel.events", "keel.event", "keel"); err != nil { + t.Fatalf("existing bucket should be OK, got %v", err) + } +} + +func TestEnsureBucketErrorsOnServerError(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusInternalServerError) + })) + defer srv.Close() + if err := New(srv.URL).EnsureBucket(context.Background(), "keel.events", "keel.event", "keel"); err == nil { + t.Fatal("expected error on 500") + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./internal/aw/` +Expected: FAIL — `undefined: New` / package has no Go files compiled. + +- [ ] **Step 3: Write minimal implementation** + +```go +// Package aw is a thin ActivityWatch REST client: it creates buckets and reads +// and writes events. It holds no Keel concepts, so it stays a leaf package. +package aw + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "net/http" + "os" + "strings" + "time" +) + +// Event is one ActivityWatch event. Duration is seconds; for Keel's discrete +// derived events it is 0. +type Event struct { + Timestamp time.Time + Duration float64 + Data map[string]any +} + +// Client talks to an aw-server over its /api/0 REST surface. +type Client struct { + baseURL string + http *http.Client +} + +// New builds a client for the given base URL (e.g. http://localhost:5600). +func New(baseURL string) *Client { + return &Client{ + baseURL: strings.TrimRight(baseURL, "/"), + http: &http.Client{Timeout: 5 * time.Second}, + } +} + +// EnsureBucket creates the bucket if absent. It is idempotent: an already-exists +// response (304) is success. +func (c *Client) EnsureBucket(ctx context.Context, id, eventType, client string) error { + body, _ := json.Marshal(map[string]string{ + "client": client, + "type": eventType, + "hostname": hostname(), + }) + req, err := http.NewRequestWithContext(ctx, http.MethodPost, + c.baseURL+"/api/0/buckets/"+id, bytes.NewReader(body)) + if err != nil { + return err + } + req.Header.Set("Content-Type", "application/json") + resp, err := c.http.Do(req) + if err != nil { + return err + } + defer resp.Body.Close() + switch resp.StatusCode { + case http.StatusOK, http.StatusCreated, http.StatusNoContent, http.StatusNotModified: + return nil + default: + return fmt.Errorf("aw: create bucket %s: status %d", id, resp.StatusCode) + } +} + +func hostname() string { + if h, err := os.Hostname(); err == nil && h != "" { + return h + } + return "keel" +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go test ./internal/aw/` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add internal/aw/aw.go internal/aw/aw_test.go +git commit -m "feat(aw): AW REST client with idempotent EnsureBucket" +``` + +--- + +## Task 2: `internal/aw` — Insert + Recent + +**Files:** +- Modify: `internal/aw/aw.go` +- Test: `internal/aw/aw_test.go` + +- [ ] **Step 1: Write the failing test** (append to `aw_test.go`) + +```go +func TestInsertPostsEventArray(t *testing.T) { + var gotPath string + var gotEvents []map[string]any + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotPath = r.URL.Path + b, _ := io.ReadAll(r.Body) + _ = json.Unmarshal(b, &gotEvents) + w.WriteHeader(http.StatusOK) + })) + defer srv.Close() + + c := New(srv.URL) + ev := Event{Timestamp: time.Unix(1000, 0).UTC(), Data: map[string]any{"_kind": "proposal_made"}} + if err := c.Insert(context.Background(), "keel.events", ev); err != nil { + t.Fatalf("Insert: %v", err) + } + if gotPath != "/api/0/buckets/keel.events/events" { + t.Fatalf("path = %q", gotPath) + } + if len(gotEvents) != 1 || gotEvents[0]["data"].(map[string]any)["_kind"] != "proposal_made" { + t.Fatalf("events = %+v", gotEvents) + } +} + +func TestRecentDecodesNewestFirst(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if got := r.URL.Query().Get("limit"); got != "5" { + t.Errorf("limit = %q, want 5", got) + } + _, _ = io.WriteString(w, `[ + {"id":2,"timestamp":"2026-06-05T10:00:00+00:00","duration":0,"data":{"_kind":"proposal_made"}}, + {"id":1,"timestamp":"2026-06-04T10:00:00+00:00","duration":0,"data":{"_kind":"action_taken"}} + ]`) + })) + defer srv.Close() + + got, err := New(srv.URL).Recent(context.Background(), "keel.events", 5) + if err != nil { + t.Fatalf("Recent: %v", err) + } + if len(got) != 2 || got[0].Data["_kind"] != "proposal_made" { + t.Fatalf("got = %+v", got) + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./internal/aw/ -run 'Insert|Recent'` +Expected: FAIL — `c.Insert undefined` / `c.Recent undefined`. + +- [ ] **Step 3: Write minimal implementation** (append to `aw.go`) + +```go +type eventJSON struct { + Timestamp time.Time `json:"timestamp"` + Duration float64 `json:"duration"` + Data map[string]any `json:"data"` +} + +// Insert posts a single event to the bucket. The AW events endpoint takes a +// JSON array, so the event is wrapped in a one-element list. +func (c *Client) Insert(ctx context.Context, bucketID string, e Event) error { + body, _ := json.Marshal([]eventJSON{{Timestamp: e.Timestamp, Duration: e.Duration, Data: e.Data}}) + req, err := http.NewRequestWithContext(ctx, http.MethodPost, + c.baseURL+"/api/0/buckets/"+bucketID+"/events", bytes.NewReader(body)) + if err != nil { + return err + } + req.Header.Set("Content-Type", "application/json") + resp, err := c.http.Do(req) + if err != nil { + return err + } + defer resp.Body.Close() + if resp.StatusCode < 200 || resp.StatusCode >= 300 { + return fmt.Errorf("aw: insert into %s: status %d", bucketID, resp.StatusCode) + } + return nil +} + +// Recent returns up to limit events from the bucket, newest first (the AW +// default ordering for the events endpoint). +func (c *Client) Recent(ctx context.Context, bucketID string, limit int) ([]Event, error) { + url := fmt.Sprintf("%s/api/0/buckets/%s/events?limit=%d", c.baseURL, bucketID, limit) + req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) + if err != nil { + return nil, err + } + resp, err := c.http.Do(req) + if err != nil { + return nil, err + } + defer resp.Body.Close() + if resp.StatusCode < 200 || resp.StatusCode >= 300 { + return nil, fmt.Errorf("aw: read %s: status %d", bucketID, resp.StatusCode) + } + var raw []eventJSON + if err := json.NewDecoder(resp.Body).Decode(&raw); err != nil { + return nil, err + } + out := make([]Event, len(raw)) + for i, e := range raw { + out[i] = Event{Timestamp: e.Timestamp, Duration: e.Duration, Data: e.Data} + } + return out, nil +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go test ./internal/aw/` +Expected: PASS (all four tests). + +- [ ] **Step 5: Commit** + +```bash +git add internal/aw/aw.go internal/aw/aw_test.go +git commit -m "feat(aw): Insert and Recent over the events endpoint" +``` + +--- + +## Task 3: `internal/memory` — Store, nopStore, Fake + +**Files:** +- Create: `internal/memory/memory.go` +- Test: `internal/memory/memory_test.go` + +- [ ] **Step 1: Write the failing test** + +```go +package memory + +import ( + "context" + "testing" + "time" +) + +func TestFakeRecordThenRecentNewestFirstByKind(t *testing.T) { + f := NewFake() + ctx := context.Background() + _ = f.Record(ctx, Event{Kind: "proposal_made", At: time.Unix(1, 0), Data: map[string]any{"n": "a"}}) + _ = f.Record(ctx, Event{Kind: "action_taken", At: time.Unix(2, 0)}) + _ = f.Record(ctx, Event{Kind: "proposal_made", At: time.Unix(3, 0), Data: map[string]any{"n": "b"}}) + + got, err := f.Recent(ctx, "proposal_made", 5) + if err != nil { + t.Fatalf("Recent: %v", err) + } + if len(got) != 2 || got[0].Data["n"] != "b" { + t.Fatalf("want newest-first [b,a], got %+v", got) + } +} + +func TestFakeRecentRespectsLimit(t *testing.T) { + f := NewFake() + ctx := context.Background() + for i := 0; i < 4; i++ { + _ = f.Record(ctx, Event{Kind: "proposal_made", At: time.Unix(int64(i), 0)}) + } + got, _ := f.Recent(ctx, "proposal_made", 2) + if len(got) != 2 { + t.Fatalf("limit not honored: %d", len(got)) + } +} + +func TestNopStoreIsInert(t *testing.T) { + s := NewNop() + if err := s.Record(context.Background(), Event{Kind: "x"}); err != nil { + t.Fatalf("nop Record: %v", err) + } + got, err := s.Recent(context.Background(), "x", 5) + if err != nil || len(got) != 0 { + t.Fatalf("nop Recent = %+v, %v", got, err) + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./internal/memory/` +Expected: FAIL — `undefined: NewFake` / `NewNop`. + +- [ ] **Step 3: Write minimal implementation** + +```go +// Package memory is Keel's durable, cross-run memory port. Modes Record derived +// events and read them back with Recent; the AW-backed adapter persists them to +// an ActivityWatch bucket. Event-kind names are owned by the modes that write +// them — this package stays a generic event store. +package memory + +import ( + "context" + "sync" + "time" +) + +// Event is one derived event Keel chooses to remember. Kind is the event type +// (e.g. "proposal_made"); Data carries pointers and small values, not copies of +// other tools' source of truth. +type Event struct { + Kind string + At time.Time + Data map[string]any +} + +// Store records derived events and reads recent ones back by kind. +type Store interface { + Record(ctx context.Context, e Event) error + Recent(ctx context.Context, kind string, n int) ([]Event, error) +} + +// nopStore is the no-memory fallback used when AW is unreachable or disabled. +type nopStore struct{} + +// NewNop returns a Store that drops writes and returns no history. +func NewNop() Store { return nopStore{} } + +func (nopStore) Record(context.Context, Event) error { return nil } +func (nopStore) Recent(context.Context, string, int) ([]Event, error) { return nil, nil } + +// Fake is an in-memory Store for tests. +type Fake struct { + mu sync.Mutex + events []Event +} + +// NewFake returns an empty in-memory Store. +func NewFake() *Fake { return &Fake{} } + +func (f *Fake) Record(_ context.Context, e Event) error { + f.mu.Lock() + f.events = append(f.events, e) + f.mu.Unlock() + return nil +} + +func (f *Fake) Recent(_ context.Context, kind string, n int) ([]Event, error) { + f.mu.Lock() + defer f.mu.Unlock() + var out []Event + for i := len(f.events) - 1; i >= 0 && len(out) < n; i-- { + if f.events[i].Kind == kind { + out = append(out, f.events[i]) + } + } + return out, nil +} + +// Events returns a snapshot of everything recorded, for test assertions. +func (f *Fake) Events() []Event { + f.mu.Lock() + defer f.mu.Unlock() + return append([]Event(nil), f.events...) +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go test ./internal/memory/` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add internal/memory/memory.go internal/memory/memory_test.go +git commit -m "feat(memory): Store port with nop and fake adapters" +``` + +--- + +## Task 4: `internal/memory` — AW-backed Store + +**Files:** +- Modify: `internal/memory/memory.go` +- Test: `internal/memory/memory_test.go` + +- [ ] **Step 1: Write the failing test** (append to `memory_test.go`) + +```go +import "keel/internal/aw" // add to the existing import block + +// fakeAWClient is the AWClient subset, recording inserts and replaying a fixed +// recent list so we can test mapping and over-fetch filtering without HTTP. +type fakeAWClient struct { + inserted []aw.Event + recent []aw.Event // newest-first, mixed kinds + lastLimit int +} + +func (f *fakeAWClient) Insert(_ context.Context, _ string, e aw.Event) error { + f.inserted = append(f.inserted, e) + return nil +} +func (f *fakeAWClient) Recent(_ context.Context, _ string, limit int) ([]aw.Event, error) { + f.lastLimit = limit + return f.recent, nil +} + +func TestAWStoreRecordMapsKindIntoData(t *testing.T) { + fc := &fakeAWClient{} + s := NewAWStore(fc, "keel.events") + at := time.Unix(1000, 0).UTC() + if err := s.Record(context.Background(), Event{Kind: "proposal_made", At: at, Data: map[string]any{"proposal_id": "x"}}); err != nil { + t.Fatalf("Record: %v", err) + } + if len(fc.inserted) != 1 { + t.Fatalf("inserted %d", len(fc.inserted)) + } + got := fc.inserted[0] + if !got.Timestamp.Equal(at) || got.Data["_kind"] != "proposal_made" || got.Data["proposal_id"] != "x" { + t.Fatalf("mapped event = %+v", got) + } +} + +func TestAWStoreRecentOverfetchesAndFiltersByKind(t *testing.T) { + fc := &fakeAWClient{recent: []aw.Event{ + {Timestamp: time.Unix(3, 0), Data: map[string]any{"_kind": "proposal_made", "n": "b"}}, + {Timestamp: time.Unix(2, 0), Data: map[string]any{"_kind": "action_taken"}}, + {Timestamp: time.Unix(1, 0), Data: map[string]any{"_kind": "proposal_made", "n": "a"}}, + }} + s := NewAWStore(fc, "keel.events") + got, err := s.Recent(context.Background(), "proposal_made", 5) + if err != nil { + t.Fatalf("Recent: %v", err) + } + if fc.lastLimit != overfetch { + t.Fatalf("over-fetch limit = %d, want %d", fc.lastLimit, overfetch) + } + if len(got) != 2 || got[0].Kind != "proposal_made" || got[0].Data["n"] != "b" { + t.Fatalf("filtered = %+v", got) + } +} + +func TestAWStoreRecentCapsAtN(t *testing.T) { + var recent []aw.Event + for i := 0; i < 5; i++ { + recent = append(recent, aw.Event{Timestamp: time.Unix(int64(i), 0), Data: map[string]any{"_kind": "proposal_made"}}) + } + s := NewAWStore(&fakeAWClient{recent: recent}, "keel.events") + got, _ := s.Recent(context.Background(), "proposal_made", 2) + if len(got) != 2 { + t.Fatalf("cap not applied: %d", len(got)) + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./internal/memory/ -run AWStore` +Expected: FAIL — `undefined: NewAWStore` / `undefined: overfetch`. + +- [ ] **Step 3: Write minimal implementation** (append to `memory.go`, add `"keel/internal/aw"` to imports) + +```go +// overfetch bounds how many recent bucket events awStore pulls before filtering +// by kind, because keel.events interleaves all kinds. Events older than this +// window are not seen — acceptable for "recent". +const overfetch = 200 + +// kindKey is the data field carrying the Event.Kind inside an AW event. +const kindKey = "_kind" + +// AWClient is the subset of *aw.Client that awStore needs. Declaring it here +// lets tests inject a fake without HTTP; *aw.Client satisfies it structurally. +type AWClient interface { + Insert(ctx context.Context, bucketID string, e aw.Event) error + Recent(ctx context.Context, bucketID string, limit int) ([]aw.Event, error) +} + +type awStore struct { + c AWClient + bucketID string +} + +// NewAWStore returns a Store that persists events to the given AW bucket. +func NewAWStore(c AWClient, bucketID string) Store { + return &awStore{c: c, bucketID: bucketID} +} + +func (s *awStore) Record(ctx context.Context, e Event) error { + data := make(map[string]any, len(e.Data)+1) + for k, v := range e.Data { + data[k] = v + } + data[kindKey] = e.Kind + return s.c.Insert(ctx, s.bucketID, aw.Event{Timestamp: e.At, Duration: 0, Data: data}) +} + +func (s *awStore) Recent(ctx context.Context, kind string, n int) ([]Event, error) { + raw, err := s.c.Recent(ctx, s.bucketID, overfetch) + if err != nil { + return nil, err + } + var out []Event + for _, ev := range raw { + if len(out) >= n { + break + } + if k, _ := ev.Data[kindKey].(string); k == kind { + out = append(out, Event{Kind: kind, At: ev.Timestamp, Data: ev.Data}) + } + } + return out, nil +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go test ./internal/memory/` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add internal/memory/memory.go internal/memory/memory_test.go +git commit -m "feat(memory): AW-backed Store with over-fetch kind filtering" +``` + +--- + +## Task 5: `settings` — KEEL_AW_URL + +**Files:** +- Modify: `internal/settings/settings.go:21-25` (struct) and `:70-80` (SeedFromEnv) +- Test: `internal/settings/settings_test.go` + +- [ ] **Step 1: Write the failing test** (append to `settings_test.go`) + +```go +func TestSeedFromEnvAWURL(t *testing.T) { + t.Setenv("KEEL_AW_URL", "http://aw.example:9999") + if got := SeedFromEnv().AWURL; got != "http://aw.example:9999" { + t.Fatalf("AWURL = %q", got) + } +} + +func TestSeedFromEnvAWURLDefault(t *testing.T) { + t.Setenv("KEEL_AW_URL", "") + if got := SeedFromEnv().AWURL; got != "http://localhost:5600" { + t.Fatalf("default AWURL = %q", got) + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./internal/settings/ -run AWURL` +Expected: FAIL — `s.AWURL undefined`. + +- [ ] **Step 3: Write minimal implementation** + +In the `Settings` struct (after `KnowledgePath`), add: + +```go + AWURL string `json:"aw_url"` +``` + +Update the struct doc comment to mention the new var: + +```go +// Settings is the user-editable configuration. Field names mirror the env vars +// they replace: KEEL_AI_BACKEND, KEEL_MARVIN_CMD, KEEL_KNOWLEDGE_FILE, KEEL_AW_URL. +``` + +In `SeedFromEnv`, before the `return`, add the default and field: + +```go + awURL := os.Getenv("KEEL_AW_URL") + if awURL == "" { + awURL = "http://localhost:5600" + } + return Settings{ + AIBackend: backend, + MarvinCmd: os.Getenv("KEEL_MARVIN_CMD"), + KnowledgePath: os.Getenv("KEEL_KNOWLEDGE_FILE"), + AWURL: awURL, + } +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `go test ./internal/settings/` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add internal/settings/settings.go internal/settings/settings_test.go +git commit -m "feat(settings): KEEL_AW_URL with localhost default" +``` + +--- + +## Task 6: Wire `Memory` into Services and `cmd/keeld` + +**Files:** +- Modify: `internal/harness/services.go:16-24` +- Modify: `cmd/keeld/main.go:16-28` (imports) and `:60-73` (buildServices) + +This task is wiring; it is verified by `go build` plus a manual smoke against the running AW server (Task 9 re-checks end to end). + +- [ ] **Step 1: Add the Services field** + +In `internal/harness/services.go`, add the import `"keel/internal/memory"` and a field to `Services`: + +```go + Memory memory.Store +``` + +Update the struct doc to note: `Memory is Keel's durable cross-run event store (AW-backed, or a nop when AW is down).` + +- [ ] **Step 2: Construct the store in buildServices** + +In `cmd/keeld/main.go`, add imports `"keel/internal/aw"` and `"keel/internal/memory"`. Replace the body of `buildServices` so it builds memory after the AI backend validates: + +```go + buildServices := func(s settings.Settings) (harness.Services, error) { + backend, err := ai.NewBackend(s.AIBackend) + if err != nil { + return harness.Services{}, fmt.Errorf("%w: %v", settings.ErrInvalidBackend, err) + } + svc := ai.NewService(backend) + + awURL := s.AWURL + if awURL == "" { + awURL = "http://localhost:5600" + } + var mem memory.Store = memory.NewNop() + awc := aw.New(awURL) + if err := awc.EnsureBucket(context.Background(), "keel.events", "keel.event", "keel"); err != nil { + log.Printf("memory: AW unavailable at %s (%v); running without memory", awURL, err) + } else { + mem = memory.NewAWStore(awc, "keel.events") + log.Printf("memory: keel.events ready at %s", awURL) + } + + return harness.Services{ + AI: svc, + Tasks: tasks.NewMarvin(s.MarvinCmd), + Knowledge: knowledge.NewFileSource(s.KnowledgePath), + Enforce: enforce.NewGuard(), + Memory: mem, + Clock: time.Now, + }, nil + } +``` + +- [ ] **Step 3: Build and vet** + +Run: `go build ./... && go vet ./...` +Expected: no output, exit 0. + +- [ ] **Step 4: Smoke against the running AW server** + +Run: `go run ./cmd/keeld` (Ctrl-C after it logs). Expected log line: `memory: keel.events ready at http://localhost:5600`. +Verify the bucket now exists: +Run: `curl -s http://localhost:5600/api/0/buckets/ | python3 -c "import sys,json; print('keel.events' in json.load(sys.stdin))"` +Expected: `True` + +- [ ] **Step 5: Commit** + +```bash +git add internal/harness/services.go cmd/keeld/main.go +git commit -m "feat(keeld): construct AW memory store, nop fallback when AW is down" +``` + +--- + +## Task 7: Off-screen write path + +**Files:** +- Modify: `internal/mode/offscreen/offscreen.go` +- Test: `internal/mode/offscreen/offscreen_test.go` + +- [ ] **Step 1: Write the failing test** (append to `offscreen_test.go`; add `"keel/internal/memory"` to imports) + +```go +// newTestModeMem is newTestMode with an injected in-memory Store. +func newTestModeMem(t *testing.T, ft *fakeTasks, backend ai.Backend, mem *memory.Fake) *Mode { + t.Helper() + svc := harness.Services{ + AI: ai.NewService(backend), + Tasks: ft, + Memory: mem, + Clock: func() time.Time { return time.Unix(1000, 0) }, + Notify: func() {}, + } + return New(svc) +} + +// waitEvent polls the fake until an event of kind appears, or fails after 2s. +func waitEvent(t *testing.T, f *memory.Fake, kind string) memory.Event { + t.Helper() + deadline := time.Now().Add(2 * time.Second) + for time.Now().Before(deadline) { + for _, e := range f.Events() { + if e.Kind == kind { + return e + } + } + time.Sleep(5 * time.Millisecond) + } + t.Fatalf("event %q never recorded (events: %+v)", kind, f.Events()) + return memory.Event{} +} + +func TestProposeRecordsProposalMade(t *testing.T) { + mem := memory.NewFake() + m := newTestModeMem(t, &fakeTasks{}, okBackend(), mem) + if err := m.Command(context.Background(), "start", nil); err != nil { + t.Fatalf("start: %v", err) + } + waitStatus(t, m, statusProposed) + + ev := waitEvent(t, mem, "proposal_made") + if ev.Data["next_action"] != "call mum" || ev.Data["proposal_id"] == "" { + t.Fatalf("proposal_made data = %+v", ev.Data) + } +} + +func TestConfirmRecordsActionTakenWithSameID(t *testing.T) { + mem := memory.NewFake() + m := newTestModeMem(t, &fakeTasks{}, okBackend(), mem) + _ = m.Command(context.Background(), "start", nil) + waitStatus(t, m, statusProposed) + made := waitEvent(t, mem, "proposal_made") + + if err := m.Command(context.Background(), "confirm", nil); err != nil { + t.Fatalf("confirm: %v", err) + } + taken := waitEvent(t, mem, "action_taken") + if taken.Data["proposal_id"] != made.Data["proposal_id"] { + t.Fatalf("action_taken id %v != proposal_made id %v", taken.Data["proposal_id"], made.Data["proposal_id"]) + } +} + +func TestDismissRecordsProposalDismissed(t *testing.T) { + mem := memory.NewFake() + m := newTestModeMem(t, &fakeTasks{}, okBackend(), mem) + _ = m.Command(context.Background(), "start", nil) + waitStatus(t, m, statusProposed) + _ = waitEvent(t, mem, "proposal_made") + + if err := m.Command(context.Background(), "dismiss", nil); err != nil { + t.Fatalf("dismiss: %v", err) + } + _ = waitEvent(t, mem, "proposal_dismissed") +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `go test ./internal/mode/offscreen/ -run 'Records|Dismiss'` +Expected: FAIL — `New` ignores `Memory`, so no events are ever recorded (waitEvent times out). + +- [ ] **Step 3: Write minimal implementation** in `offscreen.go` + +Add imports: `"crypto/rand"`, `"encoding/hex"`, `"log"`, `"keel/internal/memory"`. + +Add event-kind constants near the existing status constants: + +```go +const ( + kindProposalMade = "proposal_made" + kindActionTaken = "action_taken" + kindProposalDismissed = "proposal_dismissed" +) +``` + +Add fields to the `Mode` struct (after `know`): + +```go + mem memory.Store // may be nil; record helpers guard + now func() time.Time // event timestamps; never nil after New +``` + +and after `done`: + +```go + proposalID string +``` + +In `New`, resolve the clock and wire memory: + +```go +func New(svc harness.Services) *Mode { + now := svc.Clock + if now == nil { + now = time.Now + } + m := &Mode{ + ai: svc.AI, + tasks: svc.Tasks, + know: svc.Knowledge, + mem: svc.Memory, + now: now, + status: statusPending, + } + m.async = harness.NewAsync(&m.mu, svc.Notify) + return m +} +``` + +In `start`, the apply closure's success branch records the proposal (off-lock, fire-and-forget): + +```go + func() { + if err != nil { + m.status = statusError + m.errMsg = err.Error() + return + } + m.status = statusProposed + m.proposal = &p + m.proposalID = newProposalID() + m.recordAsync(memory.Event{ + Kind: kindProposalMade, + At: m.now(), + Data: map[string]any{ + "proposal_id": m.proposalID, + "mode": "offscreen", + "next_action": p.NextAction, + "rationale": p.Rationale, + }, + }) + }) +``` + +In `confirm`, after `tasks.Create` succeeds, capture the id under the lock and record synchronously: + +```go +func (m *Mode) confirm(ctx context.Context) error { + m.mu.Lock() + p := m.proposal + m.mu.Unlock() + if p == nil { + return errors.New("offscreen: nothing to confirm") + } + if err := m.tasks.Create(ctx, tasks.NewTask(p.NextAction)); err != nil { + return err + } + m.mu.Lock() + id := m.proposalID + m.status = statusDone + m.done = true + m.mu.Unlock() + m.recordSync(memory.Event{ + Kind: kindActionTaken, + At: m.now(), + Data: map[string]any{"proposal_id": id, "mode": "offscreen", "next_action": p.NextAction}, + }) + return nil +} +``` + +Replace the `dismiss` case in `Command` to record: + +```go + case "dismiss": + m.mu.Lock() + id := m.proposalID + m.done = true + m.mu.Unlock() + m.recordSync(memory.Event{ + Kind: kindProposalDismissed, + At: m.now(), + Data: map[string]any{"proposal_id": id, "mode": "offscreen"}, + }) + return nil +``` + +Add the helpers at the end of the file: + +```go +// recordAsync persists an event off the mode's lock, best-effort. Used on the +// propose path, which sets the event under the async apply lock. +func (m *Mode) recordAsync(ev memory.Event) { + if m.mem == nil { + return + } + go func() { + if err := m.mem.Record(context.Background(), ev); err != nil { + log.Printf("offscreen: memory record %s: %v", ev.Kind, err) + } + }() +} + +// recordSync persists an event inline, best-effort. Used on confirm/dismiss, +// which already do synchronous work. +func (m *Mode) recordSync(ev memory.Event) { + if m.mem == nil { + return + } + if err := m.mem.Record(context.Background(), ev); err != nil { + log.Printf("offscreen: memory record %s: %v", ev.Kind, err) + } +} + +// newProposalID returns a short random hex id correlating a proposal with its +// outcome event. +func newProposalID() string { + var b [8]byte + if _, err := rand.Read(b[:]); err != nil { + return "proposal" + } + return hex.EncodeToString(b[:]) +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `go test ./internal/mode/offscreen/` +Expected: PASS — new tests plus the five existing tests (which inject no `Memory`, so the nil-guard keeps them green). + +- [ ] **Step 5: Commit** + +```bash +git add internal/mode/offscreen/offscreen.go internal/mode/offscreen/offscreen_test.go +git commit -m "feat(offscreen): record proposal_made/action_taken/proposal_dismissed" +``` + +--- + +## Task 8: Off-screen read path (brief history + prompt rule) + +**Files:** +- Modify: `internal/ai/proposer.go` (prompt rule) +- Test: `internal/ai/proposer_test.go` +- Modify: `internal/mode/offscreen/collect.go` (briefHistory + relativeAge + section) +- Test: `internal/mode/offscreen/offscreen_test.go` + +- [ ] **Step 1: Write the failing prompt test** (append to `proposer_test.go`) + +```go +func TestProposePromptHasFollowUpRule(t *testing.T) { + p := buildProposePrompt("brief") + if !strings.Contains(p, "recently proposed") { + t.Fatalf("prompt missing follow-up rule:\n%s", p) + } +} +``` + +(If `proposer_test.go` does not already import `strings`, add it.) + +- [ ] **Step 2: Run it to verify it fails** + +Run: `go test ./internal/ai/ -run FollowUp` +Expected: FAIL — rule text absent. + +- [ ] **Step 3: Add the rule** in `internal/ai/proposer.go` + +In `buildProposePrompt`, add a bullet to the `Rules:` block (after the `rationale:` rule): + +```go +- If the brief lists recently proposed actions, do not simply repeat them; if a recent one was dismissed or not done and still fits, follow up on it instead of inventing something new. +``` + +- [ ] **Step 4: Run it to verify it passes** + +Run: `go test ./internal/ai/` +Expected: PASS + +- [ ] **Step 5: Write the failing brief test** (append to `offscreen_test.go`) + +```go +func TestBriefIncludesRecentProposalsWithOutcome(t *testing.T) { + mem := memory.NewFake() + ctx := context.Background() + // at = the test clock (Unix 1000); confirmed walk, dismissed stretch. + _ = mem.Record(ctx, memory.Event{Kind: "proposal_made", At: time.Unix(1000, 0), + Data: map[string]any{"proposal_id": "w", "next_action": "walk"}}) + _ = mem.Record(ctx, memory.Event{Kind: "action_taken", At: time.Unix(1000, 0), + Data: map[string]any{"proposal_id": "w"}}) + _ = mem.Record(ctx, memory.Event{Kind: "proposal_made", At: time.Unix(1000, 0), + Data: map[string]any{"proposal_id": "s", "next_action": "stretch"}}) + _ = mem.Record(ctx, memory.Event{Kind: "proposal_dismissed", At: time.Unix(1000, 0), + Data: map[string]any{"proposal_id": "s"}}) + + m := newTestModeMem(t, &fakeTasks{}, okBackend(), mem) + brief := m.assembleBrief() + + if !strings.Contains(brief, "Recently proposed") { + t.Fatalf("brief missing history section:\n%s", brief) + } + if !strings.Contains(brief, "walk") || !strings.Contains(brief, "confirmed") { + t.Fatalf("brief missing confirmed walk:\n%s", brief) + } + if !strings.Contains(brief, "stretch") || !strings.Contains(brief, "dismissed") { + t.Fatalf("brief missing dismissed stretch:\n%s", brief) + } +} + +func TestBriefOmitsHistoryWhenNoMemory(t *testing.T) { + m := newTestMode(t, &fakeTasks{}, okBackend()) // no Memory injected + if strings.Contains(m.assembleBrief(), "Recently proposed") { + t.Fatal("history section should be absent without memory") + } +} +``` + +(Add `"strings"` to the offscreen test imports if not present.) + +- [ ] **Step 6: Run it to verify it fails** + +Run: `go test ./internal/mode/offscreen/ -run Brief` +Expected: FAIL — `assembleBrief` produces no history section. + +- [ ] **Step 7: Implement briefHistory** in `internal/mode/offscreen/collect.go` + +Add `"context"` is already imported; add `"fmt"` and `"time"` to imports. Add constants near the top: + +```go +const ( + recentProposals = 5 // how many recent proposals to show + outcomeScan = 50 // how many outcome events to scan for correlation + recentWindow = 7 * 24 * time.Hour +) +``` + +In `assembleBrief`, insert the history section after the tasks block (before goals): + +```go + if hist := m.briefHistory(); hist != "" { + b.WriteString("## Recently proposed\n") + b.WriteString(hist) + b.WriteString("\n") + } +``` + +Add the methods: + +```go +// briefHistory renders recent proposals with their outcome, newest first and +// within recentWindow. Returns "" when there is no memory or no recent history. +func (m *Mode) briefHistory() string { + if m.mem == nil { + return "" + } + ctx := context.Background() + proposals, err := m.mem.Recent(ctx, kindProposalMade, recentProposals) + if err != nil || len(proposals) == 0 { + return "" + } + taken := m.idSet(ctx, kindActionTaken) + dismissed := m.idSet(ctx, kindProposalDismissed) + now := m.now() + + var b strings.Builder + for _, ev := range proposals { + if now.Sub(ev.At) > recentWindow { + continue + } + id, _ := ev.Data["proposal_id"].(string) + action, _ := ev.Data["next_action"].(string) + outcome := "unactioned" + if taken[id] { + outcome = "confirmed" + } else if dismissed[id] { + outcome = "dismissed" + } + fmt.Fprintf(&b, "- %q (%s, %s)\n", action, outcome, relativeAge(now.Sub(ev.At))) + } + return b.String() +} + +// idSet reads recent events of a kind and returns their proposal_ids as a set. +func (m *Mode) idSet(ctx context.Context, kind string) map[string]bool { + out := map[string]bool{} + evs, err := m.mem.Recent(ctx, kind, outcomeScan) + if err != nil { + return out + } + for _, e := range evs { + if id, _ := e.Data["proposal_id"].(string); id != "" { + out[id] = true + } + } + return out +} + +// relativeAge renders a coarse human age for a duration. +func relativeAge(d time.Duration) string { + switch { + case d < time.Minute: + return "just now" + case d < time.Hour: + return fmt.Sprintf("%dm ago", int(d.Minutes())) + case d < 24*time.Hour: + return fmt.Sprintf("%dh ago", int(d.Hours())) + default: + return fmt.Sprintf("%dd ago", int(d.Hours()/24)) + } +} +``` + +- [ ] **Step 8: Run it to verify it passes** + +Run: `go test ./internal/mode/offscreen/` +Expected: PASS (history tests plus all earlier offscreen tests). + +- [ ] **Step 9: Commit** + +```bash +git add internal/ai/proposer.go internal/ai/proposer_test.go internal/mode/offscreen/collect.go internal/mode/offscreen/offscreen_test.go +git commit -m "feat(offscreen): read recent proposals into the brief; prompt follows up" +``` + +--- + +## Task 9: Full verification + live smoke + +**Files:** none (verification only) + +- [ ] **Step 1: Build, vet, and full test suite** + +Run: `go build ./... && go vet ./... && go test ./...` +Expected: all packages `ok`, exit 0. + +- [ ] **Step 2: Live end-to-end smoke (AW running)** + +Start the daemon: `go run ./cmd/keeld` +In the web UI (`http://localhost:7777`), start the off-screen mode, wait for a proposal, and **confirm** it. + +- [ ] **Step 3: Verify the event landed in AW** + +Run: `curl -s 'http://localhost:5600/api/0/buckets/keel.events/events?limit=10' | python3 -m json.tool` +Expected: at least a `proposal_made` and an `action_taken` event, each with `data._kind`, `data.proposal_id`, and `data.mode == "offscreen"`, sharing the same `proposal_id`. + +- [ ] **Step 4: Verify recall on the next run** + +Stop and restart `go run ./cmd/keeld`, start off-screen again, and confirm the next proposal is informed by history (it should avoid repeating the just-confirmed action, or follow up on a dismissed one). This is a qualitative check that the brief now carries `## Recently proposed`. + +- [ ] **Step 5: Final commit (if any verification fixups were needed)** + +```bash +git add -A +git commit -m "test: verify off-screen AW memory end to end" +``` + +(If Steps 1–4 pass with no changes, skip this commit.) + +--- + +## Done when + +- `go build ./... && go vet ./... && go test ./...` is green. +- The `keel.events` bucket exists and receives `proposal_made` / `action_taken` / `proposal_dismissed` with correlated `proposal_id`s. +- Off-screen's brief carries a `## Recently proposed` section when history exists, and proposals visibly avoid repetition / follow up. +- AW being down degrades to `nopStore` with a log line, and every off-screen command still succeeds (covered by the nil-memory tests).