# Agent Instructions: `mcp_servers/nostr_db_mcp`

> **Scope:** These instructions apply to files under `mcp_servers/nostr_db_mcp/`.

> **Maintenance rule:** NOT a changelog; if something stops being true, overwrite it, don't append history.

## Purpose

nostr_db_mcp

System: Nostr → relational DB (DuckDB) via an admin MCP. Single storage = DuckDB (the
Neo4j path was cut 2026-05-14 together with IngestDaemon, the NOSTR_BACKEND=neo4j branch in
mcp_admin.py, repair_stubs.py, duckdb/backfill.py, the neo4j/ dir and the neo4j requirement).

## Files (relative to mcp_servers/nostr_db_mcp)

- admin/mcp/mcp_admin.py — thin FastMCP/transport shell. Only bootstrap (faulthandler,
  logging, dotenv), an AdminCore instance, @mcp.tool / @mcp.resource handlers delegating 1:1
  to AdminCore, async notification fan-out (_flush_notifications drains app.notify, does
  send_resource_updated per subscriber, prunes dead sessions), uvicorn entry-point. NO
  ingest/validation logic here. Write-tools (update_graph_*, backfill_*) are commented out —
  agents get read-only: run_sql (DuckDB), relay_query (NIP-01 REQ, no ingest), npub/hex
  helpers. The ingest daemon starts as a background task in the MCP process (lifespan calls
  app.warmup() in a background thread so uvicorn binds the port before replay finishes).
- admin/mcp/admin_core.py — AdminCore + NotifyBroker. All testable admin logic OUTSIDE the
  transport layer:
    * lifecycle (boot = daemon.start; warmup = revive_related_ignored → replay_on_startup →
```text
  start_auto_tracker → start_mention_sweep; start = boot+warmup; stop = reverse).
  Idempotent, synchronous.
* tool-API (1:1 with the @mcp.tool handlers): identical string returns ("Subscribed.",
  "It is already subscribed.", "Queued for rescan.", …) and the same ValueErrors on bad
  input.
* resource builders returning Python objects (dict/list/str); JSON serialization is done
  by the shell.
* helpers decode_npub / encode_npub / normalize_pubkey as module-level (testable).
* read_run_sql_description() — loads the tool description from run_sql.md.
```

  NotifyBroker is MCP-agnostic: thread-safe request(uri) + drain() + per-URI
  subscribe/unsubscribe(uri, sink) with sink as an opaque handle (shell wires in
  ServerSession). diagnostics() returns counters for the status panel. The split lets
  behavior-spec tests run on a plain AdminCore(temp_path) WITHOUT FastMCP+uvicorn (10-20×
  faster, no module reload between tests). Shell keeps full compatibility, just delegates.
- admin/mcp/ingest.py — _RelayDaemonBase: WS lifecycle + REQ/EOSE orchestration. Storage
  hooks abstract; concrete DuckDB writing is in ingest_sql.IngestDaemonSQL. Plus shared
  utils (_normalize_relay_url, _prepare_tags, _PROFILE_FIELDS).
- admin/mcp/ingest_sql.py — IngestDaemonSQL (the only concrete storage impl). Writes directly
  to the DuckDB file; schema bootstrap via CREATE … IF NOT EXISTS (fresh file = empty tables,
  no seeding). Replaceable events via ON CONFLICT … WHERE excluded.created_at > …. NIP-65
  does DELETE + reinsert of the list in one transaction.
- admin/mcp/relay_query.py, admin/mcp/update_graph_with_events_*.py — orchestration helpers
  over _RelayDaemonBase.
- admin/mcp/live_subscription.py — _RelaySubMux + LiveSubscriptionPool. ONE WS and one reader
  thread PER RELAY, multiplexing many NIP-01 REQs with separate sub_ids (the relay unions the
  filters server-side). Previously each (pubkey, relay) pair had its own WS — at N primary ×
  3 productive_relays that was 3N constant threads/sockets, suspected of native-lib heap
  corruption under load (hence faulthandler.log). The mux scales with the number of
  productive relays, not subscriptions. Filters per role: primary = {authors} + {"#p"} under
  one sub_id; secondary = {authors} only. since per sub: _last_seen_either (primary, both
  directions) / _last_seen_for (secondary, authors-only). Auto-reconnect at mux level
  (1s→60s); each reconnect re-issues all current REQs. CLOSED from a relay is logged but the
  sub stays. These WS are SEPARATE from daemon.conns (which are short-lived under backfill
  REQ+CLOSE). Per-mux _lock guards _subs/_by_pubkey and each ws.send; the reader does NOT
  hold the lock during ws.recv. `since` lookup (_last_seen_* → _sql_lock) is DELIBERATELY
  outside mux._lock — otherwise nesting mux._lock → _sql_lock during a replay burst throttles
  pool.add to ~7/s and blocks the uvicorn bind. ws_connect uses ping_interval=None (the lib
  keepalive thread leaks on dropped connections); instead a RECV_IDLE_LIMIT_S=600s watchdog
  in the recv loop force-raises → reader reconnect. An empty mux (after remove_pubkey of the
  last sub) is torn down.
- admin/mcp/track_pubkey.py — run() (discovery + activate) and replay_on_startup()
  (re-subscribe from tracked_pubkeys). Two-phase discovery:
```text
PHASE 1 — probe ALL active relays. REQs per relay (set depends on role):
  * primary:   {authors:[hex], limit} + {"#p":[hex], limit} — both directions.
  * secondary: {authors:[hex], limit} only — skip #p (the global mention/reply domain
               dwarfs 1-hop frontier value).
Plus for BOTH roles an extra {authors:[hex], kinds:[0,3,10002]} with NO limit and NO
since — guarantees ingest of profile (NIP-01), follow list (NIP-02) and relay list
(NIP-65) even when those replaceable events are older than the limit=1000 cursor. Without
it the status panel and per-pubkey routing held NULLs for active users. _backfill_one
ingests events regardless of whether a relay advances to PHASE 2.
PHASE 2 — greedy weighted max-coverage. Iteratively pick the relay adding the most NEW
ids vs. `covered`; stop at PRODUCTIVE_RELAYS_TARGET=3 or when the best candidate adds
< MIN_MARGINAL_GAIN=1. Tie-breaks: larger total |ids|, then lex url (deterministic).
Deliberately prefers a relay with 5 unique events over one with 500 fully-covered — a
live sub on the former delivers data nobody else has.
```

  tracked_pubkeys is a dedicated DuckDB table — the primary/secondary registry with
  productive_relays, last_scan_at, role, limit_per_relay.
- admin/mcp/run_sql.md — the run_sql tool description (12-table schema with all columns, hard
  rules, CTE patterns, DuckDB gotchas). Loaded at mcp_admin.py startup AS the run_sql tool
  DESCRIPTION (_RUN_SQL_DESCRIPTION = Path(...)/run_sql.md) — the agent gets it via tools/list
  with no extra resource fetch. SOURCE OF TRUTH for the schema; a DB structure change = manual
  update of this file (deliberate, so the description carries semantics, not just names/types).
  Resource schema://nostr-duckdb is a separate auto-generated dump from information_schema
  (columns + types + indexes only), kept for the JOU client that subscribes to it.
- admin/mcp/mcp_admin.sh — start script (venv + pip + run)
- admin/mcp/.env — optional NOSTR_DUCKDB_PATH
- admin/mcp/requirements.txt — fastmcp + websockets + duckdb + pyarrow + bech32 + coincurve
  + dotenv
- docs/01.md, docs/02.md, docs/schema.md — historical design docs from the Neo4j-centric era.
  Out of date. Source of truth for the current state is this AGENTS.md and run_sql.md.
- duckdb/schema.sql — relational DDL, historical source. Unused now — ingest_sql.py has its
  own idempotent bootstrap. The operational brief for the LLM lives in run_sql.md.
- duckdb/nostr.duckdb — the DB file, ~55MB, git-ignored (duckdb/.gitignore: *.duckdb).
  Override the path via NOSTR_DUCKDB_PATH; default duckdb/nostr.duckdb (relative to
  mcp_admin.py).
- admin/mcp/tests/specs/ — living spec of the system. Each spec-test = one documented
  behavior; a failing test = a production bug or drift from the contract (run_sql.md / this
  file). Rules, fixtures, sync patterns in tests/specs/README.md. Full run ~20s.
  BLACK BOX: specs test INTENT via the public API (core.subscribe_to_pubkey, core.run_sql,
  frames on MockRelay). Asserting on internals (daemon._foo, pool._subs, bus._subs, …) is
  forbidden. If an effect isn't visible through the public API, ASK the user how the contract
  should look — don't improvise or work around it in the test. Files:
  test_primary_secondary[_extended].py (role lifecycle, frontier, transitions, discovery
  shape), test_event_storage[_extended].py (NIP-01 event → 12 DuckDB tables). Mock relay +
  helpers in tests/mock_relay.py and tests/specs/_helpers.py.
- sql_agent_system_prompt.txt — just the `nostr_sql` agent role (who it is, how to format
  answers). The full run_sql description is in the tool description (run_sql.md); do NOT
  duplicate it here.
- users.json — hex → {hex, npub, name} for known pubkeys (user-curated). Load lazily, only
  when you need a name for a pubkey or vice versa.

User (Czarek): npub14t68sepw5ydq9atxhftq5sjuul8mzkudd6em8x8hxggd9vg4snwsmxv60g
(hex aaf478642ea11a02f566ba560a425ce7cfb15b8d6eb3b398f73210d2b11584dd)

## Relay management

- Fresh install / nuke: ./init_db.sh (or python init_db.py). Creates the DB file if missing,
  builds the schema via the same _ensure_schema() as runtime, inserts DEFAULT_RELAYS (6
  mainstream productive relays) with active=true. Idempotent — a second run doesn't overwrite
  existing rows. --clean deletes the file first. Schema lives in ingest_sql.py:_BOOTSTRAP_DDL
  (source of truth), seed list in init_db.py:DEFAULT_RELAYS.
- The relay write-tool is commented out; a human edits the file directly post-start (e.g.
  INSERT INTO relays(url, active) VALUES (…, true) in a DuckDB session outside the MCP, while
  the MCP is off — it holds the write-lock). Status query:
  SELECT url, status, last_checked_at, last_error FROM relays.

The daemon (_RelayDaemonBase, task in the MCP process) reads active relays every 5s, opens/
closes WS, updates status / last_checked_at / last_error. MCP restart = daemon restart.

Dead-relay cooldown: relays.consecutive_failures (BIGINT, reset to 0 on each 'ok').
_persist_status increments on 'error', zeroes on 'ok'. Past RELAY_COOLDOWN_THRESHOLD=5 the
relay is NOT deactivated (active stays true) but reconcile skips it until
RELAY_COOLDOWN_INTERVAL_MS=1h passes since last_checked_at. One success resets the counter.
The cooldown gate is in _read_active_relays_due() (SQL filter); the _persist_status snapshot
is also filtered to due_urls (else every reconcile would refresh last_checked_at and the
window never elapses).

Discovery claim/persist CAS (tracked_pubkeys.last_scan_at): the auto-tracker does an ATOMIC
claim — SELECT pendings (last_scan_at IS NULL) + UPDATE to sentinel
_SCAN_IN_PROGRESS_SENTINEL = -1, both under one _sql_lock in _claim_pending_secondaries.
_persist_tracked_pubkey overwrites the row with a CAS: WHERE hex=? AND last_scan_at=-1. If
meanwhile rescan_pubkey / subscribe_to_pubkey / set_tracking_status forced last_scan_at=NULL
(deliberate rescan), persist matches 0 rows — the stale persist doesn't win, the rescan
survives. Anti-regression spec:
test_primary_secondary.py::test_secondary_promotion_retroactively_registers_existing_p_tags.

Role ownership (tracked_pubkeys.role): set_tracking_status (and its shim register_primary
used by subscribe_to_pubkey) is the ONLY place that sets/changes role.
_persist_tracked_pubkey after discovery updates ONLY scan-metadata (last_scan_at,
productive_relays, limit_per_relay); the existing role is untouched. (Previously persist had
monotonic CASE logic upgrading secondary→primary, which overwrote a deliberate
set_tracking_status issued DURING discovery.) Spec:
test_primary_secondary_extended.py::test_forward_frontier_respects_role_at_event_ingest_time.
The only case where persist creates a new row is when the row was deleted between claim and
persist — then INSERT with the role discovery used.

Lock convention (_sql_lock): IngestDaemonSQL._sql_lock is a threading.Lock() —
NON-REENTRANT. Any method that itself takes the lock via `with self._sql_lock:` (e.g.
_p_tag_pubkeys_for_author, _read_tracked_pubkeys, _record_discovery_outcome, …) MUST NOT be
called from code already holding it — instant deadlock. Pattern for callers
(track_pubkey.py etc.): read data via a public method first (it takes+releases its own lock),
THEN enter `with daemon._sql_lock:` and call only *_unlocked variants. Methods with the
_unlocked suffix require the caller to hold the lock. Past incident: track_pubkey.run once did
`with daemon._sql_lock: for target in daemon._p_tag_pubkeys_for_author(...)` which deadlocked
discovery for EVERY primary. Spec: tests/specs/test_discovery_queries_relay.py.

Event verification (NIP-01): every event, before the _ingest_event SQL lock, goes through
module-level _verify_event, doing two checks:
  1. id == sha256(canonical NIP-01) — canonical is JSON [0, pubkey, created_at, kind, tags,
```text
 content], separators (',', ':'), UTF-8, no ensure_ascii. Catches modified content/tags.
```

  2. BIP-340 schnorr-verify sig over id with pubkey via coincurve.PublicKeyXOnly
```text
 (libsecp256k1). Catches impersonation.
```

An invalid event does NOT enter events/tags/event_seen_on. Instead _record_fake_event writes
it to fake_events (received_at, relay_url, claimed_id, claimed_pubkey, reason, raw — reason ∈
{'malformed','id_mismatch','bad_sig'}) and increments relays.invalid_events_count.
fake_events is append-only, no PK (the same fake id from different relays = multiple rows;
forensics). No size limit yet — add retention if it grows.

Auto-disable on fake: INVALID_EVENT_DISABLE_THRESHOLD=1 — the first fake event from a relay
flips it to active=false. One hard rule; no transient-failure interpretation for a bad
signature. Enforced in _record_fake_event (one UPDATE … RETURNING right after the counter
bump). Log: `auto-disabled <url> — sent N invalid event(s) (threshold M)`. To restore later:
UPDATE relays SET active=true, invalid_events_count=0 WHERE url=... manually outside the MCP.

Dependency: coincurve>=19 (requirements.txt). mcp_admin.sh installs it; manual: pip install
-r requirements.txt. Hard fail at startup if the package lacks PublicKeyXOnly (ImportError at
top-level in ingest_sql.py).

Duplicate cache (RECENT_EVENT_IDS) runs BEFORE verification — a dup with a good id is already
trusted, no second schnorr-verify. The "substitute a fake event with a real event's id"
attack would need a sha256 collision — impossible.

Auto-disable of unproductive relays: relays has discovery_picks and discovery_rejections
(BIGINT DEFAULT 0). After each track_pubkey.run we call
IngestDaemonSQL._record_discovery_outcome(winners, losers):
  - winners = relays picked in PHASE 2 → discovery_picks += 1
  - losers  = relays that took part in PHASE 1 (returned a non-empty id-set) but weren't
```text
          picked → discovery_rejections += 1
```

Relays that failed / were skipped in probe do NOT count as losers. After the bump, one UPDATE
… RETURNING sets active=false for any relay with discovery_picks=0 AND discovery_rejections >=
DISCOVERY_REJECT_DISABLE_THRESHOLD=100 AND COALESCE(active,true). One pick ever immunizes a
relay. Live subs on a disabled relay aren't killed proactively — they live until the next
rescan_pubkey (same philosophy as cooldown).
Migration: _ensure_schema checks information_schema before ALTER TABLE. If the columns didn't
exist (legacy DB), backfill: discovery_picks = count of the URL in
tracked_pubkeys.productive_relays; discovery_rejections = DISCOVERY_REJECT_DISABLE_THRESHOLD
for each relay with discovery_picks=0, else 0. Never-picked relays land AT threshold (not
above) — auto-disable waits for the first real PHASE 1 rejection.

Scan mode (daemon._scan_mode, in-memory): runtime toggle that pauses active relay scanning
without stopping the daemon. Values: 'full-scan' (default on EVERY process start — the flag
is not persisted) and 'no-scan'. In 'no-scan' the _auto_track_loop and _mention_sweep_loop
short-circuit at each tick (continue after wait(INTERVAL)), so track_pubkey.run and
MentionSweep don't fire. Live subs and the reconcile loop keep running — events relays PUSH
still hit the DB. Tool set_scan_mode(mode) flips the flag; subscribe_to_pubkey /
rescan_pubkey issued during 'no-scan' stay at last_scan_at IS NULL and are picked up after
return to 'full-scan'. MCP restart = back to 'full-scan'.

Reconcile is parallelized: connect attempts go through
ThreadPoolExecutor(RECONCILE_CONNECT_WORKERS=32) OUTSIDE self.lock. The lock is held only for
self.conns mutations and reading the snapshot. Full reconcile pass = ~max(CONNECT_TIMEOUT_S=5s)
regardless of relay count.

Per-RelayConn lock (RelayConn.lock, RLock) serializes IO on a single socket. Without it,
parallel discoveries hit the same conn.ws and could seg-fault in libwebsockets on concurrent
close+recv. The lock covers _try_connect (writes ws), _close_ws (nulls ws), and all of
_backfill_one (REQ + recv loop + CLOSE). Cross-relay parallelism stays — each conn has its
own lock. Re-entrant so _mark_broken can call _close_ws from inside a locked _backfill_one.
Discovery (track_pubkey.run) filters urls to those with conn.status='ok' before PHASE 1.

## Tracked pubkeys — three-role model (primary / secondary / ignored)

tracked_pubkeys.role distinguishes:
  * primary   — explicitly subscribed via subscribe_to_pubkey. Both directions: discovery +
```text
            live sub with {authors:[hex]} + {"#p":[hex]}, all kinds, on 3 productive
            relays.
```

  * secondary — discovered from p-tags of primary events. {authors:[hex]} ONLY (discovery
```text
            and live sub). The 1-hop frontier doesn't need global mentions/replies/
            reactions p-tagging secondaries — dominant traffic volume, marginal graph
            value. ~10× fewer incoming events per secondary.
```

  * ignored   — tombstone ONLY for pubkeys with NO p-tag relation to current primaries. No
```text
            discovery, no live sub, skipped in replay. IF such a pubkey becomes related
            to a primary (forward: primary p-tags it, reverse: it p-tags a primary) it is
            auto-promoted to 'secondary' with last_scan_at=NULL — two paths:
              (a) LIVE — _register_secondary_unlocked promotes 'ignored'→'secondary' and
                  logs `frontier: revived ignored {hex8} → secondary`.
              (b) STARTUP — revive_related_ignored() runs once in _lifespan BEFORE
                  _replay_tracked_pubkeys. Scans events+tags: any ignored hex with an
                  existing bidirectional p-tag relation to any primary is promoted.
            set_tracking_status(hex, 'ignored') does NOT validate relatedness — the user
            may deliberately put a related one in 'ignored', but the first primary event
            (live) or MCP restart (startup) revives it.
```

### Frontier expansion in _ingest_event after commit is ASYMMETRIC

  * forward (no kind restriction) — when the author is primary, each p-tag value → secondary
```text
ON CONFLICT DO NOTHING. Includes the primary's kind-3 follow list — a deliberate
"who to follow" signal chosen by the user.
```

  * reverse (with blacklist REVERSE_FRONTIER_SKIP_KINDS) — when the author is NOT primary but
```text
a p-tag targets a primary (someone replies/mentions/reacts to a primary), the event
AUTHOR → secondary. Source: live-sub primary {"#p":[hex]} on productive_relays +
MentionSweep. Skip kinds = {3, 10000, 30000} — else a popular primary with 10k followers
would get 10k kind-3s with it in the p-tag. Blacklist (not whitelist) keeps future
interaction kinds default-on.
```

Secondary does not generate further secondaries — the _is_primary_unlocked gate ensures only
CURRENT primaries trigger reverse.

Cascade demotion (tracked_pubkeys_model._cascade_demote_dependents):
set_tracking_status(primary, 'secondary'|'ignored'), after flipping the role, finds all
secondaries whose p-tag relation (forward OR reverse) is EXCLUSIVELY through this primary and
marks them 'ignored' with last_scan_at=now_ms. Secondaries related to at least one OTHER
current primary are untouched (SQL NOT EXISTS over other_primaries). Live subs for cascaded
ones are dropped (live_subs.remove_pubkey); notification fan-out via tracked://list +
tracked://pubkey/{hex} per victim. No in-place revive — after re-promoting the demoted one
back to primary, the retro-scan in track_pubkey.run (_p_tag_pubkeys_for_author →
_register_secondary_unlocked) revives them from ignored to secondary. Spec:
test_primary_secondary_extended.py::test_primary_demotion_cascades_dependents_to_ignored_and_repromote_revives.

MentionSweep (IngestDaemonSQL.start_mention_sweep): background thread every
MENTION_SWEEP_INTERVAL_S=300s (5 min) selects all hexes from tracked_pubkeys WHERE
role='primary' and calls daemon.req_and_ingest({"#p": primaries, "since":
now-INTERVAL-OVERLAP, "limit": 100}). One REQ in parallel to each ok relay (REQ_FANOUT_WORKERS=8)
— catches mentions landing on relays not in any primary's productive_relays. Events ingested
normally; reverse frontier registers authors as secondary, auto-tracker gives them discovery.
`since` uses overlap (MENTION_SWEEP_OVERLAP_S=60s) so a missed tick re-pulls the recent window
(dups eaten by cache + idempotent SQL). Lifespan starts it after replay_on_startup with the
auto-tracker and stops it before that on shutdown.

## Key tools / lifecycle

- subscribe_to_pubkey(pubkey, limit=1000) — NON-BLOCKING. Calls daemon.register_primary(hex,
  limit) (one INSERT or UPDATE) and returns immediately. States:
    * no record → INSERT (role='primary', last_scan_at=NULL) → "Subscribed."
    * existing secondary → UPDATE role='primary', last_scan_at=NULL → "Subscribed."
    * existing primary → no-op → "It is already subscribed."
  Discovery + activate is fired by the auto-tracker (record has last_scan_at IS NULL) via
  track_pubkey.run. Promotion secondary→primary: register_primary leaves
  productive_relays/live_subs untouched; the auto-tracker's next iteration runs
  track_pubkey.run (pool.remove_pubkey → fresh discovery → fresh subs, brief no-sub window),
  plus the retro-scan registers already-collected p-tag pubkeys as secondary.
- rescan_pubkey(pubkey) — NON-BLOCKING manual re-discovery for an existing tracked pubkey.
  Sets last_scan_at = NULL (productive_relays stay — old live subs live until the auto-tracker
  actually starts discovery and calls pool.remove_pubkey). Picked up next tick (≤30s), goes
  through Phase 1 + Phase 2, persists fresh productive_relays, opens new live subs. Role
  preserved. Use when old productive_relays are suspected stale. Unknown pubkey → "Not
  tracked. Use subscribe_to_pubkey first."
- Auto-tracker (IngestDaemonSQL.start_auto_tracker): background thread every
  AUTO_TRACK_INTERVAL_S=30s fetches AUTO_TRACK_BATCH=5 rows with last_scan_at IS NULL and runs
  track_pubkey.run IN PARALLEL via ThreadPoolExecutor(workers=AUTO_TRACK_BATCH). Internal
  PROBE_WORKERS=8 × 5 concurrent discoveries = ~40 in-flight connect/REQ. Shutdown: submit +
  as_completed with a stop-check, shutdown(wait=False, cancel_futures=True). Success updates
  last_scan_at; failure leaves NULL → retry. Started after replay_on_startup, stopped before
  live_subs.stop_all.
- After MCP restart: replay_on_startup re-opens live subs on saved productive_relays for each
  record (any role, no re-discovery). Auto-tracker picks up last_scan_at IS NULL records.
- Shutdown lifespan: daemon.stop_auto_tracker() → daemon.live_subs.stop_all().

## Ingested-event dedup (CPU-saver)

- IngestDaemonSQL._recent_ids — FIFO 10K recently committed event_ids (deque + set, own
  lock). _ingest_event checks the cache first; cache-hit = return WITHOUT entering _sql_lock.
  So 70+ reader threads (main pool + live subs) don't jam on duplicates.
- Second line: _upsert_event_sql does SELECT 1 FROM events WHERE id=? AND author_hex IS NOT
  NULL — if fully ingested (not a stub from another event's e-tag), only INSERT event_seen_on
  ON CONFLICT DO NOTHING and return before DELETE/REINSERT of tags. Catches cache miss after
  eviction/restart.
- Trade-off: within the cache window (10K events), 2nd+ deliveries of the same id do NOT
  record a new event_seen_on — last_seen_per_relay is "best-effort recent", not an audit log.
- Cache is MARKED only after COMMIT in _ingest_event — a race on the same id resolves via SQL
  idempotency.

## Resources (push, JSON)

- tracked://list — snapshot of all tracked (hex, npub, role, added_at, last_scan_at,
  limit_per_relay, productive_relays, live_subs_active). Notified on ROSTER change (new
  primary/secondary, promotion, new productive_relays/last_scan_at). NOT per-event — kept
  cheap+stable. For counters → status://overview.
- status://overview — small aggregate for the status panel (counters only). Shape:
```text
{
  "tracked": {primary_pending, secondary_pending, live_observed},
  "relays":  {total, active, ok, failing},
  "events_total":      int,
  "events_per_minute": int,
  "db_size_mb":        float,
  "scan_mode":         str
}
```

  *_pending = last_scan_at IS NULL in that role; live_observed = pubkeys with ≥1 active live
  sub. relays: active = COALESCE(active,true), ok = active AND status='ok', failing =
  active − ok. events_total = count(*) FROM events; events_per_minute = fresh ingests in the
  trailing 60s (rolling deque _ingest_times, push after each successful COMMIT). db_size_mb =
  os.path.getsize(self._db_path)/1MB (1 dp; 0 on stat fail). Notified (a) with tracked://list
  (fan-out in _notify), (b) after _persist_status (via internal URI relays://status).
- tracked://pubkey/{hex} — detailed stats: role, added_at, last_scan_at, productive_relays,
  profile (kind 0), events.{authored, referencing, kinds_top}, last_seen_per_relay,
  secondaries_from_me (primary only). Accepts hex and npub1... in the URI. Notified on every
  event touching the pubkey (author or p-tag), debounced by _flush_notifications
  (NOTIFY_FLUSH_INTERVAL_S=2s). pubkey_stats runs via self._conn.cursor() (DBAPI cursor)
  OUTSIDE _sql_lock — reads concurrently with ingest/discovery writes (else after each
  rescan_pubkey the UI panel timed out on read_resource). Cost: no cross-query consistency
  guarantee (acceptable for UI stats).

## Push mechanism

The daemon has notify_callback: Callable[[str], None] — called from ingest threads (LiveSub
workers, FastMCP threads) with the changed URI. In mcp_admin.py the callback is
_request_notify (thread-safe set + lock). Async task _flush_notifications (in lifespan) every
NOTIFY_FLUSH_INTERVAL_S drains the queue and sends resources/updated to each subscriber via
_subskrybenci (same path as schema://nostr-duckdb). Coalescing per URI: many requests on the
same URI within one tick = one notification.

## Logs / debugging

File: admin/mcp/mcp_admin.log (created by mcp_admin.py, append across restarts, line-buffered).
Console gets INFO+, file DEBUG+. Format: `2026-05-03 15:27:01,243 logger LEVEL message`;
logger typically mcp_admin, ingest_sql, track_pubkey, live_subscription.

Markers for "secondary treated as primary" / frontier explosion:
- `register_primary {hex8}: status=subscribed_new|promoted|already` — user path (tool
  subscribe_to_pubkey). subscribed_new/promoted for a pubkey that SHOULD be secondary =
  someone deliberately changed its role. already = no-op.
- `frontier: primary {a8} → secondary {b8} (NEW)` — _ingest_event saw a primary-authored
  event with a p-tag on b (not yet tracked) → INSERT secondary. Absence = expansion didn't
  run (author not primary, or target already tracked).
- `auto-tracker tick: claimed N row(s): {hex8(role), …}` — the thread took N last_scan_at IS
  NULL records. If a should-be-secondary shows as (primary) — the DB has a broken role.
- `auto-tracker pick {hex8}: role=…` — next record enters discovery; role= is what goes to
  track_pubkey.run and determines the retroactive p-tag expansion.
- `track_pubkey.run {hex8}: role=…` — discovery entry, should match the prior pick line.
- `persist {hex8}: requested=X final=Y relays=N limit=L` — after the UPSERT. requested=what we
  passed, final=role after SQL CASE. requested=secondary final=primary is CORRECT (no-op
  promotion). requested=primary final=secondary would be a bug.
- `discovery {hex6}: pick #N {url} (+G new, total=T, covered=C)` — greedy max-cover picks the
  next relay in PHASE 2.

### Typical sequences

### Primary subscribe

```text
register_primary {h}: status=subscribed_new
auto-tracker tick: claimed 1 row(s): {h}(primary)
auto-tracker pick {h}: role=primary
track_pubkey.run {h}: role=primary
discovery {h6}: pick #1 …
persist {h}: requested=primary final=primary
```

### Frontier expansion

```text
frontier: primary {a} → secondary {b1} (NEW)
[after 30s] auto-tracker tick: claimed …: {b1}(secondary), {b2}(secondary)
track_pubkey.run {b1}: role=secondary
persist {b1}: requested=secondary final=secondary
```

  Promotion secondary→primary:
```text
register_primary {h}: status=promoted
auto-tracker pick {h}: role=primary  ← last_scan_at reset, role primary
```

If the auto-tracker picks someone with role=primary when they should be secondary, the DB
state is broken (SELECT role, COUNT(*) FROM tracked_pubkeys GROUP BY role) — trace who pushed
them as primary (register_primary status=subscribed_new or persist requested=primary).

Data model (abbrev of 01.md §3-4 / docs/schema.md): Event, Pubkey, Relay, Profile,
Addressable, Tag, TagElement. Relations: AUTHORED_BY (events.author_hex), SEEN_ON
(event_seen_on), HAS_TAG (tags.event_id), HAS_ELEMENT (tag_elements.tag_id), REFERS_TO
(tags.ref_type/ref_value), HAS_PROFILE (profiles.pubkey), PUBLISHES_TO/READS_FROM (NIP-65,
pubkey_publishes_to / pubkey_reads_from).

Source of truth: DuckDB. Ingest writes directly to the file; the agent reads via run_sql
(read-write conn on the MCP side, writes blocked app-level by a whitelist — only
SELECT/WITH/SHOW/PRAGMA/EXPLAIN/DESCRIBE/EXECUTE + CREATE [OR REPLACE] TEMP {TABLE|VIEW|MACRO}).

Filtering on an unknown dimension (category, kind, tag, name): first SELECT … GROUP BY …
ORDER BY count(*) DESC LIMIT 50 on that column to see the real value vocabulary — then filter.
Zero results for a question that intuitively should return something is almost always a query
bug, not a fact about the data.

Typical flow (with write-tools enabled — commented out by default):
1. add a relay to the relays table (with the MCP off)
2. start the MCP, wait ~5s for the first daemon reconcile
3. check SELECT url, status FROM relays
4. ingest via update_graph_with_events_authored_by (or _following_of, _outbox,
   _matching_filter) — requires uncommenting @mcp.tool()
5. analytical queries via run_sql

## Unknowns (blind spots — fill in when you hit them)

- Agent registry conf/agenty.json — jou/uruchom.py reads ../../conf/agenty.json and
  ../../conf/agent0.json, but the conf/ dir isn't in the repo. Where does it live on disk?
- sql_agent_system_prompt.txt — the file exists but grep finds no reference in code. Who
  loads it into the agent system_prompt?
- Agent nostr_sql — shows as [nostr_sql] in chat, but which MCP server provides it is unknown.
- Gateway MCP — uruchom.py connects to gateway_url and calls list_mcp_servers. Where does the
  gateway process live?
- run_python sandbox — code exists, tool commented out. Isolation mechanism unknown.
