# AGENTS.md

## Scope

These instructions apply to everything under `mcp_servers/user_mcp/`.

This component is the human-facing MCP server. It is the bridge between the agent and the user's browser:

- the agent sends HTML and JavaScript views through `present_data`, either as a new screen or as a replacement for a screen the user already has;
- the browser returns recorded speech plus one entry per event: the element's XPath (or the `__scrolled__` sentinel for a screen the user merely arrived at), the hash of the screen it happened on, and the moment in seconds from the start of the recording;
- `what_next` waits for the resulting transcription and returns it to the agent, together with the content and identity of every screen the user passed through;
- utility tools expose current date/time, shared-drive access, MCP reconnection, and local process control.

Follow the repository-level `AGENTS.md` first. This file adds rules specific to `user_mcp`.

## Architectural Role

`user_mcp` is the project's “speak, show, and listen” layer.

It runs three cooperating interfaces in one process tree:

1. **FastMCP server**
   - Defined in `user_mcp.py`.
   - Exposes tools used by the host agent.
   - The MCP server name is `user`.

2. **Browser WebSocket bridge**
   - Defined in `gui_provider.py`.
   - Listens on `ws://localhost:8085/gui-socket/<pin>`.
   - Broadcasts rendered HTML, JavaScript, and a stable page hash to connected clients.

3. **Speech upload HTTP endpoint**
   - Defined in `lib/speech_to_text/server.py`.
   - Listens on `0.0.0.0:8084`.
   - Accepts `POST /backendSrv02/<pin>` with an audio file, an optional `hash` form field, and three repeated form fields describing the clicks (`list[str]` each).

The three click fields are strictly parallel — index `i` of each describes the same click, in click order:

| Field | Meaning |
|---|---|
| `xpath` | XPath of the clicked element |
| `xpath_hash` | page hash of the screen it was clicked on |
| `xpath_time` | seconds from the start of the recording, one decimal (`"3.4"`) |

`hash` is separate: it is the screen visible when the recording was sent, used as the displayed view when nothing was selected and as the fallback screen for an older page that does not send `xpath_hash`.

Do not merge these interfaces casually. They have different protocols, lifecycle behavior, and clients.

## Access PIN

Both browser-facing endpoints live under a four-digit PIN as their last path segment. The browser does not know the PIN: `www/pin-gate.js` collects it on an on-screen keypad at start-up and composes `uploadUrl` and `socketUrl` from it, then `index.html` publishes them as `window.pinEndpoints` for `www/script.js`. The server registers only its own PIN's paths, so a wrong PIN is a non-existent address rather than a rejected request, and an outsider who opens the page can neither see presented views nor send a recording.

Rules:

- The PIN is a secret, handled exactly like the OpenAI key: `access-pin=${USER_MCP_PIN}` in `lib/speech_to_text/config.properties`, value in `/root/.secrets/user_mcp.env`, loaded into the environment by `run.sh`. Never commit the digits. To change the PIN, edit the secrets file only.
- Both servers read it through `SimpleConfig.get_string("access-pin")` — `lib/speech_to_text/server.py` at `create_app`, `gui_provider.py` at import time, where the route decorator is evaluated. A missing `USER_MCP_PIN` therefore aborts start-up with `AppException: Brak zmiennej środowiskowej: USER_MCP_PIN`, which is the intended behavior; do not add a default.
- Keep the PIN as an extra path segment under the existing `/backendSrv02` and `/gui-socket` prefixes, so the Apache `ProxyPass` rules keep matching without a vhost change.
- Never hard-code the PIN in `www/`. Everything there must derive from the digits the user typed.
- A GUI socket that closes without ever opening means a wrong PIN. `index.html` re-asks through `pinGate.reject()` instead of entering the reconnect loop, which would retry a dead address forever.
- `script.js` sends nothing when `window.pinEndpoints` is unset; do not add a default upload URL as a fallback.
- The PIN travels in the URL, so it lands in Apache and uvicorn access logs. Treat it as a gate against casual outsiders, not as a credential.

## Main Data Flow

### Presenting a view

1. The agent calls `present_data(html_data, js_data)`, optionally with `replaces`.
2. The view is assigned a SHA-1 hash derived from `html + NUL + js`.
3. `user_mcp.py` stores the view in `lib/last_view.py`.
4. `gui_provider.py` broadcasts:
   - `html`,
   - `js`,
   - `hash`,
   - `replaces` — the hash of the page this content takes over, or `null`.
5. The browser renders the page and retains the hash for later speech submissions.

The hash is computed server-side and travels with the content. **The browser never computes it**: it stores whatever arrived and sends the same string back. Treat it as an opaque page id on the client side.

### Replacing a view in place

`present_data(replaces=<hash>)` means "this is new content for that page". The browser swaps the page's content and id where it sits: no new carousel slide, no ride, no lost scroll position, and the carousel index does not move. Without `replaces` the content enters as a new page and the carousel rides to it.

Rules that keep this correct:

- **A replaced view is not overwritten, it is superseded.** `store` keeps the old HTML and records `old hash → new hash`. A recording made a second before the replacement is still in the transcription queue and its XPaths belong to the old tree; overwriting the content under the old id would resolve them against the new tree and silently mark the wrong element. This is the reason page identity is derived from content.
- **The agent targets the view it is changing**, using the hash it received in the prompt or from a previous `present_data` result. It cannot know the resulting hash — the server computes it — so `present_data` returns the new hash for chaining.
- An unknown `replaces` is a tool-level error and nothing is shown. A silent fallback to a new page would leave the user with a second card instead of a corrected one.
- The server follows the replacement chain: targeting a hash that has since been replaced redirects to the current one, because that is the screen the user actually has.
- One spoken command may touch several screens. The agent then calls `present_data` once per screen, each with its own `replaces`; no call moves the user.

### Receiving the next user instruction

1. The browser records audio.
2. It posts the audio to `/backendSrv02/<pin>`, with:
   - zero, one, or many entries, each carrying its XPath, screen hash, and moment;
   - the page hash of the view visible when the recording was sent.
3. `NextPromptManager` queues the audio.
4. A worker transcribes it through `SpeechToText`.
5. `lib/user_selection.py` groups the entries by screen hash and inserts a `USER_CLICKED_HERE` marker, carrying the moment, before every clicked element in that screen's stored HTML. Within one screen all XPaths are resolved before the tree is modified, so one bad XPath fails the whole selection instead of leaving half the markers.
6. The completed prompt is placed on the prompt queue.
7. `what_next` blocks until that prompt is available.

Preserve the per-entry page-hash correlation. The record button docks in `#lockZone` and keeps recording, so the user can swipe the carousel to another screen mid-recording and keep clicking. One recording therefore spans several views, and a click only means something against the view it was made on.

Preserve the shared time axis too. Every moment is measured from the same zero — the start of the recording, which is also `t=0` of the uploaded audio — so the agent can line a click up with what was being said at that second.

### Screens the user only looked at

Arriving at a screen travels in the **same three parallel lists** as clicks, with `SCROLLED_XPATH` (`__scrolled__`) in place of the XPath. The browser writes one such entry at the start of every recording and one whenever a different page comes up in front of the user. There is deliberately no separate timeline field, no duration field, and no fourth list.

Why it matters: a command can be about a screen the user never touched ("and on the previous screen fix the amount"). Reusing the click triple means those screens automatically get their content into the prompt, in visit order, on the same time axis — `user_selection` already groups per screen hash.

Rules:

- The sentinel must never reach the XPath evaluator. It is intentionally not a valid XPath (real ones start with `//`, `.` or `./`), so it cannot collide with a selection or accidentally match an element.
- A screen with at least one click renders as `<user_selection>`; a screen that was only visited renders as `<user_view>`. Both carry `hash`, and visited ones also carry `seen_at`.
- Consecutive entries for the same screen are suppressed in the browser, but **returning** to a screen is recorded again — coming back is information.
- Screen changes are recorded regardless of who caused them (a user swipe or a view the agent pushed mid-recording). What matters is which screen was in front of the user in which second.
- Visits are recorded only while `mediaRecorder.state === 'recording'`. Swiping outside a recording is not part of any command.

## Entry Points and Commands

Run the component from its own directory:

```bash
./run.sh
```

`run.sh`:

- changes to its own directory;
- loads secrets from `/root/.secrets/user_mcp.env` and **aborts if the file is missing**;
- creates `.venv` when missing;
- installs `requirements.txt`;
- runs `.venv/bin/python user_mcp.py`.

Start it through `run.sh`, not with a bare `python user_mcp.py`. Without the secrets file `USER_MCP_PIN` is unset and start-up aborts with `AppException: Brak zmiennej środowiskowej`; an unset `OPENAI_API_KEY` fails the same way at the first transcription.

Do not run this component with the system Python. Use its local `.venv`.

Automated tests:

```bash
.venv/bin/python -m pytest tests          # everything
.venv/bin/python -m pytest tests/www -v   # browser state machine and carousel, per case
```

`tests/test.py` is not part of that suite. It is a manual runner that starts the prompt manager and blocks waiting for an uploaded recording.

## Important Files

```text
user_mcp/
├── AGENTS.md
├── README.md
├── user_mcp.py
├── gui_provider.py
├── request_handler.py
├── run.sh
├── requirements.txt
├── requirements-dev.txt
├── autoenter.sh
├── install-mcp.sh
├── lib/
│   ├── last_view.py
│   ├── shared_drive.py
│   ├── user_selection.py
│   └── speech_to_text/
│       ├── config.py
│       ├── config.properties
│       ├── next_prompt_manager.py
│       ├── prompt_wait.py
│       ├── server.py
│       └── speech_to_text.py
├── tests/
│   ├── test.py                  manual runner, not pytest
│   ├── test_prompt_wait.py
│   ├── test_last_view.py
│   ├── test_user_selection.py
│   └── www/
│       ├── harness_runner.py        wspólne uruchamianie harnessów w node
│       ├── test_record_button.py    pytest wrapper
│       ├── test_carousel.py         pytest wrapper
│       ├── harness.js               jsdom run of www/script.js
│       ├── carousel-harness.js      jsdom run of www/page-carousel.js
│       └── package.json             jsdom
├── docs/
│   ├── diagram.drawio
│   └── diagram.png
└── www/
    ├── index.html
    ├── script.js
    ├── page-carousel.js
    ├── pin-gate.js               klawiatura PIN-u i adresy złożone z PIN-u
    ├── pin-gate.css
    ├── style.css
    ├── sw.js
    └── gui/
```

### Source-of-truth notes

- `user_mcp.py` is the current MCP entry point.
- `lib/speech_to_text/server.py` is the active speech-upload server used by `NextPromptManager`.
- Treat the top-level `request_handler.py` as legacy or experimental unless a caller is explicitly confirmed. It contains a separate upload implementation and hard-coded filesystem paths.
- Do not commit generated logs, `.pyc` files, `__pycache__`, temporary uploads, or local virtual environments.

## MCP Tool Contracts

### `present_data(html_data, js_data="", replaces="")`

This is the primary output mechanism for user-visible results.

Rules:

- Pass HTML and JavaScript separately.
- Do not embed `<script>` tags in `html_data`.
- Keep `js_data` as executable JavaScript only.
- Ensure the view works on a phone-sized screen.
- Keep views flat and fully visible by default, and keep that rule in the docstring. **No content hidden behind an interaction** — no accordions or `<details>`, tabs, in-view carousels, `<select>`/dropdowns, modals or tooltips carrying content. Anything richer is opt-in: the user has to ask for it.
- Preserve element IDs and DOM structure when they are used by browser code or XPath selection.
- Store the view before broadcasting it.
- Return a useful status even when no browser client is connected.
- Never assume that only one browser client is connected.
- **Always return the new page hash.** It is the agent's only way to chain a second change onto a view it just rendered.
- A replacement is a **complete** view, never a patch: full `html_data` and `js_data` again. `last_view` stores only HTML, and every view resets its JS anyway, so there is no partial-update path to support. `replaces` is an identity hint, not a diff protocol.
- Validate `replaces` before broadcasting. An unknown hash must not reach the browser.
- The tool docstring is the only place where the agent learns when to replace instead of adding. It is part of the agent contract — keep the decision rule there, and keep it in Polish.

Changes to the payload shape require coordinated changes in:

- `gui_provider.py`;
- `www/script.js`;
- `www/page-carousel.js`;
- any alternate GUI client under `www/gui/`.

### `what_next(last_operation)`

This tool waits for the user. The wait is bounded only so that the host cannot abort the call.

Rules:

- `last_operation` should briefly report the result of the previous action.
- The wait length is `what-next-timeout` in `lib/speech_to_text/config.properties`, currently 2 000 000 seconds (~23 days). The intent is that it never fires: the user may come back days later. Keep it **below** the host's `MCP_TOOL_TIMEOUT`, whose ceiling is 2147483647 ms (~24.8 days, the int32 range of Node's `setTimeout`); past that limit the host kills the call and nobody re-enters the wait.
- On timeout, preserve the continuation behavior rather than returning a false user instruction. It is the last resort, not a routine cycle — the agent calls `what_next` again instead of the process hanging until restart. Do not replace it with an unbounded `queue.get`.
- The queue is polled through `lib/speech_to_text/prompt_wait.py`. `what_next` is a coroutine, so a blocking `queue.get` would freeze the MCP event loop for the whole wait and make the call uncancellable. Keep the wait non-blocking, and keep taking items with `get_nowait`: a cancelled wait must not consume a prompt nobody will read.
- `NextPromptManager.next_prompt` stays blocking for callers outside an event loop, such as `tests/test.py`.
- Preserve `UserSelectionMismatchError` handling so stale or invalid XPath data does not silently point to the wrong element.
- The returned prompt currently reminds the agent to call `what_next` again after completing the operation. Treat that suffix as part of the agent-loop contract.

### `get_current_date()`

Returns local server date and time as:

```text
YYYY-MM-DD HH:MM:SS
```

Do not silently change timezone semantics or output format without checking consumers.

### `shared_drive_get_last_uploads(count)`

Returns a JSON-encoded list of filenames, newest first.

Before changing this tool, verify whether shared-drive ownership should remain in `user_mcp`; the README already indicates that this responsibility may be moved elsewhere.

### `reconnect_mcp(server_name)`

This tool controls a `tmux` session named `claude` and pane `claude:0.0`.

It schedules:

1. `/mcp disable <server>`;
2. `/mcp enable <server>`;
3. a textual continuation message.

Critical rules:

- Sanitize line breaks in `server_name`.
- Do not make the tool wait synchronously for the restart.
- Preserve its post-call contract: after calling it, the agent must not invoke another tool until a new textual instruction arrives.
- Treat the hard-coded tmux session and pane as an environment dependency, not a portable default.

### `autoenter_off()`

Kills processes matching `autoenter.sh`.

Be careful when changing the `pkill -f` pattern. A broader pattern can terminate unrelated processes.

### `wiki_dodaj_strone(title, content)`

This imports `wiki` lazily at call time.

The module is not declared in this component's visible dependency list. Verify the runtime environment before relying on this tool, and consider moving it to the component that owns wiki integration.

## Browser Recording UI

`www/script.js` drives the red record button. It is a three-state machine; know it before touching that file.

| State | Entered by | Ends by |
|---|---|---|
| idle | — | pressing the button |
| held | press and hold | release |
| locked | releasing over `#lockZone` (🔒, right) | pressing the ball again, then releasing anywhere |

While recording, two zones appear beside the button: `#cancelZone` (❌, left) discards the recording, `#lockZone` (🔒, right) keeps it running after the finger is lifted. Both stay reachable in the locked state, so a locked recording can still be cancelled.

Selections reach the backend as three parallel lists — `selectedXPaths`, `selectionHashes`, `selectionTimes` — filled from exactly three places:

- dropping the ball on an element, through `captureSelectionAt(x, y)`;
- clicking an element while locked, also through `captureSelectionAt(x, y)` — this is how the user marks several elements, on several screens, in one recording;
- arriving at a screen, through `recordScreenVisit(hash)` — once at recording start and once per live-page change, with `SCROLLED_XPATH` instead of an XPath.

Both appenders write to all three lists at once. For a click the hash comes from `window.pageCarousel.currentHash` and the moment from `selectionOffsetSeconds()`, both read at click time; for a visit the hash comes from the carousel callback.

`recordScreenVisit` skips an entry whose hash equals `lastVisitHash`, so a re-reported identical page adds nothing. `lastVisitHash` resets with the lists in `startRecording`, otherwise the second recording would suppress its own starting screen.

Rules when editing this file:

- Keep the three lists appended in lockstep. The backend pairs them by index and drops the metadata entirely if the counts disagree, so a one-sided `push` silently loses screen and timing data.
- The sentinel value must stay identical in three places: `www/script.js`, `lib/user_selection.py`, and `tests/www/harness.js`.
- Selections are additive only. There is no deselect, and repeating an element is **not** a no-op — every click is its own entry, because returning to an element seconds later is information for the agent.
- `recordingStartedAt` is the zero of the time axis and is set immediately before `mediaRecorder.start()`, so it matches `t=0` of the uploaded audio. Do not move it into an event handler that may fire late.
- Only the live (centre) carousel slide can be selected: neighbours have their `id`s stripped and sit outside `#mainContent`, so `getXPathForElement` returns `null` for them. This is what guarantees the XPath and the hash describe the same DOM — do not weaken either side.
- A mouse swipe of the carousel ends with a `click` event (touch suppresses it via `preventDefault` in `touchmove`), so the click handler ignores it via `pageCarousel.justSwiped`. Without that guard, changing screens selects whatever sits under the cursor.
- Zone highlighting uses the `.is-over` / `.is-locked` classes, not inline styles. Inline styles survive the zone being hidden and leak into the next recording.
- `homeLeft` / `homeTop` hold the button's resting position and must not be overwritten by a gesture that starts from the lock zone.
- A gesture must start on the button (`gestureActive`), otherwise a stray click would end a locked recording.
- `mediaRecorder.onstop` fires **after** `finishRecording()` returns, so the selection list is complete by send time. Any test double must keep that ordering asynchronous.

`www/` is served directly from the working tree by Apache (`mcpforum.pl`), so edits there are live immediately. There is no deploy script. `www/sw.js` only calls `skipWaiting`/`clients.claim` and has no `fetch` handler, so there is no asset cache to invalidate.

## Page Carousel

`www/page-carousel.js` keeps every received page and lets the user swipe between them. Only the centre slide is live; neighbours are inert previews with their `id`s stripped.

Two things enter through `push`, both serialized by the same queue so they cannot land mid-ride (`drain()` returns early while `isAnimating`):

- `{html, js, hash}` — a new page, inserted after the current one, carousel rides to it. A hash already in the carousel scrolls to that page instead of duplicating it.
- `{html, js, hash, replaces}` — new content for the page with hash `replaces`, handled by `replaceAt`. The index does not move, there is no ride, `scrollTop` is preserved, and the page's id becomes the new hash.

Rules:

- `replaceAt` re-renders **only** when the replaced page is the live one. Neighbours are inert previews, so running their JS there would be wrong; `runJs` in `index.html` also clears all active intervals globally.
- Replacing the live page resets its JS state. That follows from the `present_data` contract (every view is complete and self-contained) — don't try to preserve `window.__view` across a replacement.
- If `replaces` names a page the carousel does not have (a browser reloaded after the agent learned the hash), the content enters as a new page and a warning goes to the console. This is a documented best-effort exception: the browser has no channel to report the problem back to the agent, and not showing the content at all would be worse. The server-side check catches the ordinary case first.
- Two pages must never share a hash. Same hash means same content, so `replaceAt` collapses a duplicate into one page and fixes up `index` — otherwise the hash stops identifying a page and a later replacement could hit the copy the user is not looking at.
- `onLiveChange` fires when a different page comes up in front of the user, from `showLive` and from a replacement of the live page. It exists for the voice recorder; the carousel itself knows nothing about recording.

## View State and User Selection

`lib/last_view.py` is an in-memory store:

- views are keyed by a stable SHA-1 page hash;
- only HTML is retained for selection reconstruction;
- `_replacements` maps an old page hash to the one that superseded it;
- `_MAX_VIEWS` limits both retained views and retained replacement links to 500;
- state is lost on process restart;
- access is protected by a thread lock.

When editing this code:

- keep hash generation identical between storage and WebSocket delivery;
- do not use Python's built-in `hash()`, because it is process-dependent;
- preserve thread safety;
- do not silently fall back to an unrelated view when a supplied page hash is known to be wrong;
- consider memory and privacy implications before storing JavaScript or user data;
- never overwrite a stored view in place — a replacement stores new content under a new hash and only records the link, so a recording still in the transcription queue keeps resolving against the tree it was made on;
- `current_hash` walks the replacement chain and must stay cycle-safe: an agent that restores earlier content produces the earlier hash again and closes the chain into a loop.

`lib/user_selection.py` parses stored HTML with `lxml` and evaluates browser-supplied XPath.

It emits **one block per screen**, in order of the first entry for that screen:

- `<user_selection hash="..." seen_at="..." replaced_by="...">` — the user marked something here; one marker per click:

  ```text
  <!-- USER_CLICKED_HERE @ 3.4s -->
  ```

- `<user_view hash="..." seen_at="..." replaced_by="...">` — the user only had this screen in front of them.

Build the marker through `marker_text()` / `marker_html()` rather than by hand. The number is seconds from the start of the recording; `MARKER_TEXT` / `MARKER_HTML` remain the timeless form used when the browser sent no moment.

Hidden content is why presented views must stay flat. Reconstruction returns the screen's **whole** stored HTML, so anything collapsed behind an interaction reaches the agent as if the user had seen it, while a click can only ever land on what was actually on screen. A collapsed section also cannot survive a replacement, because every render resets the view's JS.

The attributes are what let the agent act on a screen, so they are part of the contract, not decoration: `hash` is what goes into `present_data(replaces=...)`, `seen_at` places the screen on the recording's time axis, and `replaced_by` says the screen has since been superseded and the successor is the one to change. The header names the hints, so a header change and an attribute change go together.

Security and correctness rules:

- treat XPath, screen hash, and moment as untrusted input;
- keep failures explicit through `UserSelectionMismatchError`;
- never execute JavaScript while reconstructing a selection;
- accept a single XPath, a list, or `None`; `_normalize_clicks` drops blanks and the record button, but **keeps repeats** — each is a distinct timed entry;
- preserve the special handling of `//*[@id="recordBtn"]`, which means “no meaningful selection”;
- keep `SCROLLED_XPATH` out of `_resolve`; it is a marker of presence, not a locator, and feeding it to the XPath evaluator would fail the whole prompt;
- filtering an entry must drop its hash and moment with it; a shifted index pins a moment onto the wrong element;
- an explicitly supplied screen hash that is no longer stored raises **when something was clicked on that screen** — a click reconstructed against a different view would put markers in the wrong tree. Only an entry with no hash at all falls back to the shared `hash`, and then to the last view;
- insert markers in click order, so two markers on one element read chronologically;
- do not mutate the stored original HTML;
- add the marker only to the reconstructed copy returned to the agent;
- expose the page hash even when nothing was selected — without it the agent has no target to replace in the most common case (the user talks and clicks nothing).

Documented best-effort exceptions to the repository's explicit-failure rule:

- If the number of `xpath_hash` or `xpath_time` values disagrees with the number of XPaths, that metadata is dropped for the whole request and a warning is logged. The lists are produced in lockstep, so a mismatch is a client bug, and guessing an alignment would attach a moment or a screen to the wrong element. Losing the metadata is recoverable; a wrong pairing silently misleads the agent. The utterance itself is never discarded for this.
- A screen that was **only visited** and whose HTML is no longer stored is skipped instead of raising, and the skipped hashes are named in the returned block. There is nothing to attach to a wrong tree, and losing the whole utterance over a screen the user merely passed through would be a worse trade. The note is mandatory: silence would read as "these were all the screens".

## Speech-to-Text Pipeline

`NextPromptManager` starts two daemon threads during module initialization:

- an audio-processing worker;
- the FastAPI upload server.

This means importing `next_prompt_manager` has side effects.

When changing this area:

- avoid importing it from tests that are not prepared to start the HTTP server;
- keep queue item types explicit:
  - audio queue: `(Path, xpaths, page_hash, xpath_hashes, xpath_times)` — see `AudioItem`;
  - prompt queue: `str | UserSelectionMismatchError`;
- always remove temporary audio in a `finally` block;
- do not block the upload endpoint while transcription runs;
- ensure concurrent uploads receive unique filenames;
- log transcription failures without placing fabricated text on the prompt queue.

Check `lib/speech_to_text/config.py` and `config.properties` before changing paths, model configuration, or external speech-service behavior.

`config.properties` values support `${VAR}` interpolation from the environment, and a missing variable raises `AppException` instead of resolving to an empty string. Both secrets live there as references — `gpt-key=${OPENAI_API_KEY}` and `access-pin=${USER_MCP_PIN}` — while the values themselves are in `/root/.secrets/user_mcp.env` (root-only, outside the repository). Never put a literal key or PIN back into the file.

## Networking and Runtime Dependencies

Expected local ports:

| Port | Protocol | Purpose |
|---:|---|---|
| 8084 | HTTP | Speech upload endpoint |
| 8085 | WebSocket | GUI broadcast channel |

Additional environment assumptions include:

- a browser or webview running the files under `www/`;
- write access to speech-to-text temporary and log directories;
- shared-drive path `/var/www/html/uploads`;
- `tmux` with a session named `claude` for MCP restart;
- any external binaries, models, credentials, or services required by `SpeechToText`;
- an MCP host configured to launch this server.

Do not hide these dependencies behind broad exception handling. Prefer a clear startup or tool-level error.

## Dependency Management

Dependencies are currently listed without version pins:

```text
fastapi
pydantic
fastmcp
requests
lxml
```

Development-only dependencies live in `requirements-dev.txt` (`-r requirements.txt` plus `pytest`). The browser tests additionally need `node` and `jsdom`:

```bash
.venv/bin/pip install -r requirements-dev.txt
cd tests/www && npm install
```

Both are optional: without them `tests/www` skips with a hint instead of failing.

Before adding an import:

1. decide whether it belongs in this component;
2. add it to `requirements.txt`, or `requirements-dev.txt` if it is test-only;
3. verify installation in a clean `.venv`;
4. document any required system package or executable.

Be cautious when upgrading FastAPI, Pydantic, FastMCP, or Uvicorn behavior. Even though `uvicorn` is imported directly, it may currently arrive transitively; direct runtime dependencies should be declared explicitly.

## Testing Expectations

At minimum, changes should cover the affected contract. Four suites exist today:

- `tests/test_prompt_wait.py` — the `what_next` wait: delivery from the transcription thread, timeout raising `queue.Empty`, an event loop that stays free, and a cancelled wait that leaves the prompt in the queue;
- `tests/test_last_view.py` — page identity from content, replacement lineage, chain following, cycle safety, and the guarantee that a replaced view is kept;
- `tests/test_user_selection.py` — marker insertion, repeated clicks, per-screen grouping, click moments, metadata misalignment, mismatch handling, visited-only screens, and the hash / `replaced_by` attributes;
- `tests/www/` — two jsdom harnesses driven by one runner (`harness_runner.py`): the record-button state machine and screen path against the real `www/script.js`, and page replacement against the real `www/page-carousel.js`.

`tests/www/harness.js` drives a controlled clock (`window.performance.now`) and a stub `window.pageCarousel`, so timing and screen assertions are exact rather than wall-clock dependent. The stub exposes `currentHash`, `justSwiped` and `onLiveChange`; `d.swipeTo(hash)` changes the live page and fires the callback the way `showLive` does. The real carousel is not loaded there — `justSwiped` lifecycle and gesture handling are still only covered by the manual smoke test.

`tests/www/carousel-harness.js` loads the real carousel and settles rides by dispatching `transitionend` on the track instead of waiting out the 450 ms fallback timer. jsdom has no layout, so `clientWidth` is 0 and every transform is 0 — assert on `index`, `length` and `currentHash`, not on pixels.

Browser tests go in the matching harness via `check(id, description, condition, detail)`; the pytest wrappers pick up new ids automatically.

Recommended test areas:

- stable page-hash generation;
- storing and retrieving historical views;
- maximum-view eviction;
- XPath match and mismatch behavior;
- clicks spread across several screens in one recording;
- click moments measured from the start of the recording;
- screens visited without a click, and their order;
- replacement lineage, including a view replaced between recording and transcription;
- replacement of the live page versus a neighbour;
- no-selection behavior;
- unique audio filenames;
- queue timeout behavior;
- cleanup of temporary audio;
- WebSocket payload shape, including `replaces`;
- multiple WebSocket clients;
- behavior with no connected GUI;
- `present_data` HTML/JS separation;
- restart command sanitization.

`present_data` itself has no automated test: importing `user_mcp` pulls in `next_prompt_manager`, which starts worker threads and binds port 8084 at import time. Cover its pieces (`last_view`, `user_selection`, the carousel) instead of importing the tool module in a test.

Tests should not require:

- a real microphone;
- a real browser;
- a live tmux session;
- production filesystem paths;
- an external speech provider.

Use dependency injection, monkeypatching, temporary directories, and mocked transcription where practical.

After changing browser integration, perform a manual smoke test:

1. start `./run.sh`;
2. open the GUI and type the PIN on the keypad — a wrong PIN must come back to the keypad instead of reconnecting;
3. call `present_data` twice with **different** content, so the carousel holds two screens — the hash is derived from the content and the carousel deduplicates by it, so two identical views collapse into one page and the multi-screen path goes untested;
4. verify rendering and page hash;
5. dock the record button in `#lockZone`, click something, swipe to the other screen, click something there, and stop the recording;
6. call `what_next`;
7. confirm the transcription, one block per screen, a `hash` on every block, `seen_at` on the screen you only swiped through, and a plausible moment in every marker;
8. call `present_data` with `replaces` set to one of those hashes — the content must change where it sits, without a ride and without losing scroll position, and the returned hash must be new;
9. record again without clicking anything and confirm the prompt names the new hash, not the replaced one.

## Coding Conventions

- Target the Python version used by the repository environment; current committed bytecode suggests mixed local Python versions, so do not infer compatibility from `.pyc` artifacts.
- Add type annotations to new public functions and queue payloads.
- Prefer small pure helpers around protocol and transformation logic.
- Keep network startup separate from business logic where possible.
- Use `logging` rather than adding more ad-hoc `print` statements.
- Keep user-facing strings understandable; preserve Polish text where it is part of the current agent contract.
- Do not swallow exceptions unless failure is explicitly best-effort, such as delayed tmux automation.
- Avoid new absolute paths and machine-specific usernames.
- Do not introduce a second implementation of an existing HTTP or WebSocket endpoint.

## Security and Privacy

This component handles microphone recordings, rendered user data, DOM selections, uploaded filenames, and local process control.

Required precautions:

- never log raw secrets, tokens, or credentials;
- avoid logging complete sensitive transcripts or HTML in production paths;
- validate file type and size before accepting uploads;
- do not trust filenames supplied by clients;
- keep temporary files outside the repository;
- delete recordings after processing unless retention is explicitly required;
- restrict network exposure where possible;
- review any change that broadens `0.0.0.0` listeners or adds CORS;
- treat HTML and JavaScript sent to the browser as active content;
- never insert unsanitized third-party content into privileged browser contexts without an explicit trust decision.

## Known Technical Debt

Treat these as existing constraints, not patterns to copy:

- `gui_provider.py` starts its WebSocket server as an import side effect.
- `NextPromptManager` starts worker and HTTP-server threads as an import side effect.
- `request_handler.py` duplicates part of the active upload flow and uses a hard-coded `/root/...` path.
- `requirements.txt` is unpinned and does not explicitly list every direct runtime import.
- logging is configured in multiple modules.
- several utility tools appear unrelated to the core human-interface responsibility.
- some environment assumptions, including tmux pane names, the shared-drive path, and `/root/.secrets/user_mcp.env`, are hard-coded.
- `tests/test.py` is still a blocking manual runner and is not collected by pytest.
- the component has no service manager; it is started by hand through `run.sh` and dies with its shell.

Do not expand this debt while implementing unrelated changes.

## Change Checklist

Before completing a change in this directory:

- [ ] Read the repository-level `AGENTS.md`.
- [ ] Identify whether the change affects MCP, HTTP, WebSocket, browser, or more than one boundary.
- [ ] Preserve the `present_data` payload contract, including `replaces` and the returned hash.
- [ ] Preserve page-hash consistency: computed on the server, opaque to the browser, derived from content.
- [ ] Keep a replaced view stored under its old hash.
- [ ] Keep `xpath`, `xpath_hash`, and `xpath_time` parallel and index-aligned end to end.
- [ ] Keep the `__scrolled__` sentinel identical in `www/script.js`, `lib/user_selection.py`, and `tests/www/harness.js`, and out of the XPath evaluator.
- [ ] Keep `hash` on every screen block — it is the agent's only handle for replacing a view.
- [ ] Preserve `what_next` queue and timeout semantics.
- [ ] Check ports 8084 and 8085 for conflicts.
- [ ] Avoid new import-time side effects.
- [ ] Avoid new absolute filesystem paths.
- [ ] Add or update automated tests where possible.
- [ ] Run `.venv/bin/python -m pytest tests`.
- [ ] Perform a browser/audio smoke test for protocol changes.
- [ ] Update `README.md` and this file when behavior or dependencies change.
- [ ] Ensure no logs, recordings, `.pyc`, `__pycache__`, `.venv`, or temporary uploads are staged.

## Documentation Rule

This file describes the current operational contract, not project history.

When a statement stops being true, replace it. Do not append a changelog here.
