# Agent Instructions: `gateway_mcps`

> **Scope:** These instructions apply to files under `gateway_mcps/`.

> **Maintenance rule:** Keep this file current. Replace obsolete guidance instead of appending a historical changelog.

## Purpose

MCP Gateway — instructions for working with this project

=====================================================

Module: gateway_mcps/

## What This Is

A single MCP server that Claude/another client sees as one backend.
Under the hood it aggregates many MCP servers (backends) — the tools and resources from each
are exposed through the gateway and delegated back to the proper backend.

### The gateway mediates

- tools/list, tools/call (as well as notifications/tools/list_changed)
- resources/list, resources/read, resources/subscribe, resources/unsubscribe
  (as well as notifications/resources/list_changed and notifications/resources/updated)
- resources/templates/list — URI templates per RFC 6570 (placeholders `{name}`)
  aggregated from backends. URI resolution: first exact match, then first
  matching template (regex `[^/]+` per placeholder). Subscribe/read/unsubscribe
  on a concrete URI matched to a template are forwarded verbatim to the backend —
  the backend (FastMCP) matches the placeholder itself on its side.

## GLOBAL RULES — very important

1. **Every collision and every unusual situation = HARD ERROR.**
   Not a warning, not a debug log, not a silent skip. If an internal invariant is violated
   (public_name/public_uri collision, unknown content type, inconsistent registry state),
   `raise RuntimeError`/`TypeError`/`ValueError` with a specific message.
   The only exceptions where an error is suppressed: best-effort notifications to potentially
   broken MCP client sessions (logger.debug + continue) — because that is expected.

2. In-code comments and messages follow the existing code (do not change actual code).
   Do not rewrite them en masse without reason. The JOU drift to English does not apply to mcp_gateway.

3. One file = one responsibility (see the map below). Do not add network logic
   to the registry, nor registry logic to the executor.

4. Resource URIs are not namespaced with a backend prefix (unlike tool names).
   A URI is an opaque identifier — rewriting breaks the semantics. A URI collision between backends
   = `RuntimeError`. If you genuinely need it, use `public_uri` in register_resource.

5. Resource templates are first-match-wins when their regexes overlap.
   A collision of the EXACT SAME uri_template from two different backends = `RuntimeError`.
   Re-registration from the SAME backend (refresh / reconnect) replaces the entry
   without an error.

## File Structure

mcp_gateway/
├── AGENTS.md              ← this file
├── config.json             ← list of backends to load at startup
├── run.sh                  ← venv + launch main.py
├── requirements.txt
├── __init__.py
├── main.py                 ← entry point: loads config, assembles components, starts uvicorn
├── models.py               ← dataclasses: Backend, ToolDefinition, ToolMapping,
│                              ResourceDescriptor, ResourceMapping. No logic.
├── registry.py             ← Registry: in-memory mapping public_name → backend+original name,
│                              public_uri → backend+original URI. Collisions raise.
├── connection_pool.py      ← ConnectionPool + _ConnectionWorker: lifecycle of the ClientSession to each
│                              backend, liveness ping, callbacks for notifications
│                              (tools/list_changed, resources/list_changed, resources/updated, disconnect).
├── executor.py             ← Executor: calls the ClientSession for a given ToolMapping/ResourceMapping.
│                              No registry logic.
├── orchestrator.py         ← Orchestrator: the only place that mutates the Registry after startup.
│                              add/remove/refresh/enable/disable backend, reconnect loop,
│                              forwarding of notifications from backends to the gateway.
├── gateway.py              ← Gateway: the MCP server exposed to the world. Handlers list_tools/call_tool/
│                              list_resources/read_resource/subscribe/unsubscribe + broadcasting notifications
│                              to attached client sessions.
├── management.py           ← BuiltinTool + ManagementTools: built-in management__* tools
│                              (add_mcp_server, remove_mcp_server, list, refresh, disable_*).
├── management_server.py    ← Exposes management as a local MCP server (the gateway adds it as a backend).
└── rest_api.py             ← Alternative REST API (GET /tools, POST /tools/{name}, ...) for non-MCP
```text
                          clients.
```

tests/
├── conftest.py             ← Fixtures: mcp_server_url (test_server, has resources),
│                              mcp_server_b_url (test_server_b, no resources, name collision 'echo').
├── test_server.py          ← Test FastMCP: tools (echo, add, fail) + resources
│                              (test://greeting, test://config.json).
├── test_server_b.py        ← Second test FastMCP: tools (multiply, upper, echo) without resources.
├── test_registry.py        ← Unit tests for Registry (tools).
├── test_resources_registry.py  ← Unit tests for Registry (resources, URI collision).
├── test_executor.py        ← Integration tests for Executor (tools).
├── test_resources_executor.py  ← Integration tests for Executor (resources) + mock-based subscribe/unsubscribe.
├── test_connection_pool.py ← Integration tests for ConnectionPool (lifecycle, on_disconnect).
├── test_gateway.py         ← Integration tests for Gateway over the MCP protocol (tools).
├── test_resources_gateway.py   ← Integration tests for Gateway over the MCP protocol (resources, notifications).
├── test_orchestrator.py    ← Integration tests for Orchestrator (add/remove/disconnect/reconnect, tools).
├── test_resources_orchestrator.py  ← Orchestrator for resources (load/cleanup, forward updated).
├── test_notifications.py   ← End-to-end: adding a backend → tools/list_changed notification to the client.
├── test_management.py      ← Built-in management tools.
├── test_multi_backend.py   ← Multiple backends (server_a + server_b), name prefixing, no tool collisions.
├── test_backend_propagation.py
├── test_main.py            ← Config/startup tests.
├── test_integration.py
└── test_rest_api.py

## Test Conventions

- Only the `pytest-anyio` backend. Trio fails via `asyncio.get_event_loop()` in the pool —
  run `pytest -k 'not trio'`.
- Every integration test uses the `mcp_server_url` fixture (and optionally `mcp_server_b_url`).
- Two different backends in tests **must** use two different URLs — the same URL under two
  IDs will cause a resource URI collision and correctly blow up (per the global rule).
- Poll for the proper final state. The reconnect loop in the orchestrator loads tools then resources;
  poll `"id" not in _reconnect_tasks`, not `len(list_tools()) > 0`.

## How To Run

- Production:  `./run.sh`  (reads config.json)
- Tests:       `.venv/bin/python -m pytest tests/ -k 'not trio'`

## Backend Autostart

Each entry in `config.json` `backends[]` may have two extra fields:
  - `sh_path`   — path to the script that launches this backend (relative
```text
              to the config.json directory or absolute). The script must have
              the executable bit.
```

  - `autostart` — bool. When `true`, the gateway spawns the script at startup
```text
              as a subprocess (start_new_session=True, its own process
              group), waits until the backend URL starts responding
              (timeout 15s, then falls through to the reconnect loop), and in `finally`,
              after the server stops, sends SIGTERM to the process
              and — if it does not disappear within 5s — SIGKILL.
```

After the banner, BEFORE `server.serve()`, the gateway BLOCKS until every
enabled+autostart backend has tools or resources in the registry. The reconnect loop
already runs from `add_backend` in the banner phase, so this is just an `await
asyncio.sleep` in a loop. Hard timeout `AUTOSTART_ATTACH_TIMEOUT_S=120s` — on
exceeding it, raise + autostart_shutdown (better to fail loudly than to serve an
incomplete pool; clients like `nostr_explorer_mcp` harvest the pool once at
setup anyway and will not refresh it if a backend attaches later). `mcp_admin` typically
needs ~40s for a full init (replay tracked_pubkeys), so 120s gives a 3×
margin.

The banner shows `*` next to entries with `autostart: true` (e.g. `[OK] nostr_admin *`).

If the port of an autostart backend is already taken (someone launched it manually),
the gateway does NOT fail — it skips `Popen` and `add_backend` attaches to the
existing process. Such a backend does not go into `spawned`, so gateway shutdown
will not send it a SIGTERM (we did not start it, we do not stop it).
Preflight only guards the gateway and management ports — those MUST be free,
because uvicorn would fail on EADDRINUSE anyway.

`save_config` preserves `autostart`/`sh_path` per-id on every save
(dynamic `add_mcp_server`/`remove_mcp_server` through management do not
lose these fields on disk).

The standard stdout/stderr of the autostarted process is inherited by the gateway —
the backend's logs go through the same stream, without per-backend redirection.

### Example config

  {
```text
"host": "127.0.0.1", "port": 8000,
"backends": [
  {
    "id": "nostr_admin",
    "url": "http://127.0.0.1:8100/mcp/",
    "enabled": true,
    "autostart": true,
    "sh_path": "../nostr_db_mcp/admin/mcp/mcp_admin.sh"
  }
]
```

  }
