# Jarvis Gateway

Python 3.12 / FastAPI service that fronts the Jarvis mobile app. Everything the
phone needs runs over one WebSocket: the DeepSeek brain (tool-calling, streaming),
the cloned Pocket TTS voice, long-term memory, tasks/reminders, and Ultron
lead-gen lookups. Deploys as its own Railway service, mirroring `memory-service/`.

This implements **§1 (Gateway)** and the **§3 WebSocket protocol** of
[`../APP_PLAN.md`](../APP_PLAN.md).

## Layout

| File | Role |
|---|---|
| `main.py` | FastAPI app: `GET /health`, `WS /ws`. Auth gate, per-connection history (~40 turns), turn lifecycle, barge-in, passive memory save. |
| `brain.py` | Streaming tool-calling loop. `DeepSeekAdapter` (OpenAI-compatible SSE) behind a vendor-swappable `LLMAdapter`. `SentenceRegrouper` regroups token deltas into sentences. `FastLane` = the cheap tool-less front lane that streams banter and escalates via the `<<ACT>>` sentinel. |
| `persona.py` | Builds the system prompt: the Jarvis persona (ported from `voice/llm.py`, desktop/Mac tool clauses stripped) + bundled preferences + memory index + `/values`. |
| `tools.py` | Tool schemas + executors: `memory_search`, `memory_remember`, `task_add`, `tasks_list`, `task_done`, `ultron_leads`, `ultron_team_activity`, `ultron_stats`, `ultron_contact_stats`. |
| `tts.py` | Pocket TTS wrapper (lazy-loaded). Emotion tag → mood-clip mapping. Outputs 16-bit PCM WAV bytes per sentence. |
| `persona_data/` | Bundled `preferences.md` / `speech-style.md` / `humour-level.md` (small, not secret). |
| `tests/` | pytest, no network / no model downloads. |

## Environment variables

| Var | Purpose |
|---|---|
| `DEEPSEEK_API_KEY` | DeepSeek brain (required for chat). |
| `MODEL` | Full-lane DeepSeek model (tools), default `deepseek-v4-pro`. |
| `FAST_MODEL` | Fast-lane model (cheap, no tools), default `deepseek-v4-flash`. Answers banter/simple turns; escalates to the full brain via a `<<ACT>>` sentinel. |
| `FAST_CHAT` | `1` (default) enables the fast lane; `0` sends every turn straight to the full brain. |
| `DEEPSEEK_URL` | Override base URL, default `https://api.deepseek.com`. |
| `MEMORY_API_URL`, `MEMORY_API_KEY` | The memory service (same as `voice/`). Group via `MEMORY_GROUP` (default `ahmed`). |
| `SUPABASE_REST`, `SUPABASE_SECRET` | Ultron master DB (PostgREST RPCs, same as the dashboard). |
| `GATEWAY_KEY` | Bearer secret clients must send in the `hello` frame. **If unset, the auth gate is OFF (local dev only) — always set it in prod.** |
| `VOICE_DIR` | Directory of voice-ref WAVs. Default: the repo's `../voices` for local dev; a Railway volume mount in prod. |
| `POCKET_VOICE`, `VOICE_REF`, `POCKET_MAX_TOKENS` | TTS tuning (catalog fallback voice, base clone wav basename, token cap). |
| `PORT` | Set by Railway; defaults to 8080 (Docker) / 8000 (`python main.py`). |

### Where the secrets live locally
- `claude-voice/.env` has `DEEPSEEK_API_KEY`, `MEMORY_API_URL`, `MEMORY_API_KEY`.
- Supabase keys (`SUPABASE_REST`, `SUPABASE_SECRET`) live in the Ultron repo's
  `/Users/ahmed/devFolder/Ultron/.env`. Copy both sets into the gateway's
  environment for local runs (or export them). `.env` is gitignored — never commit it.

## Local dev

```bash
cd gateway
python3.12 -m venv .venv && source .venv/bin/activate
pip install -r requirements-dev.txt            # runtime + pytest
# voice (optional, heavy — pulls torch): pip install -r requirements-voice.txt
pytest                                          # all green, no network
export $(grep -v '^#' ../.env | xargs)          # DeepSeek + memory keys
python main.py                                  # serves on :8000
curl localhost:8000/health
```

Chat works without the voice deps; audio frames degrade to a single
`{"type":"error"}` per turn (the socket stays up) if Pocket TTS isn't installed.

## Railway setup

1. New service from this GitHub repo, **root directory = `gateway/`** (mirrors
   `memory-service/`). The `Dockerfile` builds it.
2. Set env vars: `DEEPSEEK_API_KEY`, `MODEL`, `MEMORY_API_URL`, `MEMORY_API_KEY`,
   `SUPABASE_REST`, `SUPABASE_SECRET`, `GATEWAY_KEY`, `VOICE_DIR=/voices`.
   Railway provides `PORT`.
3. **Voice build:** to serve audio, build with `--build-arg INSTALL_VOICE=1`
   (installs Pocket TTS + torch). Leave it off for a chat-only gateway.
4. **Volume for voiceprints:** attach a Railway volume mounted at `/voices` and
   set `VOICE_DIR=/voices`.

### Uploading voice refs WITHOUT git (voiceprints must never be committed)

The clone (`jarvis_ref.wav`) and mood clips (`jarvis_warm.wav`,
`jarvis_excited.wav`, `jarvis_dry.wav`, `jarvis_sad.wav`, `jarvis_calm.wav`) are
**never** in git or the Docker image (`.dockerignore` excludes `voices/` and
`*.wav`). Push them to the Railway volume out-of-band:

```bash
# from the repo (files live in ../voices) — use Railway's volume shell / SSH:
railway link                       # select the gateway service
railway ssh                        # into the running container
# then, in a separate local terminal, copy each wav up via the volume, e.g.
railway volume  # inspect the mount, then scp/rsync into /voices
```

Any transport that lands the WAVs in the mounted `/voices` works (Railway volume
SSH/`scp`, a one-off upload job, or `railway run` with a sync script). The point
is they arrive on the volume, not through the repo. If a mood clip is missing the
base voice is used; if `jarvis_ref.wav` is missing it falls back to the Pocket
catalog voice (`POCKET_VOICE`).

## WebSocket protocol (§3)

Client → server (JSON text): `hello` (first frame, with `auth` + `mode`),
`utterance` (`text`, `tone`, `source`), `barge_in`, `set_mode`, `ping`.

Server → client: `ready`, `sentence` (`seq`, `text`, `emotion`), then in voice
modes `audio_start` → one binary WAV → `audio_end`, `turn_end`, `error`, `pong`.
A `sentence` always precedes its `audio_start`; `chat` mode sends no audio;
after `barge_in` the server stops the turn's frames and sends `turn_end`.

## Two-lane latency design
Mirrors the desktop Jarvis (`voice/local_brain.py`). Every turn first hits the
**fast lane** (`FAST_MODEL`, `deepseek-v4-flash`) — a compact butler persona,
**no tools**, streamed. It answers greetings/banter/simple questions directly;
when a turn needs memory, tasks, Ultron data, recall, or anything nontrivial it
outputs the sentinel `<<ACT>>` (alone), and the gateway **silently reruns the
turn on the full brain** (`MODEL`, `deepseek-v4-pro` + tools). Both lanes share
**one history**: whichever lane completes commits the user + assistant turn, and
the bare sentinel is never recorded. `FAST_CHAT=0` disables the fast lane.

Per-turn timing lands in the logs as single-line, grep-friendly `TIMING …`
lines at INFO: `utterance_received → first_token → first_sentence → first_audio
→ served/turn_end`, each tagged with the serving `lane` (`fast`|`full`) and
elapsed `ms`. Grep the service logs with `grep TIMING`.

## Notes / deviations
- Base `requirements.txt` is chat-only (no torch). Voice deps are split into
  `requirements-voice.txt` so the default image stays light and the pytest/boot
  verification never downloads a model. Build with `INSTALL_VOICE=1` for voice.
- Conversation history is OpenAI-shaped; the `LLMAdapter` seam is where an
  Anthropic vendor would translate. `memory_forget` from APP_PLAN §1 is not
  wired in v1 (the phone has no delete UI yet); the other tools match the plan.