chart/README.md

# chart

FastAPI backend + Vue 3 (from CDN, no build step) served at
<https://chart.amow.com>.

The app charts Yahoo's `ES=F` feed, builds CME-session-aware timeframes, daily moving
averages, prior-day high/low/close and session VWAP, and alerts on confluence zones.
The full spec lives in
[`docs/IMPLEMENTATION_PLAN.md`](docs/IMPLEMENTATION_PLAN.md) — read it before writing
code; it records decisions and verified API facts that are expensive to rediscover.

## Local development

```bash
docker compose up --build
```

Then open <http://localhost:8010>. Override the host port with, for example,
`PORT=8020 docker compose up`. The port binds to all host interfaces, so another
machine can connect at `http://HOST_IP:8010`. The source tree is bind-mounted and uvicorn
runs with `--reload`, so edits to `main.py` or `static/` take effect without a
rebuild. Rebuild only when `requirements.txt` changes.

The Compose stack also includes Playwright for browser screenshots. It reaches
the app over the internal Compose network and writes ignored artifacts locally:

```bash
docker compose exec playwright playwright screenshot \
  --lang en-US --wait-for-timeout 5000 http://api:8000 /artifacts/chart.png
```

Without Docker:

```bash
python3 -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
uvicorn main:app --reload
```

Copy settings from `.env.example` as needed. To recalibrate the alert threshold against
Yahoo's current eight-day minute tape:

```bash
python3 -m scripts.calibrate_alerts
```

It sweeps threshold and cooldown in a single replay pass and prints alerts per session.

The 2026-08-09 calibration ran when daily moving averages were the only levels, and
`28` produced no alerts at all — there was nothing for a daily MA to cluster *with*.
Adding prior-day H/L/C and VWAP changed that completely: the same threshold went to 247
alerts over six sessions, 185 of them in one day.

Two fixes brought it back, in this order:

- **Cluster identity** was `sha1(side + round(center / tolerance))`, and `tolerance`
  derives from ATR — so it moved every bar. The same zone was continually issued a new
  id, never matched the cooldown table, and the cooldown was silently defeated. 247 → 54.
- **Alert suppression** keyed on cluster identity, so a level drifting in or out of a
  group counted as a new zone. It now suppresses by *proximity*: two zones within one
  ATR are the same zone. 54 → 40.
- **Suppression also matched on side**, and side is positional — a level sitting at
  price flips between support and resistance every time price ticks across it. Each
  flip read as a new zone. Found by putting a real line at the live price and getting
  four pushes in two minutes. 40 → 26.

Only then does the cooldown do anything useful. At threshold `28` the sweep reads:

| cooldown | total | max/session |
|---|---|---|
| 900s | 26 | 19 |
| 3600s | 18 | 12 |
| 7200s | 12 | 7 |
| **14400s (selected)** | **10** | **5** |

Note the threshold itself is a blunt control: scores are sums of 12s (moving averages,
VWAP) and 16s (prior-day levels), so `20`, `24` and `28` behave identically and `32`
falls to zero. Cooldown is the finer knob. Revisit both as more varied tapes are
recorded — six sessions is not much, and one of them dominates the totals.

## Layout

| Path | Purpose |
|---|---|
| `main.py` | FastAPI lifespan and app wiring; JSON under `/api`, SPA at `/` |
| `app/` | Market sources, aggregation, analysis, alerts, and API |
| `static/` | `index.html`, `app.js`, `style.css` — Vue 3 loaded from unpkg |
| `requirements.txt` | Python deps |
| `Procfile` | Start command; **nixpacks needs this** or the deploy has nothing to run |
| `bin/wait-deploy` | Blocks until the live site serves your latest commit |
| `Dockerfile.dev`, `docker-compose.yml` | Local dev only — production does not use them |

## Deployment

Push to `main` → Forgejo webhook → Coolify rebuilds with nixpacks → live.
Rebuild time depends on whether Docker's build cache is warm: measured at
**~20s warm** (you pushed recently) and **~90s cold** (the cache goes stale
after an idle hour or so, which is the usual case). Nothing changes on the site
until the new container swaps in at the very end, so the old version keeps
serving for the whole build.

To know when your commit is actually live, rather than guessing:

```bash
git push && bin/wait-deploy
```

It polls `/api/version` (which returns the `SOURCE_COMMIT` Coolify bakes into
the container) until it matches your local `HEAD`, then exits. Runs from any
machine — no Coolify token, no SSH tunnel. Check by hand any time with:

```bash
curl -s https://chart.amow.com/api/version
```

One more trap worth naming: if your change only touches an `/api` endpoint, the
HTML is byte-identical and a browser refresh looks like nothing happened even
after a successful deploy. Verify the endpoint, not the page.

Two things to know before changing the deploy config:

- The Coolify domain is registered as `https://chart.amow.com:8000`. The port
  suffix is how Coolify decides the Traefik target port; drop it and every
  request 502s while the container looks perfectly healthy. It does not appear
  in the public URL.
- Ignore a 502 in the first minute after a deploy — that's the rolling container
  swap, and it clears on its own.

Manual redeploy:

```bash
curl -X POST -H "Authorization: Bearer $(cat ~/.coolify-token)" \
  "http://127.0.0.1:8000/api/v1/deploy?uuid=dgvch0xqv8uvjfor7dl8bwl9&force=true"
```


## Putting the live feed on production

Production runs the default `LIVE_SOURCE=yahoo` and therefore shows "yahoo" with
a ~10 minute delay. `LIVE_SOURCE=schwab` and the credentials live in `.env`,
which is gitignored — so the switch has never crossed the deploy boundary. It
cannot: nothing in the repo carries it.

Four things are needed, and all of them are set on the VPS rather than here.

**1. Environment, in Coolify:**

```
LIVE_SOURCE=schwab
SCHWAB_API_KEY=...          # same values as the local .env
SCHWAB_APP_SECRET=...
```

`SCHWAB_CALLBACK_URL` already defaults to `https://chart.amow.com/api/qt`, which
is what the Schwab app is registered with. Do not change it — a registered
callback edit can send the app back through approval.

**2. The OAuth token, at `data/.schwab_token.json` on the persistent volume.**
No new login is required: the local token was minted against the production
callback, and Schwab tokens are bound to the app rather than the machine. Copy
the file's contents into that path on the volume. If the path is not persistent,
the next deploy erases it and production falls silently back to Yahoo.

**3. Stop the dev stream first.** One refresh token means one Schwab streaming
connection. Dev and production both streaming will fight for it — see the risk
register on duplicate connections. Set `LIVE_SOURCE=yahoo` in the local `.env`,
or stop the local stack, before production goes live.

**4. Plan for expiry.** The Schwab refresh token lasts about seven days. When it
lapses, production drops back to Yahoo and needs a fresh token by the same
route: run the flow locally, paste the callback URL from `/api/qt` into the
waiting prompt, then copy the new token onto the volume.

Verify from anywhere:

```bash
curl -s -H "X-Chart-Token: $TOKEN" https://chart.amow.com/api/status
# want: "source":"schwab","delay_minutes":0
```

## Access token

`CHART_AUTH_TOKEN` guards everything under `/api` plus the `/ws` stream. Leave
it blank and the app is wide open, which is what you want locally — nothing
prompts. Set it and every request needs the token, as the `X-Chart-Token`
header or a `?token=` query parameter (WebSocket handshakes can't carry
headers, hence the second form).

In production the token lives in **Coolify's environment variables**, not in
this repo and not in `.env` — that file is gitignored and never exists in the
built container. Coolify re-injects its env vars into every container it
builds, so the token survives redeploys and reboots.

The browser asks for it once on the first 401 and keeps it in `localStorage`.
To clear it: `localStorage.removeItem('chart-token')`.

`/api/health` and `/api/version` deliberately stay open — `bin/wait-deploy`
polls the latter from whatever machine you pushed from, and neither reveals
anything about the market data or the configuration.

## Setting an alert on a price

Type the price into **Price alert** in the sidebar. It appears as a horizontal line
across the chart with a price-axis label, and alerts when price reaches it.

A price alert is stored as a manual line with `slope = 0`, so it inherits the whole
manual-line pipeline — JSON persistence, renaming, recolouring, deletion, clustering
with nearby levels — rather than being a parallel system. Two consequences worth
knowing:

- It **alerts regardless of confluence score**, like any hand-placed level. Weight
  exists to rank levels you did not ask for; you asked for this one.
- It has no drag handles and no "end line here" menu, because a price line has no
  endpoints to grab. Manage it from the sidebar list.

If it lands near other levels it clusters with them and the score adds up, so a typed
level sitting on the prior-day close reads as one zone rather than two alerts.

## Market data: Yahoo for the past, Schwab for the present

Both sources run together. This is the intended configuration, not a fallback:

- **Schwab** streams real-time `/ES` minute bars over `CHART_FUTURES`
  (`delayed: false`), but serves **no futures history at all** — everything it
  knows starts when you connect.
- **Yahoo** has roughly 730 days of hourly data, which is what makes a 200-day
  moving average warm at startup rather than in ten months. It lags ~10 minutes.

Switch the live feed with `LIVE_SOURCE=schwab`; seeding stays on Yahoo whatever
you set, because Schwab has nothing to seed from. The symbols differ — Yahoo says
`ES=F`, Schwab says `/ES` — and `Settings.live_symbol` picks the right one. Every
bar carries a `source` tag so the seam stays visible.

**Expect a gap of up to ten minutes at the right-hand edge after a restart.**
Yahoo's history reaches to *now − 10 min* while the stream starts at *now*, so
the most recent bars are briefly missing. It backfills itself as Yahoo catches
up. Live price and alerts are unaffected — those come from the stream. Persisting
bars would remove it entirely.

`/ES` resolves to the active contract (`/ESU26` today) on Schwab's side, so
contract rolls need no handling.

### Rate limits

The developer portal shows **Order Limit: 120**, which caps orders per minute —
this app places none. Schwab separately rate-limits REST calls, commonly cited at
120 per minute.

Neither constrains us, because **streaming is not REST**. The WebSocket is a
single connection and bars arrive by push, so steady-state Schwab REST usage is
essentially zero. This is a concrete advantage of the stream over the polling
fallback: polling `get_quotes()` once a second would have sat at roughly half the
limit permanently, forever.

The exception is reconnects. Each one calls `get_user_preferences()` to fetch the
socket URL, and `StreamService` retries every five seconds, so a sustained outage
generates about twelve REST calls a minute. Comfortably under, but not nothing —
worth remembering before shortening that backoff.

Yahoo is a different service and none of these limits apply to it.

### Authenticating

```bash
python3 -m scripts.check_schwab                      # prints the login URL
python3 -m scripts.check_schwab --redirect-url '…'   # exchanges the code
python3 -m scripts.check_stream 60 /ES               # confirms bars arrive
```

Two steps, neither interactive, so the browser can be on a different machine —
you copy a URL out and paste one back. **The authorisation code expires in about
thirty seconds**, so have the second command ready before you approve. The
callback page 404s until this branch is deployed; that is cosmetic, the code is
in the address bar regardless.

The token refreshes itself for seven days, then needs the flow again. It is
per-machine: `.schwab_token.json` locally, and under `data/` in production, which
is the Coolify volume. Locally `data/` is owned by root because Docker created it
through the bind mount, which is why the local path differs.

## Alerts and ntfy

Alerts are evaluated **server-side**, once per closed 1m bar, by a single engine
living in `Runtime`. They do not depend on a browser being connected — that is the
whole point of the phone push — and opening two tabs does not double-notify.

`NTFY_TOPIC` must be set or nothing sends; `send_ntfy` returns immediately on a blank
topic. Set it in **Coolify's environment variables** for production, not in this repo.

**Use a different topic locally — or better, none.** Cooldown state is in memory, so
every restart starts with empty cooldowns and the first closed bar re-alerts whatever
zone price is sitting on. Locally that means every `--reload` save. Leaving
`NTFY_TOPIC` blank keeps the in-browser sound and banner while suppressing the push;
set a `-dev` topic only while testing the push path itself.

The same applies to production, more slowly: **a deploy resets the cooldowns**, so a
zone that alerted an hour ago can alert again right after a redeploy. Persisting the
fired-zone table would fix it.

Note that ntfy topics are public by default: anyone who knows the name can both read
your alerts and publish to it. Treat the topic name as a secret.

## Saved trendlines

Manual trendlines are written to `MANUAL_LINES_PATH` (`./data/manual_lines.json`).
In production `/app/data` is a **Coolify persistent volume** — without it the
container filesystem is ephemeral and every deploy would silently wipe every
line you've drawn.