# chart FastAPI backend + Vue 3 (from CDN, no build step) served at . The app charts Yahoo's `ES=F` feed, builds CME-session-aware timeframes, daily moving averages, prior-day high/low/close and session VWAP, and alerts on confluence zones. The full spec lives in [`docs/plan.md`](docs/plan.md) — read it before writing code; it records decisions and verified API facts that are expensive to rediscover. What it cost to get there is in [`docs/implementation.md`](docs/implementation.md): a dated log of problems and their resolutions, kept as a learning record alongside git. Both are living documents. The app is past the point of being built and into being maintained, so a change that makes either one wrong is not finished — correcting the plan, or logging what was hard, is part of the work rather than housekeeping after it. ## Local development ```bash docker compose up --build ``` Then open . Override the host port with, for example, `PORT=8020 docker compose up`. The port binds to all host interfaces, so another machine can connect at `http://HOST_IP:8010`. The source tree is bind-mounted and uvicorn runs with `--reload`, so edits to `main.py` or `static/` take effect without a rebuild. Rebuild only when `requirements.txt` changes. The Compose stack also includes Playwright for browser screenshots. It reaches the app over the internal Compose network and writes ignored artifacts locally: ```bash docker compose exec playwright playwright screenshot \ --lang en-US --wait-for-timeout 5000 http://api:8000 /artifacts/chart.png ``` Without Docker: ```bash python3 -m venv .venv && . .venv/bin/activate pip install -r requirements.txt uvicorn main:app --reload ``` Copy settings from `.env.example` as needed. To recalibrate the alert threshold against Yahoo's current eight-day minute tape: ```bash python3 -m scripts.calibrate_alerts ``` It sweeps threshold and cooldown in a single replay pass and prints alerts per session. The 2026-08-09 calibration ran when daily moving averages were the only levels, and `28` produced no alerts at all — there was nothing for a daily MA to cluster *with*. Adding prior-day H/L/C and VWAP changed that completely: the same threshold went to 247 alerts over six sessions, 185 of them in one day. Two fixes brought it back, in this order: - **Cluster identity** was `sha1(side + round(center / tolerance))`, and `tolerance` derives from ATR — so it moved every bar. The same zone was continually issued a new id, never matched the cooldown table, and the cooldown was silently defeated. 247 → 54. - **Alert suppression** keyed on cluster identity, so a level drifting in or out of a group counted as a new zone. It now suppresses by *proximity*: two zones within one ATR are the same zone. 54 → 40. - **Suppression also matched on side**, and side is positional — a level sitting at price flips between support and resistance every time price ticks across it. Each flip read as a new zone. Found by putting a real line at the live price and getting four pushes in two minutes. 40 → 26. Only then does the cooldown do anything useful. At threshold `28` the sweep reads: | cooldown | total | max/session | |---|---|---| | 900s | 26 | 19 | | 3600s | 18 | 12 | | 7200s | 12 | 7 | | **14400s (selected)** | **10** | **5** | Note the threshold itself is a blunt control: scores are sums of 12s (moving averages, VWAP) and 16s (prior-day levels), so `20`, `24` and `28` behave identically and `32` falls to zero. Cooldown is the finer knob. Revisit both as more varied tapes are recorded — six sessions is not much, and one of them dominates the totals. ## Testing Run the complete backend suite in the same container environment as the app: ```bash docker exec chart-api-1 sh -c "cd /app && python -m pytest -q" ``` Pass a path or pytest selector for a focused run: ```bash docker exec chart-api-1 sh -c \ "cd /app && python -m pytest -q tests/test_aggregator.py::test_closed_yahoo_hours_form_the_right_cme_daily_bar_across_1800_et" ``` Run every browser test against the local Compose stack, or filter by filename: ```bash ./bin/e2e ./bin/e2e trendline ./bin/e2e preferences ``` The browser suite creates drawings and removes them afterward. Run it against the local stack, not production: production has the persistent drawing store, live alerts, and an authenticated feed. The helper deliberately runs tests one at a time because the local drawing store is shared with anyone using the dev chart. Without Docker, install the development requirements before running pytest: ```bash python3 -m venv .venv && . .venv/bin/activate pip install -r requirements.txt -r requirements-dev.txt python -m pytest -q ``` Recommended pre-deploy and deploy verification: ```bash docker exec chart-api-1 sh -c "cd /app && python -m pytest -q" ./bin/e2e git push && bin/wait-deploy curl -fsS https://chart.amow.com/api/health curl -fsS https://chart.amow.com/api/version ``` For an authenticated production status smoke test, use the chart token without printing it: ```bash curl -fsS -H "X-Chart-Token: $TOKEN" https://chart.amow.com/api/status | jq . ``` ## Layout | Path | Purpose | |---|---| | `main.py` | FastAPI lifespan and app wiring; JSON under `/api`, SPA at `/` | | `app/` | Market sources, aggregation, analysis, alerts, and API | | `static/` | `index.html`, `app.js`, `style.css` — Vue 3 loaded from unpkg | | `requirements.txt` | Python deps | | `Procfile` | Start command; **nixpacks needs this** or the deploy has nothing to run | | `bin/wait-deploy` | Blocks until the live site serves your latest commit | | `Dockerfile.dev`, `docker-compose.yml` | Local dev only — production does not use them | ## Deployment Push to `main` → Forgejo webhook → Coolify rebuilds with nixpacks → live. Rebuild time depends on whether Docker's build cache is warm: measured at **~20s warm** (you pushed recently) and **~90s cold** (the cache goes stale after an idle hour or so, which is the usual case). Nothing changes on the site until the new container swaps in at the very end, so the old version keeps serving for the whole build. To know when your commit is actually live, rather than guessing: ```bash git push && bin/wait-deploy ``` It polls `/api/version` (which returns the `SOURCE_COMMIT` Coolify bakes into the container) until it matches your local `HEAD`, then exits. Runs from any machine — no Coolify token, no SSH tunnel. Check by hand any time with: ```bash curl -s https://chart.amow.com/api/version ``` One more trap worth naming: if your change only touches an `/api` endpoint, the HTML is byte-identical and a browser refresh looks like nothing happened even after a successful deploy. Verify the endpoint, not the page. Two things to know before changing the deploy config: - The Coolify domain is registered as `https://chart.amow.com:8000`. The port suffix is how Coolify decides the Traefik target port; drop it and every request 502s while the container looks perfectly healthy. It does not appear in the public URL. - Ignore a 502 in the first minute after a deploy — that's the rolling container swap, and it clears on its own. Manual redeploy: ```bash curl -X POST -H "Authorization: Bearer $(cat ~/.coolify-token)" \ "http://127.0.0.1:8000/api/v1/deploy?uuid=dgvch0xqv8uvjfor7dl8bwl9&force=true" ``` ## Putting the live feed on production Production runs the default `LIVE_SOURCE=yahoo` and therefore shows "yahoo" with a ~10 minute delay. `LIVE_SOURCE=schwab` and the credentials live in `.env`, which is gitignored — so the switch has never crossed the deploy boundary. It cannot: nothing in the repo carries it. Four things are needed, and all of them are set on the VPS rather than here. **1. Environment, in Coolify:** ``` LIVE_SOURCE=schwab SCHWAB_API_KEY=... # same values as the local .env SCHWAB_APP_SECRET=... ``` `SCHWAB_CALLBACK_URL` already defaults to `https://chart.amow.com/api/qt`, which is what the Schwab app is registered with. Do not change it — a registered callback edit can send the app back through approval. **2. The OAuth token, at `data/.schwab_token.json` on the persistent volume.** No new login is required: the local token was minted against the production callback, and Schwab tokens are bound to the app rather than the machine. Copy the file's contents into that path on the volume. If the path is not persistent, the next deploy erases it and production falls silently back to Yahoo. **3. Stop the dev stream first.** One refresh token means one Schwab streaming connection. Dev and production both streaming will fight for it — see the risk register on duplicate connections. Set `LIVE_SOURCE=yahoo` in the local `.env`, or stop the local stack, before production goes live. **4. Plan for expiry.** The Schwab refresh token lasts about seven days. When it lapses, production drops back to Yahoo and needs a fresh token by the same route: run the flow locally, paste the callback URL from `/api/qt` into the waiting prompt, then copy the new token onto the volume. Verify from anywhere: ```bash curl -s -H "X-Chart-Token: $TOKEN" https://chart.amow.com/api/status # want: "source":"schwab","delay_minutes":0 ``` ## Browser password and API token `CHART_AUTH_TOKEN` guards everything under `/api` plus the `/ws` stream. Leave it blank and the app is wide open, which is what you want locally — nothing prompts. Scripts can supply it as the `X-Chart-Token` header or a `?token=` query parameter. Set `CHART_PASSWORD` to a human-friendly passphrase for browser access. The browser posts it once to `/api/login`; the server returns a signed, HttpOnly, 30-day HS256 JWT cookie used by both API requests and the WebSocket. The opaque API token is never returned to or stored by the browser. If `CHART_PASSWORD` is temporarily absent, the login accepts `CHART_AUTH_TOKEN` as a migration fallback. Existing browsers that stored a token under the old flow exchange it once for a session and remove it from `localStorage`. `POST /api/logout` clears the session. In production both secrets live in **Coolify's environment variables**, not in this repo and not in `.env` — that file is gitignored and never exists in the built container. Coolify re-injects its env vars into every container it builds, so the token survives redeploys and reboots. `/api/health` and `/api/version` deliberately stay open — `bin/wait-deploy` polls the latter from whatever machine you pushed from, and neither reveals anything about the market data or the configuration. ## Setting an alert on a price Type the price into **Price alert** in the sidebar. It appears as a horizontal line across the chart with a price-axis label, and alerts when price reaches it. A price alert is stored as a manual line with `slope = 0`, so it inherits the whole manual-line pipeline — JSON persistence, renaming, recolouring, deletion, clustering with nearby levels — rather than being a parallel system. Two consequences worth knowing: - It **alerts regardless of confluence score**, like any hand-placed level. Weight exists to rank levels you did not ask for; you asked for this one. - It has no drag handles and no "end line here" menu, because a price line has no endpoints to grab. Manage it from the sidebar list. If it lands near other levels it clusters with them and the score adds up, so a typed level sitting on the prior-day close reads as one zone rather than two alerts. ## Market data: Yahoo for the past, Schwab for the present Both sources run together. This is the intended configuration, not a fallback: - **Schwab** streams real-time `/ES` minute bars over `CHART_FUTURES` (`delayed: false`), but serves **no futures history at all** — everything it knows starts when you connect. - **Yahoo** has roughly 730 days of hourly data, 60 days of native 30-minute bars, and eight days of minute data. Hourly history makes a 200-day moving average warm at startup; the native 30-minute seed avoids limiting that chart to the minute endpoint's eight-day window. Yahoo lags ~10 minutes. Switch the live feed with `LIVE_SOURCE=schwab`; seeding stays on Yahoo whatever you set, because Schwab has nothing to seed from. The symbols differ — Yahoo says `ES=F`, Schwab says `/ES` — and `Settings.live_symbol` picks the right one. Every bar carries a `source` tag so the seam stays visible. **Expect a gap of up to ten minutes at the right-hand edge after a restart.** Yahoo's history reaches to *now − 10 min* while the stream starts at *now*, so the most recent bars are briefly missing. It backfills itself as Yahoo catches up. Live price and alerts are unaffected — those come from the stream. Persisting bars would remove it entirely. `/ES` resolves to the active contract (`/ESU26` today) on Schwab's side, so contract rolls need no handling. ### Rate limits The developer portal shows **Order Limit: 120**, which caps orders per minute — this app places none. Schwab separately rate-limits REST calls, commonly cited at 120 per minute. Neither constrains us, because **streaming is not REST**. The WebSocket is a single connection and bars arrive by push, so steady-state Schwab REST usage is essentially zero. This is a concrete advantage of the stream over the polling fallback: polling `get_quotes()` once a second would have sat at roughly half the limit permanently, forever. The exception is reconnects. Each one calls `get_user_preferences()` to fetch the socket URL, and `StreamService` retries every five seconds, so a sustained outage generates about twelve REST calls a minute. Comfortably under, but not nothing — worth remembering before shortening that backoff. Yahoo is a different service and none of these limits apply to it. ### Authenticating ```bash python3 -m scripts.check_schwab # prints the login URL python3 -m scripts.check_schwab --redirect-url '…' # exchanges the code python3 -m scripts.check_stream 60 /ES # confirms bars arrive ``` Two steps, neither interactive, so the browser can be on a different machine — you copy a URL out and paste one back. **The authorisation code expires in about thirty seconds**, so have the second command ready before you approve. The callback page 404s until this branch is deployed; that is cosmetic, the code is in the address bar regardless. The token refreshes itself for seven days, then needs the flow again. It is per-machine: `.schwab_token.json` locally, and under `data/` in production, which is the Coolify volume. Locally `data/` is owned by root because Docker created it through the bind mount, which is why the local path differs. ## Alerts and ntfy Alerts are evaluated **server-side**, once per closed 1m bar, by a single engine living in `Runtime`. They do not depend on a browser being connected — that is the whole point of the phone push — and opening two tabs does not double-notify. `NTFY_TOPIC` must be set or nothing sends; `send_ntfy` returns immediately on a blank topic. Set it in **Coolify's environment variables** for production, not in this repo. **Cooldown state survives a restart.** The fired-zone table is written to `ALERT_STATE_PATH` (`./data/alert_state.json`), which in production is the same persistent volume as the trendlines. Before that, every deploy started with empty cooldowns and the next closed bar re-alerted whatever zone price was sitting on — with a four-hour cooldown, each push produced a burst of notifications for zones that had already had their say. Two consequences worth knowing. The file has to be on the volume, or the problem comes straight back on the next deploy. And a corrupt or unreadable state file is deliberately non-fatal: it logs and starts empty, costing one burst of duplicate alerts rather than refusing to start the stream. **Still prefer a blank topic locally.** Persistence removes the restart bursts, but an in-memory engine is only half the story — a dev instance watching the same symbol will happily push real alerts to your phone. Leaving `NTFY_TOPIC` blank keeps the in-browser sound and banner while suppressing the push; set a `-dev` topic only while testing the push path itself. Note that ntfy topics are public by default: anyone who knows the name can both read your alerts and publish to it. Treat the topic name as a secret. ## Saved trendlines Manual trendlines are written to `MANUAL_LINES_PATH` (`./data/manual_lines.json`). In production `/app/data` is a **Coolify persistent volume** — without it the container filesystem is ephemeral and every deploy would silently wipe every line you've drawn.