chart/docs/IMPLEMENTATION_PLAN.md

/ES Multi-Timeframe Confluence Chart — Implementation Plan

Audience: the implementing agent. This document is the spec; it is written to be executed top-to-bottom without re-deriving decisions.

One-line goal: stream /ES 1-minute bars from Schwab, aggregate them into every larger timeframe locally, derive trendlines and moving averages on each timeframe, project them all onto one chart in a shared (time, price) coordinate system, and alert when independently-derived levels from different timeframes converge.

Explicitly out of scope: order execution. Nothing in this codebase places a trade. See §14 for why, and for the seam left behind.

---

0. Start here

This document is now mostly history. M0–M10 are built and deployed. The build order, branch instructions and "existing repo state" that used to open this file described a greenfield app and were actively misleading by August 2026, so they are gone. What remains below is the reasoning behind decisions already made — read it to understand why something works the way it does, not to find out what to build.

For current work, start with AGENTS.md, which every agent loads automatically. It points at the live planning documents: docs/NEXT_STEPS.md for the short list, docs/async_refactor.md, docs/multi_user.md, docs/feature_undo.md and docs/mobile_enhance.md for designs not yet built.

main deploys to production. A push triggers a Forgejo webhook and Coolify rebuild of https://chart.amow.com. That is the intended workflow now, not an accident to avoid — but it means every push is a deploy, and a deploy restarts the market stream.

§16 onward is a dated log of problems and their resolutions. It is the most useful part of this file for anyone debugging: most entries record something that looked like one bug and was another.

Dependencies to add

requirements.txt currently contains only fastapi and uvicorn[standard]. Add:

httpx                 # Yahoo fetches; async, already a FastAPI-adjacent standard
pydantic-settings     # config.py

requirements-dev.txt:

pytest
pytest-asyncio

Do not add schwab-py until M6 — it is unused before then. Do not add yfinance; the Yahoo chart endpoint is a plain HTTP GET and the extra dependency buys nothing (verified — see §2.1).

Test fixture already provided

tests/fixtures/yahoo_es_1h.json is a real, trimmed Yahoo response for ES=F&interval=1h (40 bars). Use it to unit-test the parser offline. It deliberately contains a null in the OHLC arrays, which is exactly the case §2.1 warns about — if your parser doesn't filter those, that fixture will catch it.

Conventions

• All times are epoch seconds, UTC, everywhere. No naive datetimes.
• Nothing outside market/ may know which data source is in use.
• Nothing outside market/schwab.py may import broker-specific code.
• Analysis code takes lists of bars and returns values — no I/O, no clocks. This is what makes the replay harness work.

---

1. Decisions already made — do not relitigate

Decision	Choice	Why
Backend	FastAPI (already scaffolded)	Repo already runs it; native WebSocket support
Frontend	Vue 3 from CDN, no build step	Matches existing static/ setup; keeps deploy trivial
Charting	TradingView Lightweight Charts v5.2.0, standalone build	Apache-2.0, canvas, built for incremental realtime updates
Data source	Pluggable MarketDataSource. Yahoo first, Schwab later	Yahoo needs no API key and has the history Schwab lacks — see §2.1
Persistence	In-memory first, behind a BarStore interface	User confirmed deferring persistence is fine for v1
Eventual persistence	SQLite, not Postgres	Single file, zero Coolify stack expansion. Revisit only if multi-process
Deployment	Local first, VPS later	Keep all config in env vars so the VPS move is config, not rewrite
Alerts	ntfy/Pushover phone push (+ free in-browser sound)	User selected phone push
Auth	Hardcoded shared secret from env	User confirmed; only matters once VPS-exposed

| Trendlines | Manual (hand-drawn) first. Auto-detection deferred to M8 | Hand-drawn lines are correct by definition, so they validate the confluence engine with zero tuning risk — and later become the ground truth the auto-detector is tuned against | | Moving averages | Daily set (10/20/50/100/200 SMA) is the priority, shown on 1m / 30m / 1d | The 200DMA is a level people actually trade against |

Timeframe roles

4h was removed from the product on 2026-08-10. It was never enabled, and dropping it deleted the fiddliest logic in session.py — the wall-clock ET 4h anchor and its DST edge cases — for a timeframe nobody was using. Later sections of this document still use 4h in examples; read those as illustrative, not as a spec to build. Nothing below a day is session-anchored any more, so bucket_start now special-cases only 1d.

1m    base chart + alert evaluation      weight 1     ← a switchable base timeframe
2m    display only
5m    manual lines                       weight 1
15m   manual lines                       weight 2
30m   base chart + manual lines          weight 3     ← a switchable base timeframe
1h    manual lines (+ optional MAs)      weight 4
1d    base chart + THE DAILY MAs         weight 16    ← a switchable base timeframe

Base timeframe controls the candles only. Every level stays visible on every base timeframe — the 200DMA on a 1-minute chart is the point, not a side effect.

---

2.1 Data sources — build against Yahoo, swap in Schwab later

Do not block on Schwab API keys. The two sources are complementary, and the Yahoo one is strictly easier to develop against:

	Yahoo ES=F	Schwab /ES
Auth	none — plain HTTP GET	OAuth, keys, 7-day token refresh
Realtime	polled, possibly ~10 min delayed	true push websocket, realtime
1m history	8 days (per-request cap)	❌ none
1h history	~730 days (17,387 bars, verified back to 2024-03)	❌ none
1d history	~10 years (2,517 bars, verified back to 2016)	❌ none
Contract	continuous front-month, roll gaps	true contract

Endpoint, verified working with no key and no yfinance dependency:

https://query1.finance.yahoo.com/v8/finance/chart/ES=F?interval=1h&range=730d

Requires a browser User-Agent header. Returns chart.result[0] with timestamp[] and indicators.quote[0].{open,high,low,close,volume} as parallel arrays. Those arrays contain null holes — filter them before constructing Bars.

Therefore define one protocol and two implementations:

class MarketDataSource(Protocol):
    name: str
    def supports_history(self) -> bool: ...
    async def history(self, symbol, tf, start, end) -> list[Bar]: ...
    def supports_stream(self) -> bool: ...
    async def stream(self, symbol) -> AsyncIterator[Bar]: ...   # yields 1m bars

• YahooSource — real history(). stream() is a polling loop (every 15–30 s, interval=1m&range=1d, emit only bars newer than the last emitted) that presents the same async-iterator interface as a real push stream.
• SchwabSource — supports_history() -> False. stream() is the true CHART_FUTURES websocket.
• ReplaySource — reads a JSONL tape. Used by tests and offline development.

Nothing downstream of these may know which source it is using. Selection is one env var. In production both run at once: Yahoo seeds history at startup, Schwab provides the live tail.

Do not use Yahoo's daily bars

Yahoo anchors ES=F daily bars to midnight ET, but the CME futures session runs 18:00 → 17:00 ET (§6). Mixing the two definitions yields daily candles that disagree with every other chart you'll compare against.

Build daily bars yourself by aggregating Yahoo's 1h bars through the same aggregator.py used for live data — one session definition everywhere. Yahoo's 1h bars are anchored to the top of the ET hour, so they compose into session-anchored 1d buckets cleanly. The ~730-day 1h window yields ~500 sessions: enough for a daily 200SMA (~200 sessions) with room to spare.

Yahoo's native 1d bars may be used only for multi-year context, clearly labelled.

2.2 Schwab entitlements — answered empirically 2026-08-10

All verified against a live production app holding both Market Data Production and Accounts and Trading Production.

Question	Answer
Futures market data entitled?	Yes — but only via get_quotes() (plural)
Symbol format	/ES, which auto-resolves to the active contract /ESU26
CHART_FUTURES streaming	Works — one true-OHLCV minute bar per symbol per minute
LEVEL_ONE_FUTURES	Works, and reports delayed: false
Futures price history	Still none. Yahoo remains the only source of the past

Three traps found the hard way, all of which cost a round trip:

• get_quote() (singular) silently returns the wrong instrument. It puts the symbol in the URL path, where the leading slash is normalised away, so /ES comes back as ES — Eversource Energy, an equity, at $72. HTTP 200 with a populated body. get_quotes() passes symbols as a query parameter and returns the future correctly. Never treat a 200 as proof; check assetMainType.
• Streaming needs the Accounts and Trading product. StreamClient.login() reads /trader/v1/userPreference for its socket URL, and that path is not in Market Data Production. A market-data-only app cannot stream at all.
• Authorisation codes expire in about thirty seconds, and an unwritable token path spends one before revealing itself. scripts/check_schwab.py preflights the key, the secret and the token path for exactly this reason.

Because /ES resolves to the active contract on Schwab's side, contract roll handling — an open problem in §10 — needs no code here.

The remaining unknowns for Schwab

1. Futures market-data entitlement. It is not publicly documented whether CHART_FUTURES requires futures trading approval or a CME non-professional market data agreement on the Schwab account. Verify empirically in M6.
2. Symbol format. schwab-py docs show both /ES (continuous front-month) and /ESZ25 (specific contract). Determine which the stream actually accepts, and whether the continuous form auto-rolls. Record the answer in the README.
3. Bar cadence and lateness. Confirm CHART_FUTURES emits one message per symbol per minute, whether it re-sends a bar (correction), and how bars behave across the 17:00–18:00 ET settlement break.
4. Volume semantics. Confirm VOLUME is per-minute, not cumulative-for-session.

Confirmed facts (already verified — do not re-research)

Realtime futures data: fully available. Do not let §10 below suggest otherwise — these are different axes and conflating them will send you down the wrong path.

Service	Available?	Use here
CHART_FUTURES (1-min OHLCV, /ES)	✅ streaming	The base feed. Everything derives from it
LEVEL_ONE_FUTURES (live quotes)	✅ streaming	Current price/bid/ask for the status bar and alert evaluation
LEVEL_ONE_FUTURES_OPTIONS	✅ streaming	Live pricing of a proposed /ES put spread in M7
REST get_price_history for futures	❌ not available	— see §10
CHART_HISTORY_FUTURES service	❌ does not exist in schwab-py	—
Futures / futures-options order entry	❌ not offered	see §14

• ChartFuturesFields: SYMBOL=0, CHART_TIME_MILLIS=1, OPEN_PRICE=2, HIGH_PRICE=3, LOW_PRICE=4, CLOSE_PRICE=5, VOLUME=6.
• The gap is historical only, not realtime. schwab-py's docs, verbatim: "Schwab provides price history for equities and ETFs. It does not provide price history for options, futures, or any other instruments." A Schwab account does not change this — it is an API-surface limitation, not an entitlement one. Live /ES streams fine; there is simply no way to ask for yesterday's /ES bars. This is the single biggest constraint in the project; see §10.
• Lightweight Charts 5.2.0 standalone build exposes a window.LightweightCharts global containing createChart, CandlestickSeries, LineSeries, createSeriesMarkers, LineStyle. CDN: https://unpkg.com/lightweight-charts@5.2.0/dist/lightweight-charts.standalone.production.js (~196 KB).
• v5 changed the series API. Use chart.addSeries(LightweightCharts.CandlestickSeries, opts). The v4 chart.addCandlestickSeries(opts) form does not exist in v5 — most tutorials online are v4 and will not work.

---

3. Architecture

   YahooSource            SchwabSource           ReplaySource
   (history + poll)       (live websocket)       (JSONL tape)
        └──────────────────────┼──────────────────────┘
                               │  MarketDataSource protocol
                               ▼  1-minute OHLCV
                    ┌─────────────────┐
                    │  StreamService  │  single asyncio task, ONE connection
                    │  (+ recorder)   │──────► raw JSONL tape (for replay)
                    └────────┬────────┘
                             │ Bar(1m)
                             ▼
                    ┌─────────────────┐
                    │   Aggregator    │  session-aware bucketing
                    └────────┬────────┘
                             │ Bar(tf) closed / updated
              ┌──────────────┼──────────────┐
              ▼              ▼              ▼
        ┌──────────┐  ┌────────────┐  ┌──────────┐
        │ BarStore │  │  Pivots →  │  │  Moving  │
        │ (memory) │  │ Trendlines │  │ Averages │
        └──────────┘  └─────┬──────┘  └────┬─────┘
                            │              │
                            └──────┬───────┘
                                   ▼  Level[]  (unified type)
                          ┌──────────────────┐
                          │ ConfluenceEngine │  cluster + score
                          └────────┬─────────┘
                                   ▼  Cluster[]
                          ┌──────────────────┐
                          │   AlertEngine    │  state machine + cooldown
                          └────────┬─────────┘
                                   │
                    ┌──────────────┴───────────────┐
                    ▼                              ▼
             FastAPI WebSocket                 ntfy push
                    │
                    ▼
          Vue 3 + Lightweight Charts

Critical process constraint

The Schwab stream is one connection, one process, not thread-safe. Therefore:

• Run the streamer as a single asyncio task owned by FastAPI's lifespan.
• uvicorn --workers 1 always. More than one worker means more than one Schwab connection, which will fight over the session.
• --reload in dev will tear down and re-establish the stream on every file save. That is acceptable locally but expect reconnect churn.
• If the VPS deploy later needs multiple web workers, the streamer must be split into its own process with a message bus. Do not design for that now, but keep StreamService free of any FastAPI imports so the split stays cheap.

---

4. Module layout

main.py                     FastAPI app: lifespan, route mounting  (exists, extend)
app/
  config.py                 Settings from env (pydantic-settings)
  auth.py                   Shared-secret gate (no-op when unset)
  market/
    base.py                 MarketDataSource protocol, Bar emission contract
    yahoo.py                YahooSource: history() + polled stream()   ← build first
    schwab.py               SchwabSource: easy_client, CHART_FUTURES websocket
    replay.py               ReplaySource: JSONL tape
    stream.py               StreamService: owns a source, reconnect, emit Bar
    recorder.py             Record raw messages to JSONL
  bars/
    models.py               Bar, Timeframe
    session.py              CME session calendar + bucket boundary math
    aggregator.py           1m -> all timeframes
    store.py                BarStore protocol + InMemoryBarStore
  analysis/
    indicators.py           ATR, SMA, EMA (pure functions over bar lists)
    manual_lines.py         Hand-drawn lines: CRUD + JSON persistence   ← M5
    pivots.py               Fractal swing detection                     ← M8, deferred
    trendlines.py           Candidate generation, scoring, dedup        ← M8, deferred
    moving_averages.py      MTF MA levels
    levels.py               Level type + registry, rebuild orchestration
    confluence.py           Clustering + scoring
    alerts.py               State machine, cooldown, dispatch
  notify/
    ntfy.py                 Phone push
  api/
    routes.py               REST
    ws.py                   WebSocket hub
static/
  index.html                Vue 3 + LWC script tags        (exists, replace)
  app.js                    Vue app root                    (exists, replace)
  chart.js                  Lightweight Charts wrapper      (new)
  style.css                 (exists, extend)
tests/
  ...

---

5. Data model

Use dataclasses (or pydantic where it crosses the API boundary). All times are epoch seconds, UTC. Never store naive local datetimes.

class Timeframe(str, Enum):
    M1="1m"; M2="2m"; M5="5m"; M15="15m"; M30="30m"; H1="1h"; D1="1d"

    @property
    def seconds(self) -> int: ...   # D1 is session-defined, not 86400 — see §6

@dataclass
class Bar:
    tf: Timeframe
    t: int           # epoch seconds, bucket OPEN time
    o: float; h: float; l: float; c: float
    v: int
    closed: bool     # False while still forming
    symbol: str      # e.g. "/ESZ25" — carried so contract rolls stay visible

Level is the unified abstraction that makes the whole design work. Trendlines and moving averages both reduce to "a price at time t, with a weight and a side".

class LevelKind(str, Enum):
    MANUAL="manual"          # hand-drawn — ships first (M5)
    MA="ma"                  # moving average — ships first (M3)
    TRENDLINE="trendline"    # auto-detected — deferred to M8
    HORIZONTAL="horizontal"

class Side(str, Enum):
    SUPPORT="support"; RESISTANCE="resistance"

@dataclass
class Level:
    id: str              # stable hash — the UI diffs on this, so keep it stable
    kind: LevelKind
    tf: Timeframe
    side: Side
    weight: float        # timeframe weight x quality multiplier
    score: float         # raw quality score before weighting
    label: str           # "4h resistance", "1h EMA21"

    # Geometry. Trendline: price(t) = slope*(t - anchor_t) + anchor_p
    anchor_t: int
    anchor_p: float
    slope: float         # price units per SECOND. 0.0 for horizontal/MA-at-instant

    # For MAs: the stepped polyline actually drawn
    points: list[tuple[int, float]] | None

    touches: int
    first_t: int
    last_t: int
    provisional: bool    # derived from a still-forming bar
    hidden: bool         # layer-panel visibility — see §9.4 for its effect on scoring

    def price_at(self, t: int) -> float:
        return self.anchor_p + self.slope * (t - self.anchor_t)

The weight table referenced throughout — define it once, in config.py:

TIMEFRAME_WEIGHT = {
    Timeframe.M1: 1, Timeframe.M2: 1, Timeframe.M5: 1, Timeframe.M15: 2,
    Timeframe.M30: 3, Timeframe.H1: 4, Timeframe.D1: 16,
}
MA_WEIGHT_FACTOR = 0.75      # §7.4 — MAs weigh slightly less than drawn structure

@dataclass
class Cluster:
    id: str
    side: Side
    low: float; high: float; center: float
    score: float                 # sum of member weights
    members: list[Level]
    distance: float              # signed points from current price

---

6. Session and aggregation rules — read carefully

This is where a naive implementation silently produces wrong lines. CME ES is not a 9:30–16:00 instrument.

Session definition (America/New_York, DST-aware via zoneinfo):

• Trading week opens Sunday 18:00 ET.
• Daily settlement break 17:00–18:00 ET, Monday–Thursday. No bars expected.
• Week closes Friday 17:00 ET.
• A futures "day" runs 18:00 ET → 17:00 ET the following calendar day, and is conventionally labelled with the following calendar date. Sunday 18:00 bars belong to Monday's daily bar.

Bucketing rules:

• 1m, 2m, 5m, 15m, 30m, 1h — bucket on wall-clock UTC boundaries. These divide the hour evenly, so session anchoring is unnecessary and UTC keeps it simple.
• 1d — one bar per futures session as defined above. This is the only session-anchored timeframe. 4h used to be the other one and was the reason this section warned about DST; with 4h gone, that whole class of edge case went with it.

Implement this as session.py::bucket_start(t: int, tf: Timeframe) -> int and unit test it hard, including both DST transitions and the Sunday open. This function is the highest-risk piece of pure logic in the project. Write its tests first.

Aggregator behaviour:

• Maintain one forming bar per timeframe. On each incoming 1m bar:
  • if bucket_start(bar.t, tf) differs from the forming bar's t, close the forming bar (emit closed=True), then open a new one;
  • otherwise fold in: h=max, l=min, c=close, v+=, emit closed=False.
• Gaps do not close bars by time — they close by the arrival of a later bar. Never use a wall-clock timer to close a bucket; the market halts and holidays will fire it incorrectly. Exception: emit a closed=True for the previous bucket when a bar arrives that skips buckets entirely.
• Aggregation must be deterministic and replayable: feeding the same 1m sequence twice must produce byte-identical output. No datetime.now() inside the aggregator.

---

7. Analysis engines

7.1 Indicators (indicators.py)

Pure functions, list-in/list-out, no state: sma(values, period), ema(values, period), atr(bars, period=14). ATR is the universal scale unit — every tolerance in this project is expressed in ATR multiples, never in fixed points, so the same config works whether ES is at 4,000 or 8,000.

7.2 Pivot detection (pivots.py)

Fractal method, not regression. Regression fits the middle of price action; humans draw lines across extremes, and extremes are what other traders react to.

            ●  pivot high (w bars lower on both sides)
           / \
          /   \
     ●   /     \
    / \ /       \
───●───●──────────────
       pivot low

• pivot_high(bars, w): index i qualifies if high[i] >= max(high[i-w : i+w+1]) and i is the leftmost such index in ties.
• Default w = 3 for lower TFs, w = 2 for 4h/1d (fewer bars available).
• Confirmation lag is w bars — this is intentional. A pivot is only known w bars after it forms. Do not "detect" pivots on the forming bar; that repaints, and a repainting line is worse than no line.
• Significance filter: keep a pivot only if its prominence (distance to the surrounding swing in the opposite direction) >= 0.5 * ATR(14) on that timeframe. Drops noise pivots without hardcoding point values.

7.3 Automatic trendlines (trendlines.py) — deferred to M8

Not part of the initial build. Manual trendlines (§7.3a) ship first. This section is retained because it is the eventual target and because §7.3a is deliberately designed to produce the same Level objects, so adopting this later changes nothing downstream. Skip to §7.3a on a first pass.

for each timeframe, for each side (highs → resistance, lows → support):
    P = last N qualifying pivots            (N = 25; ~300 candidate pairs, trivial)
    for each pair (a, b) in P where a.t < b.t:
        line = through (a.t, a.p) and (b.t, b.p)
        evaluate(line) -> score or reject
    dedup, keep top K = 4 per side per timeframe

Evaluation of a candidate line:

Let tol = 0.25 * ATR(14) on that timeframe.

• Violation — a bar closes beyond the line by more than tol (above for resistance, below for support). Wicks do not count as violations; wicks through a level are normal and often the point.
• Touch — a bar's extreme comes within tol of the line without violating it.
• Reject the line if violations > 1 between the two anchors, or if any violation occurred after the later anchor (the line is broken and no longer active).

Score:

score =  3.0 * touches
       + 1.5 * log1p(span_in_bars)
       + 2.0 * recency_decay(last_touch)      # exp(-age_bars / halflife), halflife=50
       - 4.0 * violations

Normalize within each timeframe/side group so weights stay comparable across timeframes regardless of how many candidates a given timeframe happened to produce:

best = max(l.score for l in group)          # after dedup, before truncation to K
quality = clamp(l.score / best, 0.0, 1.0) if best > 0 else 0.0
level.weight = quality * TIMEFRAME_WEIGHT[tf]

So the strongest 4h line contributes the full 8.0, a mediocre one proportionally less, and a 5m line can never outweigh a 4h line no matter how many touches it has.

The four coefficients above (3.0 / 1.5 / 2.0 / 4.0) are starting values, not derived constants. They cannot be got right on paper — expect to tune them by eye against replayed tapes in M4. Put them in config.py, not inline, and treat "the lines land where a human would draw them" as the acceptance criterion.

Dedup: two lines are duplicates if, evaluated at now, their prices are within tol and their slopes differ by less than 20%. Keep the higher score. Without this you get a fan of ten nearly-identical lines from the same swing.

Recompute policy: only on a bar close for that timeframe, never on every tick. A 4h line recomputes 6× per day. This is what keeps the whole thing cheap.

7.3a Manual trendlines (manual_lines.py) — ships in M5

A hand-drawn line is just a Level with kind=MANUAL. It flows into confluence, projection, and alerts through exactly the same path as everything else — that is what makes deferring the automatic engine cheap rather than a detour.

Why this ordering is better than it looks: hand-drawn lines are correct by definition (you drew them). So they validate the confluence engine without the automatic detector's tuning risk, and they later become the ground truth that M8's scoring coefficients get tuned against.

Geometry. Anchors are stored in absolute epoch seconds and price — never bar indices. This is why a line drawn on the 4h chart renders correctly on the 1m chart with no conversion: both are the same (time, price) plane. The existing Level.price_at(t) already handles it.

Timeframe attribution. Tag the line with the timeframe that was displayed when it was drawn. A line drawn on the 4h chart is a 4h line and carries weight 8. This is the single most important field — without it every manual line would score identically.

Weight. quality = 1.0 always. The user drew it; it is not a candidate to be scored. weight = TIMEFRAME_WEIGHT[tf].

Drawing interaction (all APIs verified present in LWC 5.2.0):

Action	Implementation
Enter draw mode	Toolbar button; changes cursor
Place endpoints	chart.subscribeClick(handler) → two clicks
Pixel → price	series.coordinateToPrice(param.point.y)
Pixel → time	chart.timeScale().coordinateToTime(param.point.x)
Render	LineSeries with 2 points, extended right (same as §9 auto lines)
Select	Click within ~6px of a line — hit-test in price space via price_at(t)
Delete	Delete/Backspace on selection, plus a button
Edit	Delete and redraw. Endpoint dragging is real work — do not build it in M5

Snapping. When placing an endpoint, snap to the nearest bar high/low within ~8px. Cheap to implement and it is the difference between a usable drawing tool and a frustrating one. Make it toggleable; snap to high for resistance, low for support, inferred from drag direction or nearest extreme.

Persistence — required in M5, not deferred. A user who redraws their lines after every restart abandons the tool. This does not require the bar store or a database: write to a JSON file (data/manual_lines.json), loaded at startup. Bar persistence can stay deferred to M7; these two are unrelated decisions.

{"id":"ml_01H...","tf":"4h","side":"resistance","anchor_t":1754600000,
 "anchor_p":6412.5,"slope":-0.0000031,"created_at":1754700000,"note":"","hidden":false}

CRUD via POST /api/lines, DELETE /api/lines/{id}, PATCH /api/lines/{id}. On any change the server recomputes levels and broadcasts {"type":"levels",...} — the drawing client must render optimistically and then reconcile, not wait on the round trip.

7.4 Multi-timeframe moving averages (moving_averages.py)

Primary requirement: the daily MA set — SMA 10, 20, 50, 100, 200 — computed on daily bars and displayed on the 1d, 30m, and 1m charts, with the base timeframe switchable.

This is the headline use of the multi-timeframe projection: the 200DMA is a level people trade against, and on a 1-minute chart it should appear as a near-horizontal line that steps once per session. Do not compute a "200-period MA of 1-minute bars" for the 1m chart — that is a completely different and far less useful line. The MA's period is always tied to the timeframe it was computed on, never to the chart being displayed.

MA_SETS = {
    "1d":  [("sma", 10), ("sma", 20), ("sma", 50), ("sma", 100), ("sma", 200)],
    # optional, off by default:
    "1h":  [("ema", 9), ("ema", 21)],
}
BASE_TIMEFRAMES = ["1m", "30m", "1d"]     # the switcher; others remain available

Config-driven, so adding a set is a config change, not code. Other timeframes' MAs stay supported by the same machinery but ship disabled — the daily set is what matters.

History check: a 200DMA needs 200 sessions. Yahoo's 730-day 1h window yielded 17,387 bars ≈ 750 sessions, so all five daily MAs are warm from startup with roughly 3× margin. Verified, not assumed.

Expect small disagreements with thinkorswim/TradingView on the daily MAs. We build daily bars on the CME session (18:00→17:00 ET, §6); other platforms sometimes anchor differently or use settlement prices. A one- or two-point difference in the 200DMA is this, not a bug. Keep the daily anchor configurable so it can be matched if it matters.

The key rendering decision: an MA from a higher timeframe drawn on a 1-minute chart is a step function, held constant between higher-timeframe closes.

1h EMA21 rendered on a 1m chart:

      ┌────────
   ┌──┘              ← steps at each 1h close, NOT a smooth interpolation
┌──┘

Interpolating between higher-TF closes would draw a line that was never true at the time it appears to have been true. Emit points as a stepped polyline and render with LWC's lineType: LightweightCharts.LineType.WithSteps.

• The value from the forming higher-TF bar is emitted with provisional=True and rendered dashed. It will move until that bar closes — that is honest, not a bug.
• An MA's current value is a price level, so it enters the confluence engine on equal footing with trendlines, at 0.75 × timeframe_weight (MAs are slightly less reactive than drawn structure, but a 4h 200SMA is still a wall).
• Warm-up: an MA needs period closed bars on its timeframe. A daily 200SMA needs 200 sessions. Until warm, emit nothing — never emit a partially-warmed MA. Yahoo seeding (§10) makes every MA up to a daily 200SMA warm from startup; without it, the higher-timeframe MAs stay cold for months.

7.5 Confluence (confluence.py)

This is the actual product. Everything above exists to feed it.

1. Collect every active Level, evaluate price_at(now).
2. Split by side relative to current price (levels above → resistance, below → support).
   A level's stored `side` is its structural nature; its *effective* side is
   positional. Use positional — a broken resistance acting as support is the
   interesting case, not an error.
3. Sort by price. Single-linkage cluster: extend the current cluster while the gap to
   the next level is <= clusterTol.
       clusterTol = 0.4 * ATR14(15m)
4. Cluster score = sum of member weights.
5. Emit clusters with >= 2 members OR score >= 8 (a lone 1d line matters by itself).

5m  resistance   6404.25   weight 1
15m resistance   6405.00   weight 2
1h  resistance   6403.75   weight 4
4h  resistance   6404.50   weight 8
                     ↓
        RESISTANCE CLUSTER 6403.75 – 6405.00
        CONFLUENCE 15

Recompute on every closed 1m bar, and on any level rebuild.

7.6 Alerts (alerts.py)

The failure mode to design against is notification fatigue. A per-line alert makes the system useless within a day.

Per-cluster state machine, keyed on (side, round(center / clusterTol)) so a cluster keeps its identity as members drift:

ARMED ──price within alertTol of cluster──► FIRED ──► COOLDOWN ──┐
  ▲                                                              │
  └────── price moves > 2*alertTol away AND cooldown elapsed ◄────┘

• alertTol = 0.5 * ATR14(15m)
• cooldown = 15 minutes
• Minimum score threshold to fire: configurable, starting value 6 — but this certainly needs recalibrating in M4. With the daily MA set as the primary levels, each daily MA carries 0.75 × 16 = 12, so any two of them near each other scores 24 and a threshold of 6 would fire constantly. Either raise the threshold well above 24, or damp the weight when several MAs from the same timeframe cluster (they are not independent evidence the way a 4h line and a 1h line are). Decide this against a replayed tape, not on paper. Target: single-digit alerts per session.
• Re-arming requires both the price separation and the cooldown. Time alone lets a price oscillating on a level fire forever.
• Fire on closed 1m bars only, not intra-bar ticks.

Dispatch to: WebSocket (UI banner + sound) and ntfy. Payload:

BEARISH ZONE /ES 6404.25
Resistance confluence 15 @ 6403.75–6405.00
5m, 15m, 1h, 4h

---

8. API contract

Fix these shapes now; the frontend and backend are built against them.

REST

Route	Returns
GET /api/status	{stream: "connected"|"disconnected"|"replay", symbol, last_bar_t, bars_held: {tf: n}, warm: {tf: bool}}
GET /api/bars?tf=5m&limit=500	{tf, bars: [{t,o,h,l,c,v,closed}]} — oldest first
GET /api/levels?tf=all	{levels: [Level]}
GET /api/confluence	{price, clusters: [Cluster]}
GET /api/health	existing

WebSocket /ws

Client → server on connect:

{"type": "subscribe", "tf": "5m"}

Server → client:

{"type":"snapshot","tf":"5m","bars":[...],"levels":[...],"clusters":[...],"price":6404.25}
{"type":"bar","tf":"5m","bar":{"t":1754700000,"o":6403.5,"h":6405.0,"l":6403.0,"c":6404.25,"v":812,"closed":false}}
{"type":"levels","levels":[...]}
{"type":"clusters","price":6404.25,"clusters":[...]}
{"type":"alert","cluster":{...},"message":"..."}
{"type":"status","stream":"disconnected"}

Rules:

• Send bar on every update of the forming bar (that is the live chart) but batch levels — they only change on higher-TF closes.
• Always send a full snapshot on connect and after any reconnect. The client must never try to reconcile a gap.
• Levels are sent for all timeframes regardless of the displayed timeframe. That is the entire point: a 4h line drawn through a 1m chart.

---

9. Frontend

static/index.html loads Vue 3 and Lightweight Charts as script tags — no bundler, matching the existing app.

<script src="https://unpkg.com/vue@3/dist/vue.global.prod.js"></script>
<script src="https://unpkg.com/lightweight-charts@5.2.0/dist/lightweight-charts.standalone.production.js"></script>

Vue + Lightweight Charts integration — the one real gotcha

Never put the chart or series objects in ref() or reactive(). Vue's deep reactive proxy will wrap the library's internal objects, which breaks identity checks inside the library and destroys performance on every update. Use shallowRef, or better, a plain module-scoped variable / markRaw.

const chartApi = shallowRef(null);   // ✅
// const chart = ref(null);          // ❌ will appear to work, then misbehave

Logical indices address the whole chart, not your bar array

Never derive a viewport from bars.length. A logical index addresses the chart's shared time scale — the union of the time points of every series on it — not the candle array. Any series whose points pre-date the candle window prepends to that scale and shifts every logical index by its count.

// ❌ off by however many points the other series contribute
timeScale().setVisibleLogicalRange({ from: bars.length - 160, to: bars.length + 5 });
// ✅ an instant cannot be renumbered by a later series
timeScale().setVisibleRange({ from: bars.at(-160).t, to: bars.at(-1).t + step * 5 });

This is not theoretical — see the 2026-08-10 entry in §16. The daily MAs carry one point per daily bar (617 of them, back ~2 years). They are attached by syncVisibleLevels() immediately after setBars(), so a logical range that was correct when set silently slid 617 bars — about ten hours — into the past one tick later. The chart looked frozen while the socket was perfectly healthy.

Structure:

• chart.js — a plain, framework-free wrapper class owning the LWC instance: create(el), setBars(), updateBar(), syncLevels(levels), destroy().
• app.js — Vue app owning state (timeframe, connection status, clusters, alert log) and the WebSocket. Calls into the wrapper imperatively in onMounted / watchers.
• Chart lifecycle in onMounted; chart.remove() in onUnmounted; a ResizeObserver driving chart.applyOptions({width, height}).

v5 API usage

const chart = LightweightCharts.createChart(el, {...});
const candles = chart.addSeries(LightweightCharts.CandlestickSeries, {...});
const line    = chart.addSeries(LightweightCharts.LineSeries, {
  lineType: LightweightCharts.LineType.WithSteps,   // for MTF moving averages
});
candles.setData(bars);        // once
candles.update(bar);          // every tick — never re-call setData

Rendering levels

syncLevels() must diff by Level.id, not clear-and-rebuild. Rebuilding every series on each update causes visible flicker and leaks series objects.

• Trendline → a LineSeries with two points: (anchor_t, anchor_p) and (now + rightExtension, price_at(now + rightExtension)). Extend ~20% of the visible range into the future so the line is usable ahead of price.
• MA → a LineSeries fed the stepped points array.
• Colour by timeframe (one hue per TF, consistent everywhere including the confluence panel). Line width scales with timeframe weight. provisional levels render dashed via lineStyle: LightweightCharts.LineStyle.Dashed.

9.4 Layer panel (checkboxes)

Ships with M3. Visibility control is load-bearing here, not decoration — five daily MAs plus manual lines plus any optional intraday MA sets stack up fast.

CHART            [ 1m ] [ 30m ] [ 1d ]        ← base timeframe switcher

LAYERS
─────────────────────────────────
☑ Daily MAs      ██
     ☑ 10    ☑ 20    ☑ 50    ☑ 100   ☑ 200
─────────────────────────────────
☐ 1h MAs         ▓▓
     ☐ EMA9  ☐ EMA21
─────────────────────────────────
☑ Manual lines   ░░              (M5)
☐ Auto trendlines                (M8)
─────────────────────────────────
☐ Hidden levels still count toward confluence

• The base timeframe switcher changes only the candles. Every level stays on screen — a 200DMA is equally valid on a 1m chart. That is the entire premise of the product.
• The group checkbox is a master toggle — unchecking "Daily MAs" hides all five at once; individual periods nest under it.
• The colour swatch beside each timeframe is that timeframe's hue, used identically on the chart and in the confluence panel. One hue per timeframe, everywhere.
• Persist to localStorage. These settings are pure UI preference and must survive reloads; they do not belong on the server.

Visibility vs. scoring — decide this explicitly. Hiding a level defaults to also removing it from confluence scoring, because "I don't want to see this" almost always means "I don't care about this." The last checkbox decouples the two for anyone who wants a clean chart with full scoring.

That default has an architectural consequence: confluence is computed server-side, so the client's enabled set must reach the server. Send it over the WebSocket on change:

{"type":"prefs",
 "base_tf":"1m",
 "enabled":{"ma":{"1d":[10,20,50,100,200]},"manual":true,"auto":false},
 "hidden_levels_score":false}

Store it per connection. Single-user app — no need for anything more elaborate.

Panels

• Drawing toolbar (M5) — trendline tool, snap toggle, delete selection.
• Confluence panel — clusters sorted by |distance|, each showing member timeframes, the zone range, and the score. This is the primary readout; give it more visual weight than the price itself.
• Status bar — stream state, contract symbol, last bar age, which timeframes are warm. When the stream drops, the chart must say so, not quietly show stale candles.
• Alert log — recent fires, most recent first.
• Timeframe selector — switches the candle series only. Levels stay.

Keep the existing dark/light CSS-variable scheme in style.css.

---

10. The history problem

Mostly solved by the Yahoo source (§2.1) — but read the caveats.

Schwab alone would leave the system knowing nothing before the moment it connects. Since the highest-weighted timeframes need the most history, Schwab-only cold start would mean waiting months for the parts of the product that matter most:

Timeframe	Usable after, Schwab-only	With Yahoo seeding
5m, 15m	~2–4 hours	immediate
30m, 1h	~1–2 sessions	immediate
1d	months	immediate (~500 sessions)
1d 200SMA	~10 months	immediate

Startup sequence:

1. Seed from Yahoo: interval=1h&range=730d, plus interval=1m&range=8d for the fine detail near the present.
2. Aggregate through aggregator.py into every timeframe, using our own session rules.
3. Attach the live source (Yahoo poll, or Schwab once keys exist) and continue forward.
4. Persist everything (M7) so subsequent restarts need less seeding.

Caveats that must be honoured in code:

• Tag every bar with its source ("yahoo" / "schwab" / "replay"), and expose the seam in the UI. Silently blending delayed continuous data with live per-contract data produces trendlines nobody else can see.
• Roll gaps. Yahoo ES=F is a continuous front-month series; quarterly contract rolls leave price discontinuities that will read as a trendline break or generate a bogus pivot. For M0–M5 this is acceptable. If it proves noisy, the fix is either panama-adjusting the seeded series or dropping pivots within ±1 bar of a known roll date (third Friday of Mar/Jun/Sep/Dec). Do not build roll adjustment pre-emptively.
• The 1m request cap is 8 days. Longer 1m ranges must be fetched in ≤8-day windows and stitched. In practice you don't need deep 1m history — seed 1h and let the aggregator do the rest.
• Never mix Yahoo daily bars in — see §2.1.

If per-contract accuracy ever matters more than convenience, Databento sells proper CME history with real roll handling. Not needed now.

---

11. Configuration

All via env, read in config.py. Ship a .env.example; .env is already gitignored.

# --- data sources ---
LIVE_SOURCE=yahoo                 # yahoo | schwab | replay
SEED_SOURCE=yahoo                 # yahoo | none
YAHOO_SYMBOL=ES=F
YAHOO_POLL_SECONDS=20
SEED_1H_RANGE=730d
SEED_1M_RANGE=8d                  # 8d is Yahoo's hard per-request cap

# --- schwab (only needed once LIVE_SOURCE=schwab) ---
SCHWAB_API_KEY=
SCHWAB_APP_SECRET=
SCHWAB_CALLBACK_URL=https://127.0.0.1:8182
SCHWAB_TOKEN_PATH=./.schwab_token.json
SCHWAB_ACCOUNT_ID=
SCHWAB_SYMBOL=/ES

# --- timeframes & indicators ---
TIMEFRAMES=1m,2m,5m,15m,30m,1h,1d
BASE_TIMEFRAMES=1m,30m,1d         # the chart switcher
MAX_BARS_PER_TF=5000              # in-memory ring buffer bound

# Daily MA set is the primary requirement; others ship disabled. See §7.4
MA_SETS__1D=sma10,sma20,sma50,sma100,sma200
MA_SETS__4H=
MA_SETS__1H=
DAILY_ANCHOR_ET=18:00             # CME session open; change to match another platform

MANUAL_LINES_PATH=./data/manual_lines.json

CONFLUENCE_MIN_SCORE=6            # MUST be recalibrated in M4 — see §7.6
ALERT_COOLDOWN_SECONDS=900

NTFY_TOPIC=
NTFY_SERVER=https://ntfy.sh

CHART_AUTH_TOKEN=                 # empty = auth disabled (local dev)
REPLAY_FILE=                      # set to replay a tape instead of connecting

Token persistence: schwab-py's refresh token expires every 7 days and re-auth is an interactive browser flow. Locally, keep .schwab_token.json out of the repo (add to .gitignore). On the VPS later, it must live on a Coolify persistent volume or every rebuild logs you out. Same for the eventual SQLite file.

---

12. Testing

The market is closed most of the time you will be working. Build for that.

Record/replay is a milestone-1 deliverable, not a nicety. recorder.py writes every raw stream message to JSONL with its arrival timestamp; replay feeds them back through the identical code path, either at wall-clock speed or as fast as possible. Everything downstream of StreamService is then testable, deterministically, offline.

Required tests:

• session.py — bucket boundaries. Both DST transitions, Sunday 18:00 open, the 17:00–18:00 break, and Friday close. Write these first.
• aggregator.py — 1m→all TFs on synthetic bars; gap handling; idempotent replay.
• moving_averages.py — a known daily series produces known 10/20/50/100/200 values; assert nothing is emitted before warm-up; assert the stepped projection onto 1m holds its value for a whole session and changes exactly at the session boundary.
• manual_lines.py — round-trip JSON persistence; a line drawn on 4h evaluates to the same price on the 1m chart at the same instant.
• (M8) pivots.py — known fixtures; assert no repainting (a pivot, once emitted, never changes when more bars arrive).
• (M8) trendlines.py — hand-built fixtures where the correct line is obvious; assert violation rejection and dedup.
• confluence.py — synthetic levels producing a known cluster and score.
• alerts.py — assert no re-fire within cooldown, and that oscillation around a level produces exactly one alert.

Add pytest to a requirements-dev.txt.

---

13. Milestones

Ordered so the user's stated priority — live realtime charts first — lands earliest, and so nothing later is blocked on market hours.

No API keys are required until M6. M0–M5 run entirely on Yahoo.

M0 — Yahoo source (no keys, no blockers)

market/base.py protocol + market/yahoo.py: history() over the verified chart endpoint, and stream() as a polling loop presenting the same async-iterator interface. Null-filtering, 8-day 1m windowing, bar source tagging. Recorder writes a tape; ReplaySource reads it back. Done when: a script prints seeded 1h bars back to 2024 and then live-ish 1m bars, and a recorded tape replays identically.

M1 — Live chart end to end ⭐ primary deliverable

StreamService → in-memory 1m store → WebSocket → Vue 3 + LWC candlestick chart updating live. Status bar showing source + bar age. No analysis yet. Done when: the browser shows a live-updating /ES 1-minute candle chart, and the same chart can be reproduced offline from a tape.

M2 — Aggregation + timeframe switching

session.py + aggregator.py with full test suite. Timeframe selector drives the candle series. GET /api/bars. Done when: switching to 15m shows correctly bucketed bars, and replaying a tape twice yields identical output.

M3 — Multi-timeframe moving averages ⭐ start here for analysis

MAs are the right first analysis layer: fully deterministic, no parameters to tune, no judgment calls, and Yahoo seeding makes them warm from startup. They validate the entire overlay concept — stepped rendering, level diffing by id, TF colour scheme — without any of the ambiguity trendlines carry. Done when: the 10/20/50/100/200 DMAs render on the 1m, 30m and 1d charts, stepping once per session on the intraday views, dashed while the current session is unfinished.

M3.5 — Layer panel

Checkbox tree to show/hide levels by timeframe and by individual MA (§9.4). Ships with M3 because 6 timeframes × 4 MAs = 24 lines is unreadable without it. Done when: unchecking a group removes its every level from the chart and the state survives a reload.

M4 — Confluence + alerts (on moving averages alone) ⭐ first genuinely useful build

confluence.py, alerts.py, confluence panel, ntfy push, in-browser sound — scored over MA levels only. Multi-timeframe MA confluence is a real signal in its own right; this is a complete, useful product with zero hand-drawn input and zero tuning. Done when: a replayed tape produces a sane number of alerts (single digits per session), each corresponding to a real multi-timeframe convergence.

M5 — Manual trendlines

Two-click drawing, snapping, persistence, feeding the same confluence engine (§7.3a). Hand-drawn lines are authoritative — full weight, no quality discount. Done when: a line drawn on the 4h chart appears correctly projected on the 1m chart and raises the confluence score of a cluster it lands in.

M6 — Schwab live source (needs API keys)

Implement market/schwab.py against the existing MarketDataSource protocol and answer every question in §2.2. Yahoo continues to handle seeding; Schwab takes over the live tail. Done when: flipping LIVE_SOURCE=yahoo → schwab changes nothing visible except lower latency and true per-contract prices. If this milestone requires touching any file outside market/, the abstraction in M0 was wrong — fix it there, not here.

M7 — Persistence

SqliteBarStore behind the existing BarStore protocol. Backfill-on-start from disk, falling back to Yahoo seeding only for what's missing. Done when: restarting the process loses no history.

M8 — Automatic trendline detection (optional)

pivots.py, trendlines.py per §7.3. Deliberately last among the analysis work: by this point the hand-drawn lines from M5 are ground truth, so the scoring coefficients can be tuned to agree with lines you actually drew, rather than guessed at in the abstract. Auto lines render in a distinct style and are individually dismissable; they never silently replace a manual line. Done when: on a replayed tape, auto-detected lines land where the manual ones were drawn, and the layer panel can hide them independently.

M9 — Bias panel

BULLISH / BEARISH toggle recording the user's directional call against the current confluence state, persisted, with a journal view. Records only — trades nothing.

Because LEVEL_ONE_FUTURES_OPTIONS streams, this milestone can go further than the original sketch: given a bias, resolve the 1-DTE strikes, subscribe to the two legs, and display the live spread price — so the readout becomes actionable enough to hand-execute in thinkorswim:

BEARISH /ES — resistance confluence 15 @ 6403.75–6405.00
Proposed: 1-DTE 6405/6415 put spread   ~2.35 x 10   (live)

Strike/expiration symbol resolution for futures options is fiddly; treat it as its own sub-task and verify the symbol format against LEVEL_ONE_FUTURES_OPTIONS empirically, the same way M6 verifies /ES.

M10 — VPS deploy (when wanted)

Shared-secret auth on, workers=1, persistent volume for token + DB, Coolify domain port suffix preserved per README.

---

14. Why execution is out of scope

Note the asymmetry: Schwab does stream futures-options quotes (LEVEL_ONE_FUTURES_OPTIONS), so we can price a spread live — we just cannot transmit it. Read anything below as being about order entry only.

Schwab's Trader API exposes no futures or futures-options order entry. thinkScript AddOrder() places simulated orders for backtesting only. Automating clicks in the thinkorswim UI is the wrong reliability model for near-expiration leveraged instruments — window focus, stale quotes, partial fills, and dialogs all fail silently, and an execution path must be able to tell the program what the broker actually did.

The interim workflow is therefore: this app produces a decision, the human executes it in thinkorswim.

Keep the seam clean. Broker-specific code lives only in market/schwab.py; nothing in analysis/, bars/, or api/ may import it. When an execution adapter is added — Schwab, if they ever ship futures-options orders, or IBKR — it consumes Cluster and the M9 bias signal and nothing else. This is the same discipline the M6 acceptance test enforces for data sources.

---

15. Risk register

Risk	Impact	Mitigation
Futures entitlement missing on the Schwab account	Delays M6 only	No longer blocks — M0–M5 run on Yahoo
No futures history from Schwab	Higher TFs cold for months	Solved: Yahoo seeding, §10
Yahoo endpoint is unofficial — may rate-limit or change shape	Dev source breaks	Isolated in market/yahoo.py; cache seeds to disk (M7) so it's fetched rarely; back off on 429
Yahoo daily bars anchored midnight ET, not session	Daily candles disagree with every other chart	Never use them — build 1d from 1h, §2.1
Contract roll gaps fake a trendline break	Bad signals on seeded data	Store symbol + source per bar; surface rolls in UI; §10
Source abstraction leaks Schwab/Yahoo specifics upward	M6 turns into a rewrite	M6 acceptance test: no file outside market/ may change
session.py bucket math wrong	Silently wrong lines everywhere	Tests written first; both DST transitions
Repainting pivots	Lines that "were always there"	w-bar confirmation lag, enforced by test
Alert fatigue	Product becomes unusable	Cluster-level alerts, cooldown + separation re-arm
Multiple uvicorn workers	Duplicate Schwab connections	workers=1; streamer in lifespan; warned in Procfile
JWT signing key derived from the password	A leaked cookie brute-forces the password offline; blocks multi-user outright	Server-side random secret — docs/multi_user.md
Threadpool routes touching asyncio.Queue	Dropped socket wakeups, rare corruption	Post via call_soon_threadsafe — docs/async_refactor.md P0
Level rebuilds run CPU-bound on the event loop	82s startup; a stall every closed bar	Bulk seed, incremental MAs — docs/async_refactor.md P1
Alert disarm writes to disk on the loop	Stream stalls when an alert fires	Offload the write — docs/async_refactor.md P2
Schwab token expiry (7 days)	Stream dies	Surface prominently in status bar; document re-auth
Vue reactivity wrapping chart objects	Perf collapse, odd bugs	shallowRef/markRaw — §9
LWC v4 tutorials copied	Code silently wrong for v5	addSeries(SeriesType, ...) only
Viewport derived from bars.length	Chart looks frozen; feed is fine	Anchor the view by time, never by logical index — §9
Seed replays every bar through on_bar	~82 s startup; port refuses connections	Known, unfixed — §16, 2026-08-10
Headless browser without a real locale	Intl throws; blank canvas mimics an app bug	Launch Chromium with --lang=en-US — §16

16. Session log

Dated record of problems hit and how they were resolved. Times are UTC; the repo's commit timestamps are -0500.

2026-08-10 — rebuild, and a chart that looked frozen

10:30 · The rebuild was genuinely required. schwab-py had been added to requirements.txt, but the running image was built at 2026-08-09 22:05, before that line existed. The bind mount (.:/app) hides this: source edits appear live, so the Schwab commits looked deployed while pip freeze in the container showed no schwab-py at all. Anything imported rather than read from disk needs docker compose build. Rebuilt to schwab-py 1.5.1 and recreated the container.

10:30–10:31 · Startup takes ~82 seconds, and the port is closed the whole time. Runtime.start() replays every seeded bar through on_bar, and each daily-bar update re-runs rebuild_levels() → broadcast_level_delta(), which serialises and diffs five MA levels carrying ~730 points each. With a 730d/1h seed plus an 8d/1m seed that is quadratic work before uvicorn binds. Measured: 10:30:24 "Waiting for application startup" → 10:31:46 "Application startup complete". An open browser tab polling /api/status throughout logs a wall of ERR_CONNECTION_REFUSED; that is the restart window, not a fault. Unfixed. The fix is to bulk-load seeded bars and rebuild levels once at the end, rather than once per bar. Related: M7 persistence would cut the seed itself.

Diagnosing a hang that is actually slowness: docker stats reported ~0.1% CPU while the process was in fact grinding, so it pointed the wrong way. What worked was faulthandler.dump_traceback_later(25, exit=True), which named the exact frame (indicators.py:sma under runtime.py:62). py-spy is unusable here — it needs SYS_PTRACE, which the container does not have.

Do not write scratch files into the repo while diagnosing. A _probe.py dropped in the project root is inside the bind mount, so --reload restarted the lifespan and reset the 82-second clock — twice — which is what made slow startup look like an infinite hang. Pipe throwaway scripts over stdin (docker exec -i … python -) instead. Only .py changes trigger the reloader; writing screenshots into artifacts/ is safe.

10:35 · A blank chart canvas that was not a bug. The Playwright container has no usable locale, so Chromium reports en-US@posix; Lightweight Charts formats its time axis through Intl, which throws Invalid language tag and leaves the canvas empty. docker-compose.yml already sets LANG=en_US.UTF-8 for that service and it is not sufficient. Launch with chromium.launch({ args: ['--lang=en-US'] }) — with that, the page renders and reports zero console errors. Worth stating plainly: this failure looks exactly like a broken app, and it is not.

10:41–10:50 · The real bug — the chart sat ~10 hours behind a healthy feed. Symptom: header price live at 7785.00 while the last candle closed 7772.75, and the series appeared to end at 00:20. Everything downstream checked out — /api/bars newest 10:39 from schwab; store.put keeps bars strictly ascending; the WebSocket snapshot delivered 1000 ascending bars ending 10:42 and live bar events arrived every minute; the browser received all of it. Interrogating window.__chart gave the answer:

seriesLen 1000   seriesLast 08-10T10:48 (7786.25)   ← data complete
visible   08-07T20:41 → 08-10T00:35                  ← viewport wrong
logical   from 840 to 1005

The series was complete; the viewport was 617 bars too far left — exactly bars_held.1d. setBars() set a visible logical range from the candle array length, then syncVisibleLevels() attached the daily MA series, whose 617 daily points pre-date the 1m window; prepending them renumbered every logical index and dragged the view off the live edge. Fixed in static/chart.js by anchoring the viewport to a time range. Verified in a real browser: visible range 08:02 → 10:49, last candle 7786.25 matching the header. See §9.

Method note. Three checks in a row said "healthy" — the REST API, the WebSocket, and the frontend source all looked correct in isolation, because each of them was correct. Only querying the live page's own chart object separated "the data is missing" from "the data is off-screen". Screenshots alone were actively misleading here: the stale time axis was read as a session gap.

2026-08-10 (later) — real-time ticks, and what to do about cold restarts

The chart now moves between minute closes. CHART_FUTURES emits a bar only once its minute is over, so the chart stepped once a minute and sat still in between — read, reasonably, as a dead feed. LEVEL_ONE_FUTURES carries real trades on the same socket (delayed: False, verified on this account back in M6), and it was never subscribed. It is now, and it builds a forming bar for the current minute which the authoritative CHART_FUTURES bar then supersedes.

Three constraints shaped it, each of which would have caused a real bug:

• Tick bars must never reach the aggregator. It accumulates with current.v += incoming.v, so re-sending the same forming minute would add its volume into every higher timeframe on every update. Runtime.on_bar returns early for not bar.closed: store the bar, set the price, broadcast, stop.
• Ticks are throttled (SCHWAB_TICK_SECONDS, default 1.0). /ES trades many times a second and each emission costs a store write plus a broadcast to every open socket. Setting it negative drops the Level 1 subscription entirely and returns the source to closed bars only.
• A tick for a minute already closed is dropped, or a late trade would overwrite a settled exchange bar with a partial one.

Alerts deliberately stay on closed bars. A level is judged on a settled bar, not on a price that may not last the minute — and on_bar already gated on closed, so this needed no change. Intra-bar alerting is a separate decision.

Bid-only Level 1 updates are skipped rather than carried forward: a bid is not a trade and must not extend a candle's high or low. Verified live — 15 forming bars and 2 closed bars in 100 seconds, and in a browser the candle's high and low visibly extend within the minute.

Cold restarts — the options, and a recommendation. Every restart costs ~82 seconds of refused connections, re-seeds from Yahoo, and starts with empty alert cooldowns, so a deploy can re-alert whatever price is sitting on.

1. Make seeding non-quadratic. Seeding replays every bar through on_bar, and each daily-bar update rebuilds all five MA levels and diffs them. Bulk-load the seeded bars and rebuild levels once at the end. Contained, testable, and removes most of the 82 seconds. Do this first — it is the cheapest real win and needs no new storage.
2. Persist bars (M7, SQLite). Restarts then seed only the gap. Removes the Yahoo dependency from the startup path and shrinks the window further. This is the durable answer, and the plan already scopes it.
3. Persist alert cooldowns and armed state. Independent of 1 and 2, and the part that actually misbehaves rather than merely being slow: without it every deploy re-alerts. Small table, big behavioural win.
4. Serve before seeding finishes. Start uvicorn immediately and seed in a background task, so the port never refuses. The chart would open cold and fill in, which is better than an unreachable page — but it changes what "warm" means to every consumer of /api/status, so it wants its own thought.

Recommended order: 1, then 3, then 2. 4 only if the window still bites after 1.

Stale bar events across a timeframe switch. Cannot update oldest data appeared in the console once ticks were live. Switching timeframe races: the server answers subscribe with a fresh snapshot from one coroutine while another is still draining bar events for the timeframe just left, so a 1m bar can land after the 1h snapshot. Applied to the 1h series it is older than every point in it, and Lightweight Charts throws rather than ignoring it — taking the app down instead of dropping one bar. The race predates the tick feed; Level 1 made bar events ~15x more frequent, which is what surfaced it.

Guarded at both ends. app.js honours the tf the event already carries and drops anything for a timeframe that is no longer selected. chart.js refuses a bar older than the series' last point regardless of where it came from — a bar behind the last one has nothing to contribute. Verified: 36 rapid timeframe switches under a live tick feed produce zero errors, and calling candles.update() directly with a stale bar still throws while the guarded updateBar() does not.

Same-price trades were being dropped. The candle still paused for 10–20 seconds at a time after Level 1 went in. Instrumenting the raw stream settled it: 87 messages in 90 seconds, only 33 carrying LAST_PRICE. Most of the rest are pure bid/ask movement and correctly ignored — but a seventh of them look like this:

['ASK_SIZE','ASK_TIME_MILLIS','BID_SIZE','BID_TIME_MILLIS',
 'LAST_SIZE','QUOTE_TIME_MILLIS','TOTAL_VOLUME','TRADE_TIME_MILLIS','key']

Trade time, trade size, cumulative volume — and no LAST_PRICE, because Level 1 sends only changed fields and the trade printed at the price of the one before. Requiring LAST_PRICE threw those away along with their volume. parse_level_one now treats size-plus-trade-time as a trade and returns a null price for the caller to carry forward. Measured on the live feed: median gap 3.1s → 2.0s, worst 21.5s → 8.1s, and bar volume climbs within the minute instead of standing still.

Worth recording for the next person who reads a gap as a bug: the remaining pauses are the market, not the pipe. In thin pre-open tape /ES genuinely goes seconds without a price-changing trade, and then moves several ticks at once — which is what a "gap up" after a quiet spell actually is.

The time axis reads local, the data stays UTC. Lightweight Charts is timezone-agnostic: it reads epoch seconds as UTC and labels them as UTC, which is why the axis disagreed with the wall clock. Fixed with tickMarkFormatter for the axis and localization.timeFormatter for the crosshair, both going through the browser's own zone.

Deliberately not fixed by shifting the bar timestamps, which is the other common recipe. Every time in this codebase is epoch UTC by convention, and the chart's own times feed trendline anchors, indexAt, hit testing and the values posted back for manual lines — an offset applied to the data would put all of them out by the offset, which is exactly the class of bug that once priced a trendline 147 points away.

One limit worth knowing: tick placement is still computed on UTC days, so the day-change divider sits at 00:00 UTC rather than local midnight, labelled with the local date. The labels are right; the divider is in the UTC place.

2026-08-10 (afternoon) — update rate, the left scale, and volume

Schwab conflates Level 1 to one update per second. Chasing "still slow in market hours" ended at a hard ceiling rather than a bug. In regular hours the gaps between updates are whole multiples of 1.005s — 2.01, 3.02, 4.03 — which only happens if the source emits on a one-second cadence and some seconds carry no trade. SCHWAB_TICK_SECONDS was the limiter at 1.0 and is now 0.25, where it no longer binds. One update per second is the source's ceiling. Anything faster would mean inventing prices between trades, which a chart must not do.

Two real losses were found on the way and fixed:

• Higher timeframes only moved once a minute, because tick bars are 1m and the socket filters by subscriber timeframe. Runtime.provisional_higher now combines the aggregator's committed state with the live minute — without mutating it, since the aggregator accumulates volume and would double count.
• Trades carrying only a trade stamp and a moved TOTAL_VOLUME — no LAST_PRICE, no LAST_SIZE — were skipped. 66 → 74 updates per 90s.

Daily context moved to the left price scale. The right had prior-day levels, session VWAP and five daily MAs competing with the live price and hand-drawn intraday levels. The trap: a price scale takes its range from the series on it, so moving levels across draws them against a different range and puts them at the wrong height. A transparent candlestick mirror on the left scale feeds it exactly the right scale's input; verified as a zero-pixel delta between the two. Hand-drawn levels stay right, which is the space being cleared. priceScaleId is fixed at series creation, so it is passed at construction and kept out of the options reapplied afterwards.

Volume is finally drawn. It travelled the entire pipeline — parsed from both Schwab services, aggregated, stored, broadcast in every bar — and nothing rendered it. Now an overlay histogram on its own hidden scale in the bottom fifth. An overlay rather than a pane, and emphatically not the price scale: volumes are five figures against four-figure prices, and sharing a scale would flatten the candles to a line.

Sidebar vertical space. Three cuts, all in §9.4's layer panel. The daily MA periods sit on one line — flex-wrap:nowrap with tighter gaps and 12px boxes, where 10px gaps and 22px indent had pushed 200 onto a line of its own. Auto trendlines joins Manual lines as a parenthetical (auto) rather than owning a row, which suits a control that is disabled until M8. The alert log becomes a <details> like Confluence zones, closed by default with its count in the summary — collapsed by default is the point, since leaving it open would save nothing, and the count means activity is still visible while closed.

Sidebar vertical space. The right column was taller than the viewport with nothing selected. Five changes, no functionality removed:

• The five daily MA periods fit one line (flex-wrap:nowrap, tighter gaps, 12px boxes); 10px gaps and a 22px indent had pushed 200 onto a row of its own.
• Auto trendlines becomes a parenthetical (auto) on the Manual lines row rather than owning one, which suits a control disabled until M8.
• Alert log becomes a <details> like Confluence zones, closed by default with its count in the summary so activity still shows while shut.
• Tools becomes a <details> too, open by default, and each tool's panel is bound to armedTool — only the armed tool shows its label, colour, width and side controls. armTool already toggles and permits one armed tool at a time, so the panels follow it exactly.
• Order is Layers, Tools, then the rest, with Layers collapsed by default.

Measured with nothing armed: 1110px of content down to 900px, which is inside the viewport rather than past it. .sidebar-section:first-of-type carries the zeroed top margin so reordering cannot reintroduce a gap at the top.

2026-08-10 (evening) — chart comments, and Drawings

Comments are drawings, not levels. A comment is stored as a ManualLine with kind="comment", so it inherits persistence, the shared drawing-number sequence, the sidebar list, filtering and deletion without a parallel set of endpoints. The one rule that must never bend: ManualLineStore.levels() filters comments out. A comment reaching the level list would join a confluence cluster and push a phone notification about a piece of text. It is also created with armed=False, and PATCH /lines/{id} returns to_dict() rather than to_level() for one, so no caller is ever handed a level-shaped comment.

kind is derived when absent — zero slope was always a typed level, anything else a drawn trendline — so drawings saved before comments existed keep working.

Pinned or floating. Pinned comments carry anchor_t/anchor_p and move with the chart; floating ones carry x/y as fractions of the pane, hold their place through any zoom, and can be dragged. Comments render as DOM rather than canvas: they hold arbitrary text, collapse to a numbered dot, and a floating one has to ignore the time scale entirely. A pinned comment scrolled out of view parks on the edge it left, pointing back the way it went, so it is never simply lost.

"Lines & levels" becomes "Drawings", filtered by type and by text — the text match covers the label, the kind and the #number, so comment, cpi and 7 all narrow the list. Delete acts on what the filter shows, which is what makes deleting by type or by string a single button.

One CSS trap worth recording: .trendline-row span { grid-column:2 } captured the comment row's icon span and dragged it into the text column. Scoped to span:not(.drawing-icon).

A comment lost its place when the timeframe changed. Placed on a 30m bar, then switched to 15m, it slid to the far left. timeToCoordinate answers only for times that are data points on the current series, so a 30m bucket start returned null on another timeframe — and null was being read as "off the left edge". Anchors are now resolved to the bar that contains them, which is timeframe-independent: an 09:30 note sits on the 09:30 bar at 15m and on the 09:00 bar at 1h. setBars also re-renders comments, since a timeframe switch replaces the grid underneath every pinned one.

Verified across 30m → 15m → 1h → 30m: the anchor stays 08:30 throughout, resolving to the 08:30 bar on 15m and the 08:00 bar on 1h, never edge-parked, and returning to its original x on the way back. Edge-parking still works where it should — a comment scrolled 400 bars out parks right and comes back on return to live.

The trendline Side control became inert. Once snapping always lands on a bar extreme, the side is inferred from which extreme — a high is resistance, a low is support — so the dropdown could no longer affect anything. It now appears only when "Snap to highs/lows" is off, which is the one case where there is no extreme to infer from; otherwise the row reads "Side auto". Verified both ways: snap on shows the note and no dropdown, snap off shows the dropdown.

created_at (epoch seconds) is already stored on every drawing and returned by GET /api/drawings, so filtering by age needs UI only, not a migration.

Trendline placement, third pass — and a regression I shipped. Making a pending anchor always win (previous entry) fixed the twitch case and broke the opposite one: a genuine press-drag begun after an abandoned click was hijacked by that stale anchor, so the line started far from the drag. That reached production. The rule is now a single threshold — 12px of travel between press and release makes it a drag, which is wide enough to survive a twitch on a deliberate click and unambiguous for a real drag. A drag clears any half-placed anchor rather than silently adopting it.

The crosshair was lying about the anchor. Lightweight Charts defaults to CrosshairMode.Magnet, which snaps the crosshair to the bar's close. Hovering by a bar's low therefore drew the crosshair mid-bar, and a correctly-snapped anchor looked wrong — measured: aiming 4px above a bar low placed the anchor at the low (7773) and not the close (7773.25), while the crosshair sat at the close. Arming a tool now switches the crosshair to Normal, and a snap dot marks the exact point the anchor will use, coloured by the side it implies.

Four gesture paths are verified in a browser: two clicks with a twitch on the second, an abandoned click followed by a real drag, a plain press-drag, and hovering. All start where they should and land on a bar extreme.

Worth recording for diagnosis: a reported "line ended up high off the bar" turned out to render exactly on its bar — zero pixels off at 1h, 30m and 15m — because the anchor had snapped to the drawn timeframe extreme (the 09:00 1h low, 7744.25) while being checked against 1m bars, where it matches neither extreme. Always compare an anchor against the timeframe it was drawn on.

A zero price wrecked every timeframe's scale. A LEVEL_ONE_FUTURES update arrived with LAST_PRICE: 0. The parser rejected None but 0 is not None, so a minute opened at zero — o=0.0 h=7777.25 l=0.0 — and provisional_higher carried that low into 5m, 15m, 30m, 1h and the daily bar, flattening the price scale everywhere. Non-positive prices are now treated as absent, so the last real price carries forward, and the tick still counts as a trade.

The exchange's own bars were being dropped. store.put replaced a bar only when it matched the tail. That held while one closed bar arrived per minute, but ticks open the next minute before CHART_FUTURES delivers the previous one — so the authoritative bar no longer matched the tail and was discarded, leaving the tick approximation and its partial volume in place permanently. put now searches back a bounded number of buckets, and refuses to let a provisional bar overwrite a settled one.

Both were introduced by the tick feature and both are covered by tests: a zero price parses as a trade with no price, a late closed bar replaces its bucket and keeps the exchange's volume, and a tick cannot overwrite a settled bar.

Snapping now measures distance on screen, not in time. The rule was "take the bar sharing the cursor's time, then its nearer extreme", which ignored how far that extreme actually was. Pointing anywhere below a candle snapped to that candle's low however distant, and the extreme genuinely under the cursor was never considered — so zoomed out to ~360 bars at three pixels each, hitting the intended bar took several attempts. snapPoint now scans six bars either side and picks the extreme nearest in pixels. Proven by probe: with the cursor on one bar's low but nudged two pixels so coordinateToTime resolves to its neighbour, the snap takes the extreme under the cursor rather than the neighbour's.

Worth recording because it was misdiagnosed twice: a report of "the snap dot appears way above the bar" was, on the numbers, the dot landing correctly on the bar's low while the cursor sat 151 points below it. The right price scale keeps a bottom: 0.1 margin and the volume overlay is drawn in it, so the lower fifth of the pane is below every candle — an inviting place to point that contains no price action at all.

e2e tests

bin/e2e runs tests/e2e/*.test.mjs inside the playwright service against the dev stack. Node's built-in test runner, no dependencies added to this repo: Playwright is global in that container and tests/e2e is mounted at /repo/tests/e2e. Every case in there is a bug that shipped — the viewport parked ten hours back, hourly candles drawn as slivers, stale bar events throwing, comments drifting on a timeframe switch, and three separate ways a trendline anchor could disagree with its own preview. None of them could have been caught by pytest, which is the argument for the suite existing.

Tests clean up after themselves: withChart records the drawings that exist before the body runs and deletes anything new afterwards, because the dev store is shared with whoever is using the app. Select by title rather than class when asserting on chart overlays, for the same reason.

The snapping rule, stated once

x picks the bar, y picks which extreme. That is the whole rule. It is written here because changing it reactively three times is what made trendlines feel broken, not any inherent difficulty:

1. An 8px proximity gate meant a cursor between the high and the low snapped to neither, so the anchor kept a raw mid-bar price and the side silently fell back to the dropdown.
2. Removing the gate fixed that. Then a nearest-in-2D search was tried, to make a bar easier to hit when zoomed out — and broke sweeping along the bottom, because whichever nearby bar had the lowest low won on total distance and the dot skipped off the bar under the cursor. Reverted.
3. What actually made it feel wrong was never the rule: the crosshair was in Lightweight Charts' default Magnet mode, snapping to the bar's close, so the feedback pointed somewhere the anchor would never go. It is Normal everywhere now, with the snap dot showing the real target.

An e2e test sweeps the cursor along the bottom of a zoomed-out 1h chart and requires every position to land on the low of the bar beneath it — 115 of 115. That test is the rule, executable.

The snap leapt to the live edge — found by diagnostic mode. Reported from the user's own browser, which no headless run had reproduced:

cursor_x 1409.0  chart_w 1280.0  -> cursor_t None -> snapped to the last bar
cursor_x 1161.0  chart_w 1280.0  -> cursor_t None -> snapped to the last bar
cursor_x 1128.0                  -> valid time, drift 0 bars

coordinateToTime answers null over the right-hand price axis, over the whitespace past the last bar, and anywhere outside the chart — and snapPoint read that as "the newest bar", so the dot jumped to the live edge from wherever the cursor was. Two faults behind it: the tool's pointer listener is on window and therefore fires over the sidebar (x=1409 on a 1280-wide chart), and a null time meant a default rather than no answer.

Now a pointer outside the plot hides the indicator entirely, and a null time resolves to the bar nearest in pixels rather than the newest one. Covered by an e2e test that hovers a bar, the axis, the sidebar, and back.

The lesson is about method rather than geometry: four hypotheses were tested and killed by measurement here — device pixel ratio, viewport size, resize desynchronisation, and the chart scrolling under the gesture — while the actual cause was visible in one line of the client's own numbers. When the browser is on another machine, instrument it early instead of reproducing locally.

Overlays are positioned against the plot, not the element

The trendline snap was 66 pixels out, and so was everything else drawn over the chart. Lightweight Charts reports coordinates from the plot area's origin. The chart element also contains the price scales, so once the left scale was enabled for the daily labels, the plot started 66px into the element — and every overlay positioned with left: against the element was displaced by exactly that much, in both directions at once:

• the cursor's element-x was read as a plot-x, resolving a bar ~66px to the right of the pointer;
• the indicator was then drawn at that bar's plot-x interpreted as element-x, landing ~66px left of where the bar is painted.

Not near the cursor, not near the bar, and varying with zoom — 66px is a couple of bars at 30m and a dozen at 1m, which is why it looked random rather than offset. Every diagnostic number agreed with itself throughout, because dot_y, expected_y and bar_low_y all derive from the same API and shared the same wrong origin. Self-consistent instrumentation cannot see a systematic error in its own frame of reference.

All overlays now live in one container positioned over the plot canvas, so they inherit plot coordinates untranslated: the snap dot and label, the comment layer, the trendline anchor handles, the preview line, the tooltip, the price tag and the context menu. eventPoint subtracts the same offset, so a pointer position and a chart coordinate finally mean the same thing. The container is repositioned on resize.

This had been mis-diagnosed for hours: device pixel ratio, viewport size, resize desynchronisation, the chart scrolling under the gesture, and the dead band below the candles were each measured and ruled out. The measurement that found it was comparing canvas.width to element.clientWidth — 0.894 — which is the first thing that ever disagreed with itself.

---

Handoff: the snap indicator is ~10px left of where it belongs

Status: fixed. Everything below is measured, not inferred.

The symptom

With the Trendline tool armed, the snap dot sits about one bar to the left of the cursor. The height is correct and the bar it chooses is correct — only the horizontal drawing position is wrong. Reported from a real browser and reproduced headlessly.

The measurement

bar spacing            6.96 px
dot centre - cursor  -10.5 px   (= 1.5 bars at that zoom)
chosen bar            correct   (label names the bar under the cursor)

And the cause, from ConfluenceChart.syncOverlayLayer():

at load:          containerLeft 56   true plot offset 66    <- 10px stale
after a re-sync:  containerLeft 66   true plot offset 66    <- correct

Why

Lightweight Charts reports coordinates from the plot area's origin. The chart element also contains the price scales, so with the left scale enabled the plot begins 66px in. All overlays therefore live in a container (.chart-overlays) positioned over the plot, so they can use chart coordinates untranslated — see create() and syncOverlayLayer() in static/chart.js.

syncOverlayLayer() runs once in a requestAnimationFrame during create(). At that moment the left price scale has not finished sizing itself to its label text, so the measured offset is 56. It settles at 66 once labels render, and nothing re-measures. The container stays 10px left of the plot for the life of the page, which drags every overlay with it: the snap dot and label, comments, trendline anchor handles, the preview line, the tooltip and the price tag.

Only x is affected. y never passes through this offset, which is why the height has always looked right.

The fix to write

Re-measure instead of measuring once. Options, cheapest first:

1. ResizeObserver on the plot canvas — fires when the scale settles and on every later change. Probably the right answer.
2. Call syncOverlayLayer() at the top of renderComments() and showSnapDot(). Correct but does DOM reads on every mouse move.
3. Re-sync on subscribeVisibleLogicalRangeChange as well as on resize. Cheap, but misses a scale that widens without the range changing.

Beware: the left scale's width depends on its label text, so it changes when the price range gains a digit or a longer level label appears. Whatever you choose must survive that, not just the initial load.

How to verify

./bin/e2e trendline          # 7 cases, all currently pass — they do not catch this

The suite misses it because its assertions go through the same coordinate API that carries the error. Add a test that measures in page pixels: place the cursor exactly at a bar's centre and assert the dot's centre is within ~2px horizontally. The reproduction is:

const r  = el.getBoundingClientRect();
const bx = chart.timeScale().timeToCoordinate(bar.t);
await page.mouse.move(r.x + c.plotOffsetX() + bx, r.y + c.candles.priceToCoordinate(bar.l) - 6);
// dot centre x should equal the cursor x; today it is ~10px left

Live numbers from the client are available without a console: open the chart with ?diag=1, then docker compose logs api | grep SNAPDBG. Note that SNAPDBG will not show this bug — dot_y, expected_y and bar_low_y all derive from the same API and share the same origin, so they agree with each other while being wrong together. The error is only visible by comparing against something outside that frame of reference: canvas.getBoundingClientRect() against element.getBoundingClientRect(), or painted pixels.

Context worth having

• window.__chart is a deliberate debug handle exposing the wrapper.
• The dev stack is at http://localhost:8010, and http://api:8000 from inside the playwright container. It runs on a remote machine; the user's browser does not. Headless passes prove little about their screen — see "Where things run" in AGENTS.md.
• Related history is above under "Overlays are positioned against the plot, not the element", which fixed the 66px case this 10px residue survived.
• One e2e test, "clicking a comment collapses it", is flaky (roughly one run in three) and unrelated. Worth fixing before trusting the suite.

Resolution

ConfluenceChart now attaches a ResizeObserver to the plot canvas itself. Unlike the outer chart element, that canvas changes width when Lightweight Charts finishes sizing the left price scale or a longer price label appears. The observer repositions the shared overlay layer and redraws its anchored DOM content after each such change.

The trendline suite now compares the snap dot and cursor in page pixels, then forces a six-digit left-scale label and repeats the assertion. This catches the stale coordinate frame that the chart API's self-consistent coordinates could not. The comment-collapse flake was also fixed: a broad CSS rule had re-enabled pointer events on the full-size handles SVG, which intermittently covered a comment. Overlay surfaces now ignore pointers unless the actual control opts in.

2026-08-11 02:55 CDT — test-only coverage, mobile plan and feed research

Added regression coverage without changing production code. Pytest grew from 96 to 103 cases: Yahoo H1 seed bars must form the correct CME-session daily OHLCV, manual alerts must remain disarmed through rebuild and restart, daily anchors must follow Eastern DST in UTC, and two WebSocket clients must keep their layer preferences isolated from one another and from global alert state. The browser suite grew from 15 to 19 cases: partial or malformed persisted preferences must still boot, volume must remain paired with candles on its own scale across timeframe changes, and Backspace/Delete in a rename input must edit text rather than delete the drawing. Full results: 103 backend and 19 browser tests passing.

One proposed test exposed a current defect and was deliberately not committed as a failing test: withinPlot() compares against the outer chart element, which includes the right price axis. Hovering that axis can therefore leave the snap dot visible at the plot edge. The production fix is to compare plot-relative coordinates against the measured plot canvas width and height, then add the page-pixel price-axis case. Startup rebuild-count coverage is also deferred until the bulk-seeding optimization exists; a duration assertion against today's slow startup would encode the problem rather than protect a fix.

Local and deploy test commands now live in README. Browser E2E stays local because it creates and deletes drawings; production verification uses bin/wait-deploy, /api/health, /api/version, and an authenticated /api/status smoke check.

The mobile audit is recorded in docs/mobile_enhance.md. Its recommended first steps are one coordinate path for every gesture, touch-sized invisible hit areas, persistent first-anchor feedback, and a sticky mobile tool rail before more ambitious gesture changes.

No durable unauthenticated source of free real-time CME /ES data was found. Schwab remains the verified entitled live source and Yahoo the practical delayed seed source. Tastytrade/dxLink is the best free-with-broker-account candidate to test next; IBKR is the strongest low-cost fallback rather than a free one. CME, TradingView and Barchart free pages are delayed displays, not licensed backend APIs. For freshness UI, existing bar WebSocket messages can supply browser receipt time without another call, but exact trade time and source heartbeat are not retained yet. The best placement is the existing status strip below the chart, with a compact mobile form such as Updated 3s ago · Live.

2026-08-11 03:19 CDT — make recommendations discoverable

Added docs/NEXT_STEPS.md as the concise home for deferred production fixes, freshness UI semantics, mobile priorities and market-data alternatives. Linked it, the implementation/session log and docs/mobile_enhance.md directly from AGENTS.md, which every coding agent reads before working in this repository. This keeps current advice visible without turning the historical implementation plan into an undifferentiated backlog.

2026-08-11 03:39 CDT — one price scale, left labels and editable-line snapping

The left side was introduced to keep daily moving-average, VWAP and prior-day labels away from the intraday labels on the right. That decluttering intent was correct; implementing it as a second Lightweight Charts price scale was not. Each scale autoscaled independently once its own overlays were attached, so the same numeric price could occupy a different y-coordinate on each side. Daily and intraday structure then looked directly comparable while being geometrically unrelated.

All price-bearing series and flat levels now share the candle series' right scale. Long-term labels remain on the left as DOM overlays whose y positions are computed through candles.priceToCoordinate(), so they preserve the intended separation without creating another coordinate system. Context series suppress their built-in right-side titles. A browser invariant verifies every overlay's scale id, every flat level's host, and sub-pixel agreement between an MA price and the candle coordinate for that same price.

Editing a selected trendline had a separate defect: handle dragging used element-relative x, snapped only to the nearest bar time, and retained a raw cursor price. Initial placement used plot-relative coordinates and high/low snapping, so moving an anchor could visibly jump off the bar and change the line to an unusable slope. Placement, selection, handle dragging and context actions now all use eventPoint() against the measured plot canvas. Handle movement passes through the same snapPoint() high/low rule and displays the same snap feedback as placement. This also closes the known right-price-axis containment bug; a page-region test now proves the dot disappears over the actual axis and returns over the plot.

The 30-minute chart had only about one week of history because it was derived solely from Yahoo's eight-day 1-minute seed. The 730-day hourly seed cannot reconstruct half-hour candles. Yahoo's native 30m interval was verified live at range=60d (2,843 bars on 2026-08-11), so startup now seeds H1/730d, M30/60d, then M1/8d. The finer minute aggregation replaces the recent overlap; the native feed supplies older 30-minute bars. Regression tests pin both Yahoo's native interval request and the startup seed sequence. Final verification: 105 backend tests and 22 browser tests passing.

Daily moving averages had also been rendered with WithSteps on every base timeframe. Holding a daily value constant is correct when projecting it over intraday candles, but on the daily chart it made the SMA itself look like a staircase. MA line type is now timeframe-aware: stepped on intraday charts and simple point-to-point lines on 1d, with both modes pinned by browser coverage.