Two things kept the chart quieter than the feed.
Higher timeframes only moved once a minute. Tick bars are 1m and the socket
filters bar events by the subscriber's timeframe, so on the hourly chart every
tick was discarded and only a closed minute passing through the aggregator
showed up. They cannot simply be fed to the aggregator — it accumulates with
current.v += incoming.v, so the same forming minute re-sent on each tick would
add its volume to every higher timeframe again and again. provisional_higher
combines the aggregator's committed state with the live minute instead, without
mutating it; the next closed minute goes through normally and replaces the
result, because the store keys on the bucket timestamp. A test pins the
behaviour: five ticks in one minute leave the hour's volume at closed plus live,
counted exactly once.
Trades known only by their volume were skipped. Level 1 resends only changed
fields, so some trades carry a trade stamp and a moved TOTAL_VOLUME with neither
LAST_PRICE nor LAST_SIZE. Those now count, with size left at zero rather than
guessed from the volume delta — CHART_FUTURES replaces the minute's volume with
the exchange's own figure moments later, and two ways of counting the same
trades is how double counting starts. Measured: 66 to 74 updates per 90s.
The tick throttle drops to 0.25s, which no longer binds. Measured in regular
hours the gaps between updates are whole multiples of 1.005s — 2.01, 3.02,
4.03 — which is Schwab conflating LEVEL_ONE_FUTURES to one update per second
per symbol. One per second is the source's ceiling, not ours; the longer gaps
are seconds in which their feed carried no trade.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CHART_FUTURES emits a bar only once its minute is over, so the chart stepped
once a minute and sat still in between, which reads as a dead feed.
LEVEL_ONE_FUTURES carries real trades on the same socket and the same login — no
extra REST call, no extra rate limit — and reports delayed: False on this
account. It was verified back in M6 and never subscribed to. It is now, building
a forming bar for the current minute that the authoritative CHART_FUTURES bar
then supersedes.
Three constraints shaped it, each a real bug avoided:
- Tick bars never reach the aggregator. It accumulates with current.v +=
incoming.v, so re-sending the same forming minute would add its volume into
every higher timeframe again on every update. Runtime.on_bar returns early for
an unclosed bar: store, set price, broadcast, stop.
- Emissions are throttled, SCHWAB_TICK_SECONDS default 1.0, because /ES trades
many times a second and each emission is a store write plus a broadcast to
every open socket. Negative drops the Level 1 subscription entirely.
- A tick for a minute CHART_FUTURES has already closed is dropped, or a late
trade would overwrite a settled exchange bar with a partial one.
Bid-only updates are skipped rather than carried forward: a bid is not a trade
and must not extend a candle's high or low. Alerts stay on closed bars — a level
is judged on a settled bar, not a price that may not last the minute — which
needed no change, since on_bar already gated on closed.
Verified against the live socket: 15 forming bars and 2 closed bars in 100
seconds, the closed bar superseding each forming minute. Verified in a browser:
the last candle's high and low visibly extend within the minute, no console
errors. 85 tests pass, four of them new.
The plan gains the cold-restart options asked for: make seeding non-quadratic
first, then persist cooldowns, then persist bars.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Verified against a live account before and after writing it. CHART_FUTURES
delivers one true-OHLCV minute bar per symbol per minute, LEVEL_ONE_FUTURES
reports delayed: false, and consecutive bars arrived sixty seconds apart through
the production code path.
Yahoo stays. Schwab serves no futures history whatever, so seed_source resolves
to Yahoo even when SEED_SOURCE=schwab is asked for — the pairing is the intended
configuration rather than a fallback. The symbols differ, ES=F against /ES, so
Settings.live_symbol picks the live one while seeding always uses Yahoo's.
Three findings worth keeping, each of which cost a round trip:
- get_quote() singular returns the wrong instrument entirely. It puts the symbol
in the URL path, where the leading slash is normalised away, so /ES resolves to
Eversource Energy at $72 and returns HTTP 200 with a populated body. Only
get_quotes() plural, which passes symbols as a query parameter, returns the
future. A 200 is not evidence; assetMainType is.
- Streaming requires the Accounts and Trading product. StreamClient.login() reads
/trader/v1/userPreference for its socket URL, and that path does not exist in
Market Data Production.
- /ES resolves to the active contract on Schwab's side, so the contract roll
handling the plan left open needs no code.
The stream drops the oldest queued message rather than stalling the socket, and
surfaces a dead pump task instead of waiting forever on a queue nothing fills.
schwab-py moves into requirements.txt, imported only when LIVE_SOURCE=schwab.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
config.py had no Schwab fields at all, so the keys listed in .env.example were
being silently dropped by extra="ignore". They exist now, blank, and nothing
reads them while live_source is yahoo.
The token path moves under data/, which is the Coolify persistent volume. Left
at the repository root it would vanish on every rebuild, and re-authenticating
is an interactive browser flow, not something a deploy can do for itself.
scripts/check_schwab.py answers empirically what the app is entitled to rather
than inferring it from documentation: whether the credentials authenticate,
whether /ES quotes return (futures market data is a separate entitlement from
equities), and whether the streamer bootstrap responds.
That last one is the decision. StreamClient.login() reads
/trader/v1/userPreference for its socket URL and credentials, and that path
belongs to the Accounts and Trading product — so an app registered for Market
Data Production alone cannot stream, and CHART_FUTURES is unreachable until the
app adds it. The script reports which of the two paths is open instead of
leaving it to be discovered halfway through an implementation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The trendline bug: a line continued past its second anchor at a different
slope. Two conventions were fighting, and both were wrong.
Lightweight Charts spaces bars evenly however much time separates them — a
weekend is forty-nine hours and one bar wide. The renderer extended the line by
interpolating between bar indices, which looked straight but disagreed with the
server, since price_at() advances per second. Measured on real bars that reached
147 points: the chart drew a level the alerts did not believe in. Making the
renderer match price_at() fixed the disagreement and made the visible kick worse,
because now the line really did climb an hour's worth of slope across a one-bar
maintenance break.
Neither convention is what a person means by drawing a line. A trendline advances
per bar, so both sides now evaluate in bar space: a new bar_space module the
runtime uses to position sloped levels, mirrored by indexAt() in the chart. The
line is straight on screen and the alert fires where it is drawn.
Also, from testing against the live chart:
- A plain click with the trendline tool armed did nothing and left the tool
armed, so the next click began a new line — which is how the slope change was
first noticed. Click-click and press-drag-release are both supported now, with
the rubber band following the cursor between clicks.
- Hand-placed levels are armed, fire once, then disarm themselves, and can be
re-armed from the sidebar. Verified end to end: created armed, tripped within
thirty seconds, disarmed, re-armed.
- Layers is collapsible.
- The 1h moving averages are gone; only the daily set remains.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It was never in the enabled timeframes, so it held zero bars and produced no
levels, but it still carried weight 8 in the scoring table and forced
bucket_start to special-case a wall-clock ET anchor whose entire purpose was
surviving DST transitions. That was the most intricate logic in session.py,
maintained for a timeframe nobody used.
Daily is now the only session-anchored bucket, which is a much easier rule to
state and to keep correct. The DST parametrised tests go with it; the Sunday
open and daily boundary cases remain.
Manual-line tests move to 1h, so the weight assertions drop from 8 to 4.
The plan document keeps its 4h examples — rewriting a dozen illustrative
sentences would churn more than it clarifies — but the timeframe-roles section
now records the removal so nothing reads as a spec to build.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The confluence engine had nothing to work with. Daily moving averages were the
only level source, and they sat 163 to 697 points from price, so every cluster
had exactly one member and no alert could ever fire.
Two new sources, chosen for having a real following — the engine is a bet that
many participants watch the same price, which is what makes a level hold:
- Prior day high/low/close, from the last *closed* daily bar so mid-session the
levels do not silently switch to today's own developing range. Full daily
weight rather than the 0.75 average discount: a traded high is structure, not
a derived average.
- Session VWAP, anchored to the 18:00 ET open like the daily bars. Institutional
execution is benchmarked against it, and zero-volume overnight minutes are
skipped rather than dividing by zero.
Both are stamped 1d, so they get their own colours to stay distinguishable from
the daily averages. Prior-day levels draw as price lines, which span the chart
and label the axis instead of relying on bar-index interpolation.
VWAP re-prices every minute while a daily average carries hundreds of points and
changes once a session, so broadcasting the whole level set on the VWAP cadence
would have pushed the entire history every minute. Levels now go out as a delta
that clients merge by id.
Adding the levels then exposed two defects that had been invisible while nothing
could cluster:
- Cluster identity was sha1(side + round(center / tolerance)), and tolerance
derives from ATR, so it changed every bar. The same zone was continually
issued a new id, never matched the cooldown table, and the cooldown did
nothing. Identity is now the set of converging levels.
- Alert suppression keyed on that identity, so a level drifting in or out of a
group read as a new zone. It now suppresses by proximity: two zones within an
ATR are the same zone, and the strongest is the one reported.
Over six replayed sessions at threshold 28 that is 247 alerts, then 54, then 40;
raising the cooldown to 4h — which only affects repeats of the same area, never
a genuinely new zone — gives 17 total with a worst session of 9.
calibrate_alerts.py now sweeps threshold and cooldown together in one pass,
since the threshold turns out to be quantised and nearly useless as a control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
