Trendlines project into the whitespace beyond the newest candle, but the time
axis stopped there, so a converging pair could be seen without knowing when it
converges. Lightweight Charts only labels times present on its scale, so the
chart now carries whitespace points past the last bar: no value, nothing drawn,
but the axis has something to label and timeToCoordinate answers out there.
Future times repeat the most recent bar interval, which is what timeAtIndex
already does for the projections themselves. That drifts across the daily halt
and the weekend; agreeing with the projected line matters more than abstract
accuracy, and session-accurate projection needs server-side session rules the
client does not have.
Two bugs surfaced while measuring it. Padding meant to be five bars measured as
sixty-seven, because the interval came from the gap between the final two bars;
barInterval now takes a median over recent bars and ignores a ragged tail.
And that gap was two seconds on a one-minute chart because Yahoo stamps its
in-progress candle with the time of the request, while the poller emitted
anything newer than the last thing it sent. Every poll therefore appended a new
"1m" bar seconds after the previous one, interleaved with the real ones — live
in production, which is still on Yahoo. Timestamps are bucketed on parse, and
the final candle is emitted unclosed so it revises the current minute rather
than entering the aggregator and adding its volume to every higher timeframe
again on each poll. Verified live: eight consecutive bars, all aligned, all
sixty seconds apart.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Producing a real alert to check the wiring showed #47 twice in one Events row:
once as the badge the browser draws from the `number` field, and again at the
start of the message, because the number had been prefixed onto the shared
string.
ntfy carries plain text and has nowhere else to put a number or a timestamp, so
those belong in a push body built for it. The browser already receives `number`
and `at` as fields and formats its own local time, so its message stays clean.
Alert now carries both: `message` for a screen, `push` for a phone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Alerts get a number, assigned server-side and shown in both the push and the
Events list, so a notification on a phone can be matched to a row on a screen
when several fire together. It could not come from the browser: that counter
restarts on reload and differs between tabs. It is persisted next to the
cooldown state, because numbering restarting after a deploy would collide with a
phone's existing notification history — which changed that file from a list to
an object, with the loader still reading the old shape.
Pushes now carry a timestamp in the configured zone rather than the server's.
ALERT_TIMEZONE defaults to America/Chicago; containers run UTC, and a push
reading 02:14 to someone seeing 21:14 costs a translation every time. The
browser already formats its own times locally and is unchanged.
Confluence zones are one line each, ordered by price rather than by proximity,
so the list reads top to bottom the way the chart does and all of them fit on
screen — sixteen zones in 394px, about 25px each, where each previously took a
four-line block. Ordering is a display concern only: the server still returns
them nearest-first, which is what the alert path wants.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Uploading a capture required a token; retrieving one did not. That was a
deliberate capability-URL design with a test asserting it, and the reasoning
held: it lets whoever is debugging fetch a capture without the chart password.
Changed because of what a capture contains. getDisplayMedia returns a picture of
someone's screen, and preferCurrentTab is a preference rather than a constraint,
so a mis-click shares a different window. An unguessable id stops guessing but
not leakage: capability URLs escape through proxy logs, browser history and
pasted links.
Retrieval now uses the dependency the rest of the API uses, which already
accepts the session cookie — so a logged-in browser needs nothing extra, which
was the condition for making this change at all. An agent on the server reads
the capture directory directly; one working over HTTP sends the API token.
Both handlers moved from meta.py to routes.py. meta.py is the deliberately open
router — health, version, login, logout — and a screenshot endpoint did not
belong there. The existing test now asserts 401 without credentials, and a new
one covers the browser path: log in, then retrieve with only the cookie.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
P0 from docs/async_refactor.md. The mutating routes are sync `def`, so FastAPI
runs them in a threadpool, and they reach Runtime.broadcast through
rebuild_levels — writing asyncio.Queue directly from there. That queue is not
thread-safe: it wakes a consumer by resolving a Future, which only the loop
thread may do. A dropped wakeup means a drawing made in one browser does not
reach another until the next market tick.
broadcast now posts through call_soon_threadsafe when it is off the loop, and
publishes directly when it is on it, so the stream's own path pays nothing.
Worth being straight about the tests: the race is timing-dependent and did not
reproduce in twenty attempts — a foreign-thread put_nowait usually lands in the
ready queue before the loop sleeps, and a tick every second covers the rest.
Even asyncio's debug thread-affinity check stays quiet unless a consumer is
parked on the Future at that instant. So the tests assert the contract rather
than provoke the failure: a broadcast from a worker thread must go through
call_soon_threadsafe, one from the loop must deliver synchronously, and both
must arrive.
Also adds the loop-lag probe, which reports scheduling drift as loop_lag_ms on
/api/status. It found P1 on its first run: 19,441ms worst against 1.5ms in
steady state, which is seeding blocking the loop. "The chart feels laggy" is now
a number.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Duplicate and delete from the line context menu.
- Copies shift ten bars right.
- Default names are up and down; copies become up 2, down 2, etc.
- Exact local data receipt time including seconds.
- Deployment timestamp removed.
- Test cleanup no longer deletes drawings created from your browser.
- JWT password session flow.
The trendline snap indicator was 66 pixels out, and so was every other overlay.
Lightweight Charts reports coordinates from the plot area's origin. The chart
element also contains the price scales, so enabling the left scale for the daily
labels moved the plot 66px into the element — and each overlay positioned with
left: against the element inherited that error twice over. The cursor's
element-x was read as a plot-x, resolving a bar about 66px right of the pointer;
the indicator was then drawn at that bar's plot-x interpreted as element-x,
landing 66px left of where the bar is painted. Neither near the cursor nor near
the bar, and scaling with zoom — two bars at 30m, a dozen at 1m — which is why
it read as random rather than as an offset.
Every diagnostic number agreed with itself the whole time, because dot_y,
expected_y and bar_low_y all come from the same API and shared the same wrong
origin. Instrumentation cannot see a systematic error in its own frame of
reference; what found this was comparing canvas.width to element.clientWidth and
getting 0.894.
Overlays now live in one container positioned over the plot canvas and inherit
plot coordinates untranslated — snap dot and label, comments, anchor handles,
preview line, tooltip, price tag, context menu — and eventPoint subtracts the
same offset so a pointer position and a chart coordinate mean the same thing.
The container follows the plot on resize.
Verified in page pixels rather than through the coordinate API: with the cursor
placed 30px below a known bar's low, the dot lands 30px above the cursor, on
that low, labelled 7789.00 L against a bar low of 7789.
Device pixel ratio, viewport size, resize desynchronisation and the chart
scrolling under the gesture were each measured and ruled out before this. The
e2e helper computed expected times from element-relative x, the same mistake in
the tests, and is corrected here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Diagnostic mode found in one hover what four local hypotheses could not.
coordinateToTime answers null over the right-hand price axis, over the
whitespace past the last bar, and anywhere outside the chart. snapPoint read
that as "the newest bar":
const fallbackT = point.t ?? this.bars[this.bars.length - 1]?.t ?? null;
so the indicator jumped to the live edge from wherever the cursor actually was —
hundreds of points away, on a bar the user was nowhere near. It matches the
screenshot exactly: the crosshair read 22:42 while the snap label read 22:58,
which was the last bar.
Two faults behind it. The tool's pointer listener is on window, so it processes
moves over the sidebar — a real sample reported x=1409 on a chart 1280 wide. And
a null time meant a default instead of no answer.
A pointer outside the plot now hides the indicator, and a null time resolves to
the bar nearest in pixels rather than the newest. An e2e test hovers a bar, the
price axis, the sidebar and back, asserting the indicator never sits at the live
edge from mid-chart and disappears once the cursor leaves.
Worth recording that device pixel ratio, viewport size, resize desynchronisation
and the chart scrolling under the gesture were each measured and ruled out
before this, none of which was the cause. With the browser on another machine,
instrumenting it should have come first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sweeping the cursor along the bottom of the chart rarely landed on a bar's low,
and often nowhere near the bar at all. The cause was a nearest-in-2D search I
added an hour earlier to make bars easier to hit when zoomed out: whichever bar
nearby had the lowest low won on total distance, so the dot skipped off the bar
under the cursor instead of tracing each low in turn. Reverted.
The rule is now written down rather than adjusted per complaint — x picks the
bar, y picks which of its extremes — and an e2e test asserts it by sweeping a
zoomed-out 1h chart and requiring every position to land on the low of the bar
beneath it. 115 of 115.
The crosshair is the other half of why this felt broken. Lightweight Charts
defaults to Magnet, which snaps it to the bar's close, so hovering beside a low
displayed a price several ticks from the one an anchor would use. Arming a tool
already switched to Normal, but the chart is read before a tool is armed, which
is when the misleading reading was being taken. Normal everywhere now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Snapping took the bar sharing the cursor's time and then its nearer extreme,
which ignored how far away that extreme was. Pointing below a candle snapped to
that candle's low however distant, while the extreme genuinely under the cursor
was never considered. Zoomed out to some 360 bars at three pixels apart, that
made hitting the bar you meant a matter of several tries. snapPoint now scans
six bars either side and takes the extreme nearest in pixels.
Proven by probe: with the cursor sitting exactly on one bar's low but nudged two
pixels so coordinateToTime resolves to its neighbour, the snap takes the extreme
under the cursor rather than the neighbour's.
A report of the snap dot appearing "way above the bar" turned out to be the dot
landing correctly on the low while the cursor was 151 points below it: the right
price scale keeps a bottom margin of 0.1 and the volume overlay is drawn in it,
so the lower fifth of the pane sits below every candle.
bin/e2e runs tests/e2e against the dev stack inside the playwright service —
Node's own test runner, no dependency added here, since Playwright is global in
that container. Eleven cases, each one a bug that shipped: the viewport parked
ten hours in the past, hourly candles drawn as slivers, stale bar events
throwing, comments drifting across a timeframe switch, and three ways a
trendline anchor could disagree with its preview. Not one was reachable from
pytest. Tests delete any drawing they create, because the dev store is shared
with whoever is looking at the app.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two faults, both introduced by the tick feed, both visible as a huge bar that
flattened the price scale.
A LEVEL_ONE_FUTURES update arrived carrying LAST_PRICE: 0. The parser rejected
None, but 0 is not None, so a minute opened at zero — o=0.0 h=7777.25 l=0.0 —
and provisional_higher carried that low into 5m, 15m, 30m, 1h and the daily bar.
Non-positive prices are treated as absent now, so the last real price carries
forward and the update still counts as the trade it is.
Separately, store.put replaced a bar only when it matched the tail. That was
sufficient while one closed bar arrived per minute, but ticks open the next
minute before CHART_FUTURES delivers the previous one, so the exchange's own bar
stopped matching the tail and was silently dropped — leaving the tick-built
approximation, with its partial volume, in place permanently. put now searches
back a bounded number of buckets for the one it belongs to, and refuses to let a
provisional bar overwrite a settled one.
Tests cover all three invariants: a zero price parses as a trade with no price,
a late closed bar replaces its bucket and keeps the exchange's volume, and a
tick cannot overwrite a settled bar.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two faults, both of which made a finished line disagree with its own preview.
A placed anchor was being discarded by a twitch. finishToolGesture recomputed
`dragged` from the second click's own pointerdown/up, so a few pixels of
movement while pressing was read as a fresh press-drag-release: the anchor from
the first click was thrown away and the line began at the second click instead,
to the right of where it was meant to start. The preview had been rubber-banding
from the real anchor the whole time, which is why the result jumped on commit. A
pending anchor now wins over the current click's drag.
Snapping ignored where you pointed. snapPoint took the nearer of the bar's high
and low but only within 8px, so a cursor between the two snapped to neither and
returned a raw mid-bar price — and the side quietly fell back to the dropdown.
The gate is gone: with snapping on, an anchor always lands on the nearer extreme
of the nearest bar, and that choice *is* the side, a high being resistance and a
low support. app.js already preferred snappedSide over the dropdown, so the
inference was written and simply never fired.
snapPoint also guards against being handed an already-snapped point, which
carries no cursor y and previously compared against NaN.
Verified by driving the gesture in a browser: click, move, then a second click
with 4px of movement while pressed now yields anchor_t equal to the first
click's time and anchor_p exactly equal to the bar's high.
Also lands the groundwork for chart comments, inert until the UI is wired: a
`kind` on ManualLine with `pinned`, `x`, `y` and `collapsed`, a POST /comments
endpoint, GET /drawings, and the chart's DOM comment layer. Comments are stored
with the lines so they share numbering, filtering and deletion, and
ManualLineStore.levels() excludes them — a comment reaching the level list would
join a confluence cluster and push a notification about a piece of text. Tests
cover the exclusion, the derived kind for lines saved before comments existed,
and that numbering is shared.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things kept the chart quieter than the feed.
Higher timeframes only moved once a minute. Tick bars are 1m and the socket
filters bar events by the subscriber's timeframe, so on the hourly chart every
tick was discarded and only a closed minute passing through the aggregator
showed up. They cannot simply be fed to the aggregator — it accumulates with
current.v += incoming.v, so the same forming minute re-sent on each tick would
add its volume to every higher timeframe again and again. provisional_higher
combines the aggregator's committed state with the live minute instead, without
mutating it; the next closed minute goes through normally and replaces the
result, because the store keys on the bucket timestamp. A test pins the
behaviour: five ticks in one minute leave the hour's volume at closed plus live,
counted exactly once.
Trades known only by their volume were skipped. Level 1 resends only changed
fields, so some trades carry a trade stamp and a moved TOTAL_VOLUME with neither
LAST_PRICE nor LAST_SIZE. Those now count, with size left at zero rather than
guessed from the volume delta — CHART_FUTURES replaces the minute's volume with
the exchange's own figure moments later, and two ways of counting the same
trades is how double counting starts. Measured: 66 to 74 updates per 90s.
The tick throttle drops to 0.25s, which no longer binds. Measured in regular
hours the gaps between updates are whole multiples of 1.005s — 2.01, 3.02,
4.03 — which is Schwab conflating LEVEL_ONE_FUTURES to one update per second
per symbol. One per second is the source's ceiling, not ours; the longer gaps
are seconds in which their feed carried no trade.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The candle still paused for ten to twenty seconds at a time. Instrumenting the
raw Level 1 stream settled why: 87 messages in 90 seconds, only 33 carrying
LAST_PRICE. Most of the remainder is bid and ask movement, correctly ignored,
but a seventh carry LAST_SIZE, TRADE_TIME_MILLIS and TOTAL_VOLUME with no
LAST_PRICE — trades that printed at the price of the one before, so the field
did not change and Level 1 did not resend it.
Requiring LAST_PRICE discarded those trades and their volume with them.
parse_level_one now recognises size-plus-trade-time as a trade and returns a
null price, which stream() fills from the forming bar. A quote carrying neither
a price nor any trade field is still skipped: a bid is not a trade and must not
extend a candle's high or low.
Measured on the live feed: median gap between updates 3.1s to 2.0s, worst gap
21.5s to 8.1s, roughly 9 updates a minute to 22, and bar volume climbs within
the minute instead of standing still.
The pauses that remain are the market rather than the pipe. Thin pre-open tape
goes seconds without a price-changing trade and then moves several ticks at
once, which is what a gap up after a quiet spell is.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CHART_FUTURES emits a bar only once its minute is over, so the chart stepped
once a minute and sat still in between, which reads as a dead feed.
LEVEL_ONE_FUTURES carries real trades on the same socket and the same login — no
extra REST call, no extra rate limit — and reports delayed: False on this
account. It was verified back in M6 and never subscribed to. It is now, building
a forming bar for the current minute that the authoritative CHART_FUTURES bar
then supersedes.
Three constraints shaped it, each a real bug avoided:
- Tick bars never reach the aggregator. It accumulates with current.v +=
incoming.v, so re-sending the same forming minute would add its volume into
every higher timeframe again on every update. Runtime.on_bar returns early for
an unclosed bar: store, set price, broadcast, stop.
- Emissions are throttled, SCHWAB_TICK_SECONDS default 1.0, because /ES trades
many times a second and each emission is a store write plus a broadcast to
every open socket. Negative drops the Level 1 subscription entirely.
- A tick for a minute CHART_FUTURES has already closed is dropped, or a late
trade would overwrite a settled exchange bar with a partial one.
Bid-only updates are skipped rather than carried forward: a bid is not a trade
and must not extend a candle's high or low. Alerts stay on closed bars — a level
is judged on a settled bar, not a price that may not last the minute — which
needed no change, since on_bar already gated on closed.
Verified against the live socket: 15 forming bars and 2 closed bars in 100
seconds, the closed bar superseding each forming minute. Verified in a browser:
the last candle's high and low visibly extend within the minute, no console
errors. 85 tests pass, four of them new.
The plan gains the cold-restart options asked for: make seeding non-quadratic
first, then persist cooldowns, then persist bars.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Verified against a live account before and after writing it. CHART_FUTURES
delivers one true-OHLCV minute bar per symbol per minute, LEVEL_ONE_FUTURES
reports delayed: false, and consecutive bars arrived sixty seconds apart through
the production code path.
Yahoo stays. Schwab serves no futures history whatever, so seed_source resolves
to Yahoo even when SEED_SOURCE=schwab is asked for — the pairing is the intended
configuration rather than a fallback. The symbols differ, ES=F against /ES, so
Settings.live_symbol picks the live one while seeding always uses Yahoo's.
Three findings worth keeping, each of which cost a round trip:
- get_quote() singular returns the wrong instrument entirely. It puts the symbol
in the URL path, where the leading slash is normalised away, so /ES resolves to
Eversource Energy at $72 and returns HTTP 200 with a populated body. Only
get_quotes() plural, which passes symbols as a query parameter, returns the
future. A 200 is not evidence; assetMainType is.
- Streaming requires the Accounts and Trading product. StreamClient.login() reads
/trader/v1/userPreference for its socket URL, and that path does not exist in
Market Data Production.
- /ES resolves to the active contract on Schwab's side, so the contract roll
handling the plan left open needs no code.
The stream drops the oldest queued message rather than stalling the socket, and
surfaces a dead pump task instead of waiting forever on a queue nothing fills.
schwab-py moves into requirements.txt, imported only when LIVE_SOURCE=schwab.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
