chris/chart
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Seed in bulk and coalesce level rebuilds

Browse source 
P1 from docs/async_refactor.md. Measured on the dev stack: the port now accepts
connections 4 seconds after a restart rather than 121, and the worst loop lag
falls from 19,545ms to 526ms, with steady state between 0.2 and 0.6ms.

Seeding replayed years of history through on_bar, rebuilding every level from
scratch per bar and broadcasting each one to nobody. It now fills the store
quietly and derives price, ATR and the level set once at the end, from the
finished history. Alerts are deliberately not evaluated over replayed bars: a
level touched two years ago is not news, and firing on history is one way a
deploy re-alerts.

The seed was not all of it. Yahoo's first poll emits a whole day of minutes in a
single burst, each one taking the full live path, which was most of the
remaining twenty seconds. request_rebuild now coalesces to at most one rebuild
per 250ms and a background pass flushes anything deferred, so a burst costs a
handful of rebuilds instead of hundreds and the last bar is still never the one
dropped.

Verified unchanged after the change: bar counts across every timeframe, all five
daily moving averages with their full point sets, prior-day levels and VWAP. 125
python tests and 31 e2e tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Chris Amow 2026-08-11 15:42:27 -05:00
parent 919a71feb3
commit ff1b9982d1
1 changed files with 24 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
docs/async_refactor.md
View file

@ -1,6 +1,6 @@
# Async refactor — findings, priorities, and how to keep it that way
**Status: P0 and the loop-lag probe are done (2026-08-11). P1–P3 outstanding.** To be implemented once the in-flight chart
**Status: P0, P1 and the loop-lag probe are done (2026-08-11). P2 and P3 outstanding.** To be implemented once the in-flight chart
work has landed. Everything below is from reading the code on 2026-08-11 and
measuring the running app; each finding names the path it was found on.
@ -72,7 +72,29 @@ intervening. Today that passes by luck.
---
## P1 — Level rebuilding is CPU-bound on the loop (latency)
## P1 — Level rebuilding is CPU-bound on the loop (latency) — DONE
Measured on the dev stack, Yahoo source:
| | before | after |
|---|---|---|
| port accepting connections | 121s | 4s |
| worst loop lag | 19,545ms | 526ms |
| steady-state lag | — | 0.2–0.6ms |
Two changes did it. Seeding now accumulates into the store and derives price,
ATR and the level set **once** at the end (`settle_after_seed`) instead of
rebuilding per replayed bar. And `request_rebuild` coalesces rebuilds to at most
one per 250ms with a guaranteed trailing pass, which matters because Yahoo's
first poll emits a whole day of minutes in one burst — that burst, not the seed,
was most of the remaining 20 seconds. Bar counts, all five daily MAs, prior-day
levels and VWAP are unchanged; 125 python tests and 31 e2e tests pass.
Still open from the original list: incremental moving averages, and diffing
levels by fingerprint rather than by re-serialising every point. Neither is
needed while lag sits under a second.
### Original analysis
`rebuild_levels()` recomputes all five daily moving averages and re-serialises
their points to diff them, on every closed bar. Measured consequence: the seed