chart/docs/feature_undo.md

# Undo and Redo

Status: proposed. An in-session client stack (Ctrl/Cmd+Z and an undo
button) plus `POST /api/lines/restore` shipped as an interim so delete
undo keeps the same id and number. Redo and the command journal are still
unbuilt.

## Goal

Add reliable Undo and Redo for user-authored drawings without silently
overwriting newer changes, changing drawing identity, or allowing REST and
WebSocket ordering to corrupt the result.

The durable design is a server-authoritative drawing command journal with a
frontend command stack. This is deliberately smaller than full event sourcing:
current drawing state remains directly persisted, while a bounded journal keeps
the before/after snapshots needed to reverse accepted user commands.

## Scope

Undoable persistent operations:

- create and duplicate a trendline;
- move a whole trendline or either anchor;
- end a trendline at a cutoff;
- rename, recolor, resize, arm, or re-arm a drawing;
- create or edit a typed price level;
- create, edit, recolor, pin, float, move, or delete a comment;
- delete one drawing, a selection, or all drawings matching a filter.

Transient behavior:

- If a trendline has a pending first anchor, Undo cancels that anchor before it
  touches persistent history. This is not written to the command journal.
- Escape should cancel a pending anchor/tool/context menu without creating an
  undo entry.

Excluded:

- MA, VWAP, prior-day and other derived levels;
- chart viewport, timeframe and layer preferences;
- selection and expanded/collapsed sidebar sections;
- automatic alert disarming performed by the runtime;
- notifications that have already been sent.

Comment collapse/expand should initially remain outside command history. It is
persisted display state, but including every toggle would make useful history
hard to reach.

## Why Frontend-Only Undo Is Insufficient

The frontend knows which gesture occurred, but the current CRUD endpoints do
not provide enough guarantees to invert it safely:

- Recreating a deleted drawing through a create endpoint produces a new ID,
  number and creation time. IDs also affect cluster identity.
- `ManualLineStore` has no revisions, command IDs, transactions across multiple
  drawings, or history.
- Drawing numbers are derived from the current maximum and can be reused after
  deleting the highest-numbered drawing.
- REST responses and WebSocket level deltas can arrive in either order.
- Comments are fetched separately and are not synchronized through WebSocket
  drawing deltas.
- Optimistic drag/delete failures currently have no general rollback path.
- A stale inverse from one tab could overwrite a newer edit from another tab.
- Filtered bulk deletion is a series of requests and can partially succeed.

Undo therefore needs one atomic backend command boundary. The frontend remains
responsible for interaction, labels and optimistic rendering, but the server
decides whether a command or inverse is still valid.

## Semantics

1. Undo reverses the most recent accepted command made by the current browser
   actor, not blindly the most recent global mutation.
2. Redo reapplies the command that Undo reversed.
3. Any new forward command clears that actor's redo stack.
4. Undo and Redo preserve every persisted drawing field, including ID, drawing
   number, creation time, geometry, cutoff, style, comment state and `armed`.
5. A completed drag is one command, regardless of how many pointer-move frames
   were rendered.
6. A multi-selection or filtered deletion is one atomic command and one history
   entry.
7. Undo conflicts rather than overwriting a drawing changed by another command
   after the target command completed.
8. Undo restores drawing state only. It cannot retract a browser/ntfy alert or
   reverse historical market evaluation.
9. History is initially bounded and may be cleared by a server restart. Durable
   cross-deploy history belongs with the eventual SQLite persistence stage.

Suggested initial limits are 100 commands or 24 hours, whichever is reached
first. Limits should be configuration, not part of command correctness.

## Command Model

Each accepted mutation produces a command record:

```text
DrawingCommand
  command_id       client-generated UUID used for idempotency
  actor_id         stable random ID for one browser profile
  action           create, patch, delete, bulk_delete, undo, redo
  target_ids       affected drawing IDs
  expected         target revisions supplied by the client
  before           complete ManualLine snapshots before mutation
  after            complete ManualLine snapshots after mutation
  store_revision   monotonically increasing drawing-store revision
  created_at
  undone_by        inverse command ID or null
  redone_by        redo command ID or null
```

The complete snapshot is the persisted `ManualLine` state:

```text
id, tf, side, anchor_t, anchor_p, slope, last_t, created_at,
note, hidden, color, line_width, number, cutoff_t, armed,
kind, pinned, x, y, collapsed, revision
```

The server must assign a monotonically increasing drawing number from persisted
store metadata. It must not recalculate the next number from only the currently
existing drawings.

### Revisions and Conflicts

Each drawing receives a revision. A command that changes existing drawings
includes their expected revisions. The backend rejects the complete command
with `409 Conflict` if any expected revision is stale.

Undo carries the revisions produced by the original command. If a later command
changed one of those drawings, Undo is rejected rather than restoring a stale
full snapshot. The UI should retain the failed command in history and explain
that a newer change prevents undoing it.

### Idempotency

Create and retry behavior must use `command_id`. Repeating an accepted command
returns its original receipt instead of creating a second drawing. This matters
when persistence succeeds but a response, rebuild, or network connection fails.

## Backend Architecture

Introduce a `DrawingService` above `ManualLineStore`. All drawing create, patch,
delete, restore and batch operations go through it.

Responsibilities:

1. Acquire the drawing-store lock.
2. Reject a duplicate `command_id` or return its existing receipt.
3. Validate expected revisions.
4. Capture complete `before` snapshots.
5. Build and validate the complete proposed `after` state.
6. Apply all affected records atomically and save once.
7. Append the command record and increment the store revision.
8. Rebuild levels once when level-bearing drawings changed.
9. Broadcast one drawing delta and one resulting cluster update.
10. Return the same command receipt used by WebSocket reconciliation.

The JSON implementation can write drawing state, store metadata and the bounded
journal together through the existing temporary-file-and-replace pattern. A
later SQLite implementation should preserve the service and API contracts while
moving state and journal writes into one database transaction.

### Required Persistence Corrections

- Add explicit restore/upsert that preserves ID, number and creation time.
- Persist `next_number`, drawing revisions and store revision.
- Support atomic multi-record mutation and one save.
- Change PATCH processing from `exclude_none=True` to `exclude_unset=True` so
  an inverse can explicitly restore `cutoff_t` to `null`.
- Validate complete resulting geometry, not only supplied PATCH fields.
- Keep comments excluded from levels, clustering and alert evaluation.

## API Shape

Exact route names can follow the existing API style, but mutation responses
need a common command receipt.

```json
{
  "command_id": "uuid",
  "action": "move",
  "store_revision": 42,
  "changed": [{ "id": "ml_...", "revision": 8 }],
  "removed": [],
  "undo_label": "Move up 2"
}
```

Recommended operations:

```text
POST /api/drawing-commands             execute create/patch/delete/batch
POST /api/drawing-commands/{id}/undo   atomically apply the inverse
POST /api/drawing-commands/{id}/redo   atomically reapply the result
GET  /api/drawing-commands?actor_id=   recover bounded own history
```

Existing drawing routes can initially become adapters that call
`DrawingService`, but new frontend work should use the command endpoint so every
mutation has idempotency, revisions and a receipt.

## WebSocket Synchronization

Replace line-only assumptions with a drawing delta that covers trendlines,
price levels and comments:

```json
{
  "type": "drawings",
  "seq": 42,
  "command_id": "uuid",
  "actor_id": "browser-id",
  "changed": ["complete drawing DTOs"],
  "removed": ["ml_..."]
}
```

The initial snapshot should include all drawings or the current drawing
revision plus a required refetch. Every drawing delta has a monotonically
increasing sequence. If a client detects a gap, it refetches a complete drawing
snapshot instead of applying uncertain incremental state.

The frontend reconciles REST acknowledgement and WebSocket delivery by
`command_id`; receiving both must not apply a command twice. Drawing removal
also clears nonexistent IDs from single and multi-selection state.

## Frontend Command Manager

Add one command manager in `static/app.js` rather than separate undo logic in
each control:

```text
execute(command, optimisticProjection)
undo()
redo()
canUndo
canRedo
undoLabel
redoLabel
```

Execution flow:

1. Capture the relevant current revisions and before state.
2. Apply an optimistic projection when useful for interaction.
3. Send a command with a UUID and actor ID.
4. Commit it to the local Undo stack only after server acknowledgement.
5. Roll back the optimistic projection on failure.
6. Reconcile the matching WebSocket command receipt.

Commands should serialize per drawing. Completed drag operations should
coalesce into one PATCH/command on pointer release; pointer movement remains
local rendering only. Repeated style changes may be coalesced later, but are not
required for the first release.

## Controls and Shortcuts

- `Ctrl+Z` and `Cmd+Z`: Undo.
- `Ctrl+Shift+Z` and `Cmd+Shift+Z`: Redo.
- `Ctrl+Y`: Redo on platforms where it is conventional.
- Keep the existing input/editing guard so text fields retain native Undo.
- If a first trendline anchor is pending, Undo cancels it first.
- Call `preventDefault()` only when the chart actually handled the shortcut.
- Add visible Undo and Redo buttons with labels/tooltips such as
  `Undo move up 2`; keyboard shortcuts cannot be the only mobile access.
- Disable controls while an Undo/Redo request is pending.

Selection itself is not undoable. Undoing deletion may select the single
restored drawing, but batch restoration should leave selection empty unless a
clear product rule is chosen.

## Failure Handling

- Network/server failure: restore the optimistic before state and show a visible
  error; do not add a history entry.
- Revision conflict: refetch drawings, preserve the failed history entry, and
  explain that a newer change prevents Undo.
- Missed WebSocket sequence: refetch the complete drawing snapshot.
- Partial bulk failure: impossible by contract; the whole command commits or
  none of it does.
- Rebuild failure after persistence: return the stored idempotent receipt on
  retry and recover derived levels from authoritative drawing state.
- Session expiration: authentication may retry, but the same `command_id` must
  be reused so a create cannot duplicate.

## Rollout

### Stage 1: Reliable Command Boundary

- Add `DrawingService`, revisions, stable restore, monotonic numbering,
  idempotent command IDs and atomic batches.
- Route all drawing mutations through it.
- Return common command receipts.
- Add in-memory/bounded server journal and frontend Undo/Redo stacks.
- Implement create, patch, move, delete and bulk-delete inverses.
- Add shortcuts and visible controls.

Stage 1 history may clear on restart, but every command during the process
lifetime must be safe and conflict-aware.

### Stage 2: Complete Multi-Client Synchronization

- Broadcast full drawing deltas, including comments.
- Add sequence-gap recovery and complete drawing snapshots.
- Reconcile REST and WebSocket acknowledgements by command ID.
- Clean stale selection state after remote mutations.

### Stage 3: Durable History

- Move drawing state and the bounded command journal into SQLite.
- Preserve command/API contracts.
- Retain history by count/time policy across restart and deployment.
- Add audit-only, non-undoable records for automatic runtime changes if useful.

## Tests That Earn Their Place

Backend:

1. Undo delete restores every field with the same ID and drawing number.
2. Undo end-here restores `cutoff_t` to `null`.
3. Bulk delete is all-or-nothing and saves/rebuilds/broadcasts once.
4. Retrying one `command_id` cannot create a second drawing.
5. Undo rejects a stale revision after another actor edits the drawing.
6. Redo restores the exact accepted result; a new command clears redo.
7. Comment commands synchronize but never enter levels or confluence.
8. Automatic alert disarm does not enter user Undo history.

Browser:

1. Create, Undo and Redo preserve the drawing ID.
2. Whole-line and anchor drags Undo to exact prior geometry.
3. Single and filtered batch deletion restore the complete set.
4. Pending first anchor consumes Undo before persistent history.
5. Undo inside a text field remains native text editing.
6. A conflict displays an error and does not overwrite newer state.
7. A second tab observes drawing and Undo deltas, including comments.
8. Touch-accessible controls expose the same command labels and states.

## Decisions to Confirm Before Implementation

- Whether bounded history should survive a normal server restart in Stage 1 or
  wait for SQLite in Stage 3.
- Whether persisted comment collapse/expand should remain excluded.
- Whether undoing one deleted drawing should select the restored drawing.
- The initial history count/time limits.
- Whether actor identity remains per browser profile or later becomes the
  authenticated user ID. The command model supports either.

The first implementation task is the atomic, revisioned `DrawingService`, not
the Undo button. Building the controls first would make deletion restoration,
bulk actions and cross-tab edits appear to work while retaining silent data-loss
races.