# Where a Hesper turn's fifteen seconds go

**Question (2026-09-12, session 41).** `hesper doctor` reports the live turn at
14.75 s with two landed citizens, while H71 measured 2.60 ms an action and
600 crowded citizens in 564 ms. The author asked, in the WAKE note, whether
something fixed and large dominates a turn before anybody acts — the state
hash over a 512 x 512 map, report building, or the seal — and whether the
new per-round loop (H78) or an index on `actions.round` is the scaling
problem. Measured rather than guessed.

**Method.** A copy of the dev world (`~/data/hesper-dev/h.db`, turn 0, the
same generator and map size as the live world; the live database is not
readable from my account) resolved one turn under `cProfile`:

    cp ~/data/hesper-dev/h.db /tmp/prof.db
    python -m cProfile -o /tmp/turn.prof -m hesper.cli turn --now \
        --db /tmp/prof.db --state-dir /tmp/profstate
    python research/hesper/turn-cost/profile_top.py /tmp/turn.prof

No seal (no `SEAL_SECRET` in a dev run), no render, no citizens: the floor.

## Result

| what                                   | seconds | share |
|----------------------------------------|--------:|------:|
| whole `hesper.cli turn --now`          |   13.21 |  100 % |
| `turn.resolve`                         |   12.44 |   94 % |
| `db.state_hash` (inside resolve)       |   12.32 |   93 % |
| of which `json.dumps`, 262,200 calls   |    5.33 |   40 % |
| of which the row loop itself (`sqlite3.Row` name lookups, dict building) | 6.28 | 48 % |
| module loading, phases, events, everything else | < 0.9 | < 7 % |

The world's own `duration_ms` for the same turn: 12,379 — so the doctor's
14.75 s is this plus the seal's two HTTP calls and a little more state.

**The hash is the turn.** 262,144 of the 262,200 hashed rows are map tiles,
and the serialisation is one `json.dumps(dict)` per row over ~20 columns:
92.8 MB of canonical bytes. The phases, with or without rounds, are under a
second. So:

- The round loop is **not** the scaling problem, and an index on
  `actions.round` is premature at any population this side of H71's numbers.
- The 15 s is spent before the first citizen acts, and it is the same 15 s
  at zero citizens as at a thousand. It is a function of map size only.

## Can the same bytes be produced faster? Yes, but only by 1.4-1.8x

`fastlines.py` reproduces the frozen serialisation without `json.dumps`
(sorted keys once per table, a tuple cursor, hand-encoded values through the
same C `encode_basestring`) and proves it byte-identical on the copy — seeded
with `seed_awkward.py`: unicode, emoji, quotes, backslashes, newlines, tabs,
a control character, fractional and huge and negative-zero floats, NULLs —
for every hash version:

    version 1: identical=True bytes=92839330 old=10.18s new=7.60s
    version 2: identical=True bytes=92839437 old=10.01s new=7.97s
    version 3: identical=True bytes=92839437 old= 9.62s new=6.95s
    version 4: identical=True bytes=92839447 old= 9.11s new=5.48s
    ALL IDENTICAL

(Standalone timings, not under the profiler; the row loop is the floor, and
calling CPython's C encoder directly per row — `c_make_encoder` — is no
faster, 5.68 s for the tiles alone.) Within a frozen version, five seconds is
about the best the current line format allows.

## What a new hash version could do

The bytes are the definition, so the real lever is the *next* version's
serialisation, not a faster copy of this one. Measured on the same tiles:

| serialisation of `tiles` (262,144 rows)           | seconds | bytes |
|---------------------------------------------------|--------:|------:|
| one `json.dumps(dict)` per row (versions 1-4)     |    9.1  | 92.8 MB |
| `fastlines.py`, same bytes                        |    5.5  | 92.8 MB |
| the bare `SELECT ... ORDER BY x, y` fetch         |    2.7  |       |
| **one `json.dumps` of the rows as value lists**, keys named once per table | **0.74** + fetch | 20.9 MB |
| tab-separated `str()` lines                       |    1.0 + fetch | 11.6 MB |
| sha256 over 21 MB                                 |    0.04 |       |

So a version that writes each table as a header line of its sorted columns
followed by one JSON array of value rows would take the hash from ~12 s to
~3.5 s, of which the SQLite fetch is most (the `ORDER BY` on the composite
primary key walks the autoindex: 2.7 s ordered against 1.7 s unordered — a
`WITHOUT ROWID` tiles table would make the two the same, and that is a
migration, not a hash change).

**This is a decision, not a task**, because part 1 §5.5 defines the canonical
serialisation and every seal checks against it: a version 5 (or 6, since
H79's `arrived_turn` column moves the version today) with a new line format
is exactly the seam Round 42/43 named, one more time. Recorded for the author
in `docs/WORLD-DISCUSSION.md`; not built in this session.

## Also answered from the WAKE note

- `actions_queue` and `idx_actions_turn_status` are both `(target_turn,
  status)` on `actions`: two identical indexes, every insert maintains both.
  Dropping one is safe once its name is not referenced (grep before you
  choose); it is a write cost, not a turn cost, and it is not why a turn is
  slow.
- Report building, the seal and rendering were not in this measurement's
  12.4 s at all; the seal is the difference between 12.4 and 14.75.

Files: `profile_top.py` (prints the table above from a `.prof`),
`fastlines.py` (the byte-identical prototype and its proof),
`seed_awkward.py` (the rows that make the proof mean something).