GitHub

Upstream TigerBeetle Backport Ledger

ArcherDB’s VSR/LSM/IO core derives from TigerBeetle. This ledger tracks the fork lineage and the disposition of every upstream change we have reviewed, so backporting is a repeatable process with explicit verdicts rather than a one-off sweep.

  • Fork point: TigerBeetle commit 0baa07d3 (2025-12-29) — verified by byte-identical src/vsr/replica.zig blob at ArcherDB’s import commit f77fa3cf.
  • Reviewed through: TigerBeetle 4abc0229a = release 0.17.7 (2026-06-12); 1056 upstream commits, ~470 touching shared subsystems; 147 curated changelog entries across releases 0.16.68–0.17.7 triaged.

Process rules:

  • Every ported hunk carries the upstream PR number and SHA in a code comment.
  • Consensus-path ports require a local VOPR campaign (both state machines, multiple seeds) before merge, on top of the unit/integration battery.
  • Wire-format-adjacent changes (message headers, on-disk layout) are never bundled with routine ports; they get their own change with compat analysis.

Ported (2026-06-12 batch)

Upstream What ArcherDB location
PR #3726 (44865e796) CheckpointTrailer.open bounds assertion (security-audit): corrupt trailer_size must not index past block arrays src/vsr/checkpoint_trailer.zig
PR #3717 (b1b968b4d) IO event-listener crash: non-nullable completion context; event_listen passed undefined into an @alignCast trampoline (latent UB) src/io/linux.zig
PR #3693 (bba77d0c0) File-creation permissions: data file/probe 0o666→0o600; executables 0o777→0o755 src/io/linux.zig, src/shell.zig, src/multiversion.zig, src/build_multiversion.zig
PR #3729 (1c17caa1c) AMQP method decode fails closed on invalid enum (@enumFromInt UB on broker-controlled bytes → intToEnum catch error.Unexpected); generator script patched; embedded-spec checksum guard updated src/cdc/amqp/spec.zig, spec_parser.py, amqp.zig
PR #3704 (4e275db15, behavioral core only) Prepare-timeout retries fan out to every replica missing from the ack set, instead of cycling one candidate per timeout (tail latency / liveness under partial partitions). Ported without the star-replication doc rewrite it shipped with src/vsr/replica.zig on_prepare_timeout

Already present before this batch (no action): clock skew warn threshold at 50ms (src/vsr/clock.zig:540 matches upstream 0d2f7323c).

Staged — phase 2 (medium, individually portable)

Upstream What Notes
PR #3701 (983ba27d4) Reformat reentrancy: IO.run invoked from within IO.run_for_ns Pattern present in src/vsr/replica_reformat.zig; M-size
PR #3748 (2875a188a) Accept-path makes room for replica connections (evict hogging clients/unknown peers) src/message_bus.zig + constants; M-size
PR #3769 (f3fa72c70) Client pings during registration; eviction reply on ping Touches message_header.zig — wire-adjacent, needs compat pass
e1082a878 Per-replica budget for block repair Depends on repair_budget structure; verify our copy’s lineage first
PR #3592 (0d7fd8953) DISCARD on block-device format (SSD perf) Small; block-device path only
PR #3780 (e7c4e454b) Ban std.fmt.parseInt in protocol paths (tidy rule + sites) L but mechanical; aligns with our parser-honesty stance

Staged — phase 3 (large chains; require dedicated VOPR campaigns)

  1. Grid block ownership via reference counting (PR #3686, 52e149b75, ~400 lines) — correctness-critical lifetime management; prerequisite for the compaction work. Complicated by ArcherDB’s radix_buffer scratch and geo grooves.
  2. Tombstone semantics on the lookup path (PR #3735 chain: 81f7e49d0, 8a8d77bb3, 8116232a7, …) — explicit tombstone variants + fuzzing.
  3. Incremental compaction merge (PRs #3673 + #3770, ~930 lines) — the headline upstream tail-latency win (merge spread across the bar instead of one big end-of-bar merge). Must integrate with ArcherDB’s compaction_throttle, compression, and TTL-expiry accounting. Port order: (1) → (2) → (3).
  4. Linux IO event-loop refactor (PR #3619, a6ec07462) — timeout embedded in io_uring_enter, next_tick as top-level; upstream measured ~8% on their standard benchmark plus tail-latency gains. Large; our io/linux.zig has local modifications to merge through.
  5. Star replication routing (PRs #3669/#3668) — strategy change (ring-forward → primary broadcast). Evaluate deliberately: changes bandwidth/latency trade-offs; we kept the fork-era routing and ported only the #3704 retry fan-out which is strategy-independent.

Rejected / not applicable

Upstream Verdict
PR #3529 (5d934474e) key_range_contains fastpath N/A as-is — would assert-crash. Upstream’s fix assumes the post-fork caller convention (concrete prefetch snapshots, assert(snapshot < snapshot_latest)); ArcherDB still passes the snapshot_latest sentinel (src/lsm/groove.zig:830), under which our existing code already takes the fast path. Revisit only together with the snapshot-convention migration.
TB accounting state-machine changes (src/state_machine.zig, transfers logic) ArcherDB uses its own geo state machine
TB REPL parser overhaul (PR #3786) Our REPL diverged; evaluate separately if parser issues surface
TB client-library fixes (.NET/Java/Go, PRs #3778/#3762 leak+overflow) Our SDKs diverged at fork but shared ancestry — investigate whether the leak/overflow patterns exist in our Java/Go clients before dismissing
Release/CI tooling (their wheel builder, release validation serialization) We have our own (and already build Python wheels without hatchling-equivalent issues post our packaging fix)
Edit this page