Skip to main content
Source baselineNereuscommit c820391dc1de·verified 2026-08-07·authority reader-facing-summary

Example: Pulsar with BookKeeper async Object

This scenario shows why a Pulsar batch Entry remains one Nereus offset even when it contains ten sub-messages.

Initial state and mapping​

streamId = S-payments-1
virtualLedgerId = V-9001
committedEndOffset = 1000
active BookKeeper ledger = 880
next BookKeeper entry = 200

The Entry contains ten batch messages, but the facade produces:

AppendEntry.recordCount = 1
logical range = [1000,1001)

The ten Pulsar MessageIds differ by batchIndex; they do not consume ten stream offsets.

BK append and immediate read​

  1. The stream lane validates the session and expected offset.
  2. Reservation confirms ledger 880 can hold the complete append.
  3. The complete Entry is written as BookKeeper entry 200.
  4. Bookie quorum succeeds; range metadata and fixed protection slots are published.
  5. Commit intent is written and head advances 1000 -> 1001.
  6. The async profile completes at the stable-head boundary.
  7. Pulsar returns Position(V-9001,1000).

Before objectization, generation 0 reads the exact target (ledger=880, firstEntryId=200, entryCount=1). Pulsar receives complete Entry bytes and applies batch decoding itself.

Async materialization​

The worker freezes exact generation-0 BK ranges, establishes dynamic source protection, reads the ranges, writes a verified NCP1 Object, and publishes generation 1. Ordinary readers then prefer the Object; while the BK source remains healthy, a physical Object failure can fall back to generation 0 in the same COMMITTED view.

BK ledger retirement​

Ledger 880 is not deleted when the Object appears. It must be sealed, every range must be trimmed or covered by a healthy replacement, recovery checkpoint proof must exist, and reader/task/repair/cursor/ append-recovery references must be absent. Whole-ledger retention then marks, drains, deletes, and performs delayed double-absence verification.

Source anchors​