Skip to main content
Source baselineNereuscommit c820391dc1de·verified 2026-08-07·authority reader-facing-summary

Primary WAL

What problem does it solve?​

The storage core needs a durable representation before it can recover from a broker crash or an uncertain client response. The primary WAL is that first durable representation for a profile. It may be BookKeeper or an immutable Object WAL.

Durable is not committed​

A successful WAL operation proves that bytes exist under a physical identity. It does not by itself prove that the logical offset is visible to readers. Logical visibility is published by the stream-head commit protocol.

WAL durable ≠ offset committed
object PUT ≠ generation published

This distinction is the basis for recovery after a crash between provider I/O and metadata publication.

Object WAL​

Object WAL uses immutable objects. An object may contain slices for multiple streams, so its manifest, root identity, byte range, checksum, and stream slice must be tracked independently. Multi-stream packing improves physical layout but makes exact reference tracking essential for GC.

BookKeeper WAL​

The Nereus BookKeeper WAL uses a per-stream ledger boundary rather than treating a stock ManagedLedger ledger as the logical stream identity. A ledger may roll over; the logical stream and offsets remain continuous. Ranged entries let the Kafka adapter preserve a RecordBatch that covers multiple logical offsets.

Profile-specific completion​

The primary WAL is common to all profiles, but the producer success boundary is not:

Profile familyPrimary WALCompletion implication
BookKeeper-onlyBookKeeperNo message-data Object generation is required for the producer path.
Async objectBookKeeper or Object WALA stable WAL projection can be acknowledged before a read-optimized object exists.
Sync objectBookKeeper or Object WALThe profile may require an additional visible/indexed object representation before success.

The full five-profile matrix will be added in the storage migration stage.

Source anchors​