Skip to main content
Source baselineNereuscommit c820391dc1de·verified 2026-08-07·authority reader-facing-summary

Primary WAL

What problem does it solve?

The storage core needs a durable representation before it can recover from a broker crash or an uncertain client response. The primary WAL is that first durable representation for a profile. It may be BookKeeper or an immutable Object WAL.

Durable is not committed

A successful WAL operation proves that bytes exist under a physical identity. It does not by itself prove that the logical offset is visible to readers. Logical visibility is published by the stream-head commit protocol.

WAL durable ≠ offset committed
object PUT ≠ generation published

This distinction is the basis for recovery after a crash between provider I/O and metadata publication.

Object WAL

Object WAL uses immutable objects. An object may contain slices for multiple streams, so its manifest, root identity, byte range, checksum, and stream slice must be tracked independently. Multi-stream packing improves physical layout but makes exact reference tracking essential for GC.

BookKeeper WAL

The Nereus BookKeeper WAL uses a per-stream ledger boundary rather than treating a stock ManagedLedger ledger as the logical stream identity. A ledger may roll over; the logical stream and offsets remain continuous. Ranged entries let the Kafka adapter preserve a RecordBatch that covers multiple logical offsets.

Profile-specific completion

The primary WAL is common to all profiles, but the producer success boundary is not:

Profile familyPrimary WALCompletion implication
BookKeeper-onlyBookKeeperNo message-data Object generation is required for the producer path.
Async objectBookKeeper or Object WALA stable WAL projection can be acknowledged before a read-optimized object exists.
Sync objectBookKeeper or Object WALThe profile may require an additional visible/indexed object representation before success.

The full five-profile matrix will be added in the storage migration stage.

Source anchors