Flagship · v0.1.0 testing

Nereus

Shared stream storage for Pulsar and Native Kafka.

Protocol systems keep their semantics. Nereus manages durable bytes, logical offsets, committed heads, physical generations, recovery, and safe reclamation beneath them.

Read the docs Nereus on GitHub

5storage profiles

1stream truth

2protocol paths

Pulsar

Kafka

stream
truth

WAL · Object · Oxia

What Nereus does

One storage core beneath two protocol worlds.

Nereus sits below protocol-specific brokers. It gives each integration a stable logical stream while allowing the physical WAL, object layout, and recovery path to evolve independently.

Pulsar ClientKafka Client

Protocol entry points

Pulsar Broker and native Kafka retain topic, partition, subscription, transaction, and leader semantics.

Protocol adapters

ManagedLedger, Pulsar, and Kafka adapters translate protocol coordinates into streamId + offset operations.

Nereus storage core

Append, read, recovery, generation publication, retention, and physical reclamation share one logical stream truth.

Metadata and bytes

Oxia owns visibility and coordination; BookKeeper and object storage hold durable physical representations.

The model

Three invariants anchor every path.

Stable logical coordinates

streamId + offset

Physical data may move between WALs and generations without changing the protocol-facing logical position.

One logical commit point

stream head version-CAS

A successful WAL write or object PUT is not a committed offset. The head CAS is the linearization point.

Physical evolution through generations

same range, new representation

Read-optimized generations can replace older physical layouts while readers pin and drain the source safely.

Storage profiles

Choose a persistence boundary per stream.

Understand the write path →

sync01

BOOKKEEPER_WAL_ONLY

Primary WAL: BookKeeper
Object work: Disabled
Producer boundary: BK durable + stable projection

sync02

BOOKKEEPER_WAL_SYNC_OBJECT

Primary WAL: BookKeeper
Object work: Synchronous
Producer boundary: BK + visible object index

async03

BOOKKEEPER_WAL_ASYNC_OBJECT

Primary WAL: BookKeeper
Object work: Background worker
Producer boundary: BK durable + stable projection

sync04

OBJECT_WAL_SYNC_OBJECT

Primary WAL: Object WAL
Object work: Synchronous
Producer boundary: Object WAL + visible index

async05

OBJECT_WAL_ASYNC_OBJECT

Primary WAL: Object WAL
Object work: Background read-optimized objects
Producer boundary: Object WAL durable + stable projection

Explore the documentation

Follow the dependency order.

01ArchitectureBoundaries, authorities, and source anchors 02Logical coordinatesStream, record, entry, batch, and offset 03Commit protocolWAL durability, head CAS, and recovery 04Current statusImplementation baseline and migration coverage

Current status

v0.1.0 testing

The public documentation is being migrated from the architecture snapshot into a dependency-ordered Docusaurus site. Detailed implementation claims remain tied to an explicit source commit and verification date.

Source baseline: c820391dc1de
PDF snapshot: 894fc4e4d9af
Verified: 2026-08-07
Documentation: English architecture tree migrated; translation and future snapshots remain separate