Technology · Ebook

Oxia: A Scalable Metadata Store and Coordination System

by Shriira Press

Oxia is a distributed metadata store and coordination system, built at StreamNative to free Apache Pulsar from ZooKeeper's scaling ceiling and now an open cloud-native project. It keeps the familiar model — linearizable keys, sessions, ephemeral records, notifications, and compare-and-swap — but partitions each namespace into independently replicated shards spread across a fleet of storage nodes. A lightweight coordinator stays off the data path, assigning shards and running elections, while each shard runs its own write-ahead log and an LSM-backed store. This book builds Oxia up from the coordination problem through its data model, client API, sessions and notifications, sharded architecture, the coordinator, a replication protocol that separates log replication from recovery, and finally deployment and backing Pulsar in practice.

Contents

1. 1Preface
2. 2Chapter 1 — The Coordination Problem
3. 3Chapter 2 — The Data Model
4. 4Chapter 3 — The Client API
5. 5Chapter 4 — Sessions, Ephemerals, and Notifications
6. 6Chapter 5 — Sharded Architecture
7. 7Chapter 6 — The Coordinator
8. 8Chapter 7 — Replication and Consistency
9. 9Chapter 8 — Oxia in Practice