Oxia: A Scalable Metadata Store and Coordination System
Shriira Press
A metadata store and coordination system that keeps ZooKeeper's familiar programming model but shards it across many nodes — so it can finally scale.
Welcome to Oxia: A Scalable Metadata Store and Coordination System.
Oxia is a distributed metadata store and coordination system, built originally at StreamNative to free Apache Pulsar from the scaling ceiling of ZooKeeper, and now an open project under the cloud-native umbrella. It keeps the programming model that a generation of distributed systems was built on — linearizable keys, sessions, ephemeral records, notifications, and compare-and-swap — but instead of forcing every node to hold the entire dataset, it partitions each namespace into independently replicated shards spread across a fleet of storage nodes. A lightweight coordinator, deliberately kept off the data path, assigns shards and handles leader election, while each shard runs its own write-ahead log and an LSM-backed key-value store. This book opens with the coordination problem and why the old guard runs out of room, then builds Oxia up layer by layer: its data model of namespaces, keys, versions, and sequences; the API of puts, gets, ranges, and conditional writes; ephemeral records, sessions, and notifications that make coordination possible; the sharded architecture and the coordinator's role; the replication protocol that separates log replication from recovery; and finally how Oxia is deployed, operated, and used to back real systems like Pulsar. By the end you will understand not just what Oxia does, but why each design choice was made.
This title is part of the ShriIra library and is free to read in full, right here — our small contribution to making world-class knowledge easy to reach.
A note on reading it: open the Contents menu at the top of the reader to jump between chapters, use the Aa menu to set a comfortable text size, theme (light, sepia, or night), and single- or two-page layout. Your place is saved automatically, so you can always pick up where you left off.
We hope it serves you well.
— Shriira Press