Tremor: Event Processing at Scale
Shriira Press
A Rust-powered event processing engine for the moments when the data never stops coming.
Welcome to Tremor: Event Processing at Scale.
Tremor is a real-time event processing system, written in Rust and forged at Wayfair to tame the firehose of logs and metrics flowing from thousands of applications. It exists for the situation every platform engineer eventually meets: an unbounded, unstructured, high-volume stream of events that must be classified, reshaped, enriched, rate-limited, and routed to downstream systems without ever falling over. This book begins with the problem Tremor was built to solve and the cost it cut at Wayfair, then walks through its directed-acyclic-graph dataflow model and how events travel through it. From there we meet the connectors, codecs, and processors that bridge Tremor to Kafka, Elasticsearch, S3, and the wider world; the Trickle query language for filtering, grouping, and windowed aggregation; and the Tremor scripting language for pattern-matching and transformation. We assemble whole applications with the Troy deployment language, learn how backpressure and guaranteed delivery keep the pipeline honest under load, and close with how teams actually run Tremor and where it sits in the cloud-native landscape.
This title is part of the ShriIra library and is free to read in full, right here — our small contribution to making world-class knowledge easy to reach.
A note on reading it: open the Contents menu at the top of the reader to jump between chapters, use the Aa menu to set a comfortable text size, theme (light, sepia, or night), and single- or two-page layout. Your place is saved automatically, so you can always pick up where you left off.
We hope it serves you well.
— Shriira Press