Technology · Ebook

Tremor: Event Processing at Scale

by Shriira Press

Tremor is a real-time event processing system, written in Rust and forged at Wayfair to tame the firehose of logs and metrics flowing from thousands of applications. It exists for the situation every platform engineer eventually meets: an unbounded, unstructured, high-volume stream that must be classified, reshaped, enriched, rate-limited, and routed to downstream systems without ever falling over. This book covers Tremor's directed-acyclic-graph dataflow model and contraflow backpressure, the connectors and codecs that bridge it to Kafka, Elasticsearch, and S3, the Trickle query language for windowed aggregation, the scripting language for transformation, the Troy deployment language for assembling flows, and how guaranteed delivery and load shedding keep pipelines standing under load.

Contents

1. 1Preface
2. 2Chapter 1 — What Tremor Is
3. 3Chapter 2 — The Dataflow Model
4. 4Chapter 3 — Connectors, Codecs, and Processors
5. 5Chapter 4 — The Trickle Query Language
6. 6Chapter 5 — The Tremor Scripting Language
7. 7Chapter 6 — Troy: The Deployment Language
8. 8Chapter 7 — Backpressure and Guaranteed Delivery
9. 9Chapter 8 — Tremor in Practice