Higress: The AI-Native API Gateway
Shriira Press
An AI-native API gateway built on Envoy and Istio that unifies ingress, microservices, and LLM traffic behind one console.
Welcome to Higress: The AI-Native API Gateway.
Higress is a cloud-native API gateway built on the Envoy proxy and the Istio control plane, born inside Alibaba's production infrastructure and now a CNCF sandbox project. It folds three gateways that organizations usually run separately — the traffic gateway at the cluster edge, the microservices gateway in front of internal services, and an emerging AI gateway in front of large language models — into a single system with one configuration model and one console. This book traces that story from the ground up. The opening chapters explain what Higress is and the Nginx-reload problem that motivated it, then dissect the Envoy-and-Istio architecture and the control plane that turns Kubernetes resources into proxy configuration. From there we cover routing through Ingress and the Gateway API, service discovery across registries like Nacos and Consul, and the WebAssembly plugin system that lets you extend the gateway in Go, Rust, or JavaScript without restarts. The later chapters dive into the AI gateway — unified LLM routing, token rate limiting, semantic caching, and MCP server hosting — before closing with operations, migration, and where Higress fits in the cloud-native landscape.
This title is part of the ShriIra library and is free to read in full, right here — our small contribution to making world-class knowledge easy to reach.
A note on reading it: open the Contents menu at the top of the reader to jump between chapters, use the Aa menu to set a comfortable text size, theme (light, sepia, or night), and single- or two-page layout. Your place is saved automatically, so you can always pick up where you left off.
We hope it serves you well.
— Shriira Press