KAITO turns a few lines of YAML into a running large language model on Kubernetes — provisioning the GPUs, loading the weights, and serving inference, fine-tuning, and RAG without the usual toolchain sprawl.

Welcome to KAITO: The Kubernetes AI Toolchain Operator.

KAITO is a CNCF sandbox operator suite that automates the deployment of large language model inference, fine-tuning, and retrieval-augmented generation on Kubernetes. Running an LLM normally means wrestling with GPU node pools, distributed-inference flags, multi-gigabyte weight files, and a serving engine — a toolchain so fiddly that most teams give up before they start. KAITO replaces all of that with a single custom resource: you name a model and a GPU instance type, and the operator estimates the memory you need, provisions the right GPU nodes, pulls the weights, and brings up an OpenAI-compatible vLLM endpoint. This book walks through KAITO from the ground up. The early chapters explain what KAITO is and the problem it solves, then the Workspace custom resource and the controller-plus-provisioner architecture behind it. From there we cover GPU node auto-provisioning, the vLLM inference path and preset models, fine-tuning with LoRA and QLoRA, the RAGEngine for retrieval-augmented generation, and autoscaling and routing with InferenceSet and InferencePool. A closing chapter pulls it together with operations, best practices, and where KAITO fits in the cloud-native AI landscape.

This title is part of the ShriIra library and is free to read in full, right here — our small contribution to making world-class knowledge easy to reach.

A note on reading it: open the Contents menu at the top of the reader to jump between chapters, use the Aa menu to set a comfortable text size, theme (light, sepia, or night), and single- or two-page layout. Your place is saved automatically, so you can always pick up where you left off.

We hope it serves you well.

— Shriira Press

Contents

1. Chapter 1 — What KAITO Is
2. Chapter 2 — The Workspace Custom Resource
3. Chapter 3 — Architecture and Controllers
4. Chapter 4 — GPU Node Auto-Provisioning
5. Chapter 5 — Inference with vLLM and Preset Models
6. Chapter 6 — Fine-Tuning with LoRA and QLoRA
7. Chapter 7 — Retrieval-Augmented Generation
8. Chapter 8 — Scaling and Routing Inference
9. Chapter 9 — KAITO in Practice