Aphrodite Engine

User and Developer Documentation

Breathing Life Into Language

Paged Attention

Efficiently manage KV cache using vLLM's Paged Attention kernels.

Continuously batch incoming requests in the Async server.

Run almost any Hugging Face format LLM seamlessly.

Support for almost all quantization formats, with optimized kernels for efficient deployment.

Quickly deploy models with the integrated OpenAI API, supporting Text/Chat Completions, Vision, and Batch API.

Accelerate inference using various state-of-the-art spec-decoding methods.

Deploy hundreds or thousands of LoRAs efficiently using Punica, and PEFT-style Prompt adapters.

Aphrodite supports NVIDIA & AMD GPUs, Intel XPUs, Google TPUs, AWS Inferentia/Trainium, AVX2/AVX512/ppc64le CPUs.