vLLM

Cost-efficient and pluggable Infrastructure components for GenAI inference

A framework for efficient model inference with omni-modality models

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Community maintained hardware plugin for vLLM on Ascend

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Community maintained hardware plugin for vLLM on Apple Silicon

Common recipes to run vLLM

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

TPU inference for vLLM, with unified JAX and PyTorch support.

A high-performance and light-weight router for vLLM large scale deployment

Agent skills for vLLM

vLLM Daily Summarization of Merged PRs

The vLLM XPU kernels for Intel GPU

Community maintained hardware plugin for vLLM on Intel Gaudi

Community maintained hardware plugin for vLLM on AWS Neuron

Stateful API logic for agentic applications using vLLM

Manages vllm-nccl dependency

vLLM plugin for block-based diffusion language model (dLLM) support

vLLM Model plugin for the encoder-decoder BART model

vLLM Logo Assets

Performance dashboard for vLLM

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling