vLLM

A framework for efficient model inference with omni-modality models

Intelligent Mixture-of-Models Router for Efficient Heterogeneous LLMs Inference

Community maintained hardware plugin for vLLM on Apple Silicon

Common recipes to run vLLM

TPU inference for vLLM, with unified JAX and PyTorch support.

An LLM post-training framework with vLLM for RL Scaling

A high-performance and light-weight router for vLLM large scale deployment

Agent skills for vLLM

Stateful API logic for agentic applications using vLLM

The vLLM XPU kernels for Intel GPU

vLLM plugin for attention-ffn disaggregation support

vLLM Daily Summarization of Merged PRs

High-performance Rust benchmark client for vLLM serving endpoints.

Community maintained hardware plugin for vLLM on Intel Gaudi

Community maintained hardware plugin for vLLM on AWS Neuron

vLLM plugin for block-based diffusion language model (dLLM) support

vLLM Quantization plugin for GGUF

FA4-based Relative Attention Kernel developed by TML and Colfax

vLLM Model plugin for the encoder-decoder BART model

Performance benchmark & accuracy evaluation for vLLM

Performance dashboard for vLLM

vLLM Quantization plugin for bitsandbytes

A framework for few-shot evaluation of language models.

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling