Red Hat AI Inference Server
A supported, packaged build of the open source vLLM server, so an IBM Power shop can serve its own large language models on hardware it already owns. Red Hat announced it on May 20, 2025 and IBM posted general availability on IBM Power on December 11, 2025, where it runs in a Linux partition rather than inside IBM i.
What Red Hat AI Inference Server does
It holds a large language model, a program trained to read and write text, and answers requests your own applications send it. Red Hat calls it a hardened, supported distribution of vLLM, the open source serving engine. You can run it as a single container image on its own, or get it inside Red Hat Enterprise Linux AI or Red Hat OpenShift AI. Red Hat also allows other Linux and Kubernetes platforms under its third-party support policy.
What it adds on top of plain vLLM
- A Red Hat support contract and a product version it keeps up to date
- LLM Compressor, an open source library that shrinks a model using quantization and sparsity
- A set of models Red Hat has already tested, published in the Red Hat AI organization on Hugging Face
- A container image built for each kind of accelerator, including one for the IBM Spyre Accelerator
Red Hat says those tested models run 2 to 4 times more efficiently without giving up accuracy. That figure is the vendor's own claim.
Can an IBM i shop use it?
Yes, but not inside IBM i itself. It runs on Linux on an IBM Power server. IBM describes the supported setup as IBM Power processor-based servers running Linux. So an IBM i shop puts it in a Linux partition on the same machine and talks to it over the network, the same way it would reach an outside service.
IBM's December 11, 2025 announcement names Linux on Power only. IBM has not published a way to run it inside an IBM i partition.
What the Power version needs
Red Hat AI Inference 3.5.0 requirements for IBM Spyre on IBM Power
| Piece | Version |
|---|---|
| Container image | vllm-spyre-rhel9:3.5.0 |
| vLLM | v0.24.0 |
| IBM Spyre Enablement Stack | 1.1.1 |
| vllm-spyre | 1.6.1 |
| vllm-tgis-adapter | 0.9.2 |
| Python | 3.12.9 |
| PyTorch | 2.7.1 |
Red Hat lists this image as generally available on IBM Power, which is ppc64le, and on IBM Z, which is s390x, and as a technology preview on x86. Red Hat's own wording on the host is that single-host deployments for IBM Spyre AI accelerators on IBM Z and IBM Power are supported for RHEL AI 9.6+.
Red Hat AI Inference Server timeline
- Jan 13, 2025Red Hat completes its purchase of Neural Magic. vLLM, LLM Compressor and pre-optimized models move into Red Hat AI.
- May 20, 2025Red Hat announces AI Inference Server at Red Hat Summit in Boston as a hardened, supported distribution of vLLM.
- Oct 14, 2025Red Hat AI 3 is announced, folding AI Inference Server, RHEL AI and OpenShift AI into one platform, generally available in November.
- Dec 11, 2025IBM posts that Red Hat AI Inference Server is generally available on IBM Power.
Related Terms
vLLM
An open source server that keeps a large language model loaded and answers requests over an OpenAI style API, with PowerPC CPUs on its supported hardware list. That lets an IBM i shop run a model in a Linux partition on the same IBM Power server as its production workload, and Red Hat ships a generally available vLLM container image for ppc64le in Red Hat AI Inference 3.5.0.
IBM Spyre Accelerator
A PCIe4 AI inferencing card IBM ships for IBM Z, LinuxONE and Power11 systems to offload generative and agentic AI workloads from the main processor. On Power11 it is driven from a Red Hat Enterprise Linux or Red Hat OpenShift AI partition, and as of September 2026 IBM has not documented a way to use it from an IBM i partition.
Red Hat OpenShift AI
Red Hat's add-on platform for serving and governing AI models on a Red Hat OpenShift cluster, supported self-managed on ppc64le, the IBM Power architecture that also runs IBM i. It lets a Power shop host open weight models on its own hardware instead of calling an outside service.
IBM Power 11
IBM's current generation of Power processor and server hardware, announced July 8, 2025 and generally available July 25, 2025, carrying forward Power10's on-chip Matrix Math Acceleration and becoming the first Power generation to support the IBM Spyre Accelerator card.
ppc64le
The name Linux, Red Hat packages, container images and Python wheels use for a 64-bit IBM Power processor running in little endian byte order, and the word an IBM i shop has to match when it installs AI software on Power hardware. IBM i itself is not ppc64le, so a ppc64le build runs in a Linux partition on the same Power server, never inside IBM i or PASE.
IBM i
IBM's operating system for Power servers running traditional AS/400-lineage business applications, with Db2 for i built directly into the OS.
Large Language Model (LLM)
A machine learning model trained on large volumes of text, used as the underlying technology behind generative AI tools including IBM Bob and general-purpose assistants like ChatGPT and Claude.
Matrix Math Acceleration (MMA)
An on-chip acceleration feature in IBM Power10 and Power11 processors that speeds up the matrix math operations used in AI inferencing, without requiring a separate GPU.