What happened
Red Hat released OpenShift 5 on 9 November. For AI, three changes stand out against OpenShift 4. The MCP server, which lets AI agents read and act on a cluster, moves from Technology Preview in OpenShift 4.22 to full support. The MCP gateway also reaches general availability. OpenShift Lightspeed changes from a chat assistant to an agent that diagnoses cluster problems and assesses update risk. OpenShift 5 is also the base of Red Hat AI Factory with NVIDIA.
Why it matters
Agents need fast model answers. On OpenShift AI, Red Hat runs model inference with vLLM and its llm-d router. In Red Hat's published test, llm-d sent each follow-up turn to the server that already held the conversation in its cache. The KV cache hit rate rose from about 62% to about 90%. The mean time to first token fell from 208.96 to 112.54 milliseconds.
The data
| Metric | vLLM, round-robin | llm-d, prefix-aware | Unit |
|---|---|---|---|
| Mean time to first token | 208.96 | 112.54 | milliseconds |
| Later-turn time to first token (average) | 195.23 | 86.89 | milliseconds |
| 95th percentile time to first token | 727.26 | 237.96 | milliseconds |
| KV cache hit rate | about 62 | about 90 | percent |
| Throughput | 0.52 | 0.49 | requests per second |