The financial services industry, like many other sectors, is aiming to make best use of their hardware in the age of resource-intensive AI workloads. Financial services companies have come to rely on the advantages of a container-based architecture, but may worry that container orchestration platforms add performance penalties that could exacerbate resource constraints. The results of a new STAC-AI™ LANG6 (Inference-Only) audit, the industry-standard benchmark for evaluating large language model (LLM) inference performance in financial services, may help assuage those worries when it comes to Red Hat OpenShift. In collaboration with NVIDIA and Supermicro, the Red Hat performance and scale engineering team ran the full STAC-AI LANG6 benchmark suite on OpenShift, using 2 NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs in a Supermicro SuperServer SYS-222C-TN. These are the first audited STAC-AI results produced on a containerized Kubernetes platform.Red Hat has a long history of demonstrating that OpenShift delivers bare-metal-like performance for the most demanding workloads, and we back that claim with independently audited results. In the STAC-N1 network benchmark, we showed that OpenShift matches (and in some cases improves upon) bare-metal latency for market data, tying the lowest mean and median latencies at both 100,000 and 1.4 million messages per second while reducing maximum latency by 37%. In the record-breaking STAC-A2 market risk benchmark, we set multiple records for GPU-accelerated Monte Carlo simulation on OpenShift with NVIDIA DGX A100 systems. And in STAC-A2 with Intel CPUs, we showed that the containerization overhead of OpenShift is negligible even for CPU-intensive financial workloads.This new STAC-AI submission extends that track record into the domain of LLM inference. This is an area of rapidly growing importance as financial institutions adopt AI-driven workflows for fraud detection, security, sentiment analysis, and regulatory compliance.A quick primer on STAC-AI LANG6The Strategic Technology Analysis Center (STAC) produces standardized benchmarks that the world's largest financial institutions use to evaluate technology stacks. STAC-AI LANG6 measures LLM inference performance on financial-services use cases. Its datasets are derived from real SEC EDGAR financial filings, the kinds of documents that drive RAG (retrieval-augmented generation) and long-context workloads at financial firms.The benchmark covers 2 inference modes. Batch mode measures maximum throughput by handing the full dataset to the system under test (SUT) in a single API call. Interactive mode simulates real-world usage by sending requests at varying arrival rates following a Poisson distribution, while measuring reaction time (analogous to time to first token), response time, and output rate under load.We tested 2 models (Llama-3.1-8B-Instruct and Llama-3.1-70B-Instruct) across 4 EDGAR datasets that vary in prompt length and complexity. The benchmark also measures fidelity: how closely the optimized model's output matches the same model running at native precision on a STAC reference SUT. In total, the specification calls for 7 required workloads. Table 1. Model, dataset, and execution modes of the STAC-AI benchmarkModelDatasetBatchInteractiveLlama-3.1-8BEDGAR4aRequiredRequiredLlama-3.1-8BEDGAR5aRequiredRequiredLlama-3.1-70BEDGAR4bRequiredRequiredLlama-3.1-70BEDGAR5bRequired—The stackHardware:Supermicro SuperServer SYS-222C-TN, a 2U DC-MHS server2x Intel Xeon 6730P CPUs (64 physical cores total)2 TiB DDR5-5200 (32x 64 GiB DIMMs)2x NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs (96 GiB GDDR7 each)Software:Red Hat OpenShift 4.20 (single-node OpenShift)Red Hat Enterprise Linux CoreOS 9.6Node feature discovery operatorNVIDIA GPU operatorSTAC-AI Operator (custom operator developed by Red Hat)NVIDIA TensorRT-LLM 1.2.0rc2 (PyTorch backend) with NVFP4 quantization via NVIDIA Model Optimizer 0.37.0The operator stack follows the same pattern we established in our STAC-A2 work. The node feature discovery operator is the prerequisite: it discovers hardware features on each node and surfaces them as Kubernetes labels. The NVIDIA GPU operator uses those labels to determine where to deploy the GPU driver containers, making GPUs available to the scheduler as nvidia.com/gpu resources. From that point on, a pod requests GPUs in the same way that it requests CPU or memory.With these 2 operators installed, GPUs become first-class Kubernetes resources. No manual driver installation, special host configuration, or custom kernel modules are required. The same approach works whether you have a single GPU node at the edge or a cluster of them in a datacenter.On top of that foundation, we added the STAC-AI Operator to orchestrate the benchmark itself.The STAC-AI OperatorRunning a STAC-AI benchmark end to end involves a lot of moving parts: quantizing the model to the target precision, locking GPU clocks and power limits, executing the benchmark harness across multiple workloads and arrival rates, collecting power and temperature traces synchronously with each run, and, finally, running fidelity analysis against a reference model. Doing all of this manually is time-consuming, error-prone, and difficult to reproduce. The STAC-AI Operator makes the entire workflow declarative.Custom resourcesThe operator introduces 3 custom resource definitions (CRDs):Implementation registers an available LLM backend (such as TensorRT-LLM) along with its container images and supported models and datasets.BenchmarkRun is the primary resource. It declares a single benchmark execution: which implementation to use, which model and dataset to test, the quantization method, GPU allocation, and whether to run post-processing fidelity analysis. The operator handles everything from there.BenchmarkImageBuild manages the one-time container image build for the TensorRT-LLM stack.Lifecycle managementWhen a BenchmarkRun is created, the operator drives it through a state machine:Pending → Queued → Initializing → Quantizing → Building → Running → RunCompleted → PostProcessing → CompletedAt each phase, the operator creates Kubernetes Jobs to perform the work. It automatically discovers and applies maximum GPU clock speeds and power limits using nvidia-smi, manages a sequential execution queue so that only 1 benchmark runs at a time (preventing GPU contention between runs), and handles the fidelity analysis phase using vLLM as the reference. If any step fails, the run transitions to a Failed state with a typed reason code, which makes diagnosis straightforward.Running a benchmarkHere is what it takes to run an audited workload:apiVersion: stac.ai/v1alpha1