ai and ml
Third-party testing shows heterogeneous compute platform combining H200s and SN50 RDUs churning out 763 tok/s in MiniMax M2.7
Intel's big bet on SambaNova appears to be paying off in a big way. This week, the AI chip startup shared benchmark results showing its latest generation of AI acceleration, which combines Nvidia GPUs and the company's accelerators, beating GPU-only inference platforms by a wide margin.The testing, conducted by the AI benchmarking gurus at Artificial Analysis, showed SambaNova's SN50-series accelerators, announced in February, churning out 763 tokens a second in MiniMax M2.7 at short context lengths (10,000 input tokens) — several times faster than competing inference providers running on GPUs alone.Meanwhile, for longer context lengths, the company says that its platform is able to sustain more than 450 tokens a second.
At short context lengths (10,000 tokens) Artificial Analysis found SambaNova's heterogeneous compute platform, which combined four H200 GPUs with 16 of its SN50 RDUs was able to achieve decode speed of 763 tok/s in MiniMax M2.7.
Artificial Analysis
