Inference PerfKubernetes SIGs
Model-server-agnostic GenAI inference benchmarking tool that measures TTFT, TPOT, throughput, and goodput.
AI/LLM Inference Open Source Self-hosted
| TOOL / VENDOR | DESCRIPTION | CLASS | LOAD PROFILE | COMPARE |
|---|---|---|---|---|
| Inference PerfKubernetes SIGs | Model-server-agnostic GenAI inference benchmarking tool that measures TTFT, TPOT, throughput, and goodput. | AI/LLM Inference Open Source Self-hosted | ||
| AIPerfNVIDIA | NVIDIA CLI that profiles generative AI endpoints and reports LLM latency and throughput percentiles. | AI/LLM Inference Open Source Self-hosted | ||
| I am speedNaveenKumar Namachivayam | Fast.com-style browser benchmark for LLM APIs, streaming live tokens/sec, TTFT, and TTLT per provider. | AI/LLM Inference Open Source Hybrid
personal pick
| ||
| LLMPerfRay ProjectDiscontinued | Open-source benchmark suite for measuring large-language-model inference performance. | AI/LLM Inference Open Source Self-hosted | ||
| GuideLLMvLLM | Open-source LLM serving benchmark for capacity planning and endpoint performance analysis. | AI/LLM Inference Open Source Self-hosted |
Model-server-agnostic GenAI inference benchmarking tool that measures TTFT, TPOT, throughput, and goodput.
NVIDIA CLI that profiles generative AI endpoints and reports LLM latency and throughput percentiles.
Fast.com-style browser benchmark for LLM APIs, streaming live tokens/sec, TTFT, and TTLT per provider.
Open-source benchmark suite for measuring large-language-model inference performance.
Open-source LLM serving benchmark for capacity planning and endpoint performance analysis.