Inference PerfKubernetes SIGs
Model-server-agnostic GenAI inference benchmarking tool that measures TTFT, TPOT, throughput, and goodput.
AI/LLM Inference Open Source Self-hosted| TOOL / VENDOR | DESCRIPTION | CLASS | LOAD PROFILE | COMPARE |
|---|---|---|---|---|
| Inference PerfKubernetes SIGs | Model-server-agnostic GenAI inference benchmarking tool that measures TTFT, TPOT, throughput, and goodput. | AI/LLM Inference Open Source Self-hosted | ||
| AIPerfNVIDIA | NVIDIA CLI that profiles generative AI endpoints and reports LLM latency and throughput percentiles. | AI/LLM Inference Open Source Self-hosted | ||
| I am speedNaveenKumar Namachivayam | Fast.com-style browser benchmark for LLM APIs, streaming live tokens/sec, TTFT, and TTLT per provider. | AI/LLM Inference Open Source Hybrid ♥ pick |
Model-server-agnostic GenAI inference benchmarking tool that measures TTFT, TPOT, throughput, and goodput.
AI/LLM Inference Open Source Self-hostedNVIDIA CLI that profiles generative AI endpoints and reports LLM latency and throughput percentiles.
AI/LLM Inference Open Source Self-hostedFast.com-style browser benchmark for LLM APIs, streaming live tokens/sec, TTFT, and TTLT per provider.
AI/LLM Inference Open Source Hybrid ♥ pick