# LLMPerf

> LLMPerf is an open source, self-hosted LLM inference benchmarking tool from Ray Project, scripted in Python. It is discontinued.

- Canonical: https://perf.jmeter.ai/tools/llmperf/
- Official: https://github.com/ray-project/llmperf
- Source: https://github.com/ray-project/llmperf
- Vendor: Ray Project
- Category: AI/LLM Inference
- License: Open Source — Free; Apache License 2.0.
- Deployment: Self-hosted
- Status: Discontinued (successor: GuideLLM)
- Scripting: Python
- Protocols: HTTP, HTTPS
- OS support: Linux, macOS
- First released: 2023

- Last verified: 2026-08-12

## What it is
LLMPerf generates concurrent requests to language-model APIs and reports throughput and latency for inference workloads. The repository is archived, so it remains useful as a historical benchmark reference rather than an actively maintained runner.

## Choose it when
- Token-level latency and throughput benchmarks against LLM inference endpoints
- Teams that prefer open-source licensing and self-hosting options
- Groups comfortable scripting in Python

## Look elsewhere when
- New greenfield programs (prefer an active successor or alternative)
- Teams unwilling to operate generators, results storage, and scaling

## Architecture
LLMPerf is commonly operated as self-hosted software with scenario authoring in Python. Concurrency and distribution depend on how you size workers or injectors.

## Pricing
Free; Apache License 2.0.

## Compare

## FAQ
### Who is LLMPerf for?
LLMPerf is best for teams benchmarking LLM and generative AI endpoints on token latency and throughput. It is open source, so teams can self-host and extend it. Vendor: Ray Project. Deployment model: self-hosted. Free; Apache License 2.0. Confirm details against the official site and a short proof-of-concept in your environment.

### How does LLMPerf differ from I am speed?
Compared with I am speed, deployment is self-hosted versus hybrid for I am speed; scripting centers on Python rather than None. LLMPerf lists protocols HTTP, HTTPS; I am speed lists HTTPS, SSE. Status is discontinued versus active. Use the Test Rig to compare full specs side by side before a proof-of-concept.

### What scripting language and concurrency model does LLMPerf use?
LLMPerf uses Python for scripting or scenario definition. Runtime model: code-first scenarios where virtual users execute scripted behavior under load. OS support: Linux, macOS. Protocols: HTTP, HTTPS. Match these to how your team already authors tests and which systems you must drive under load.

### What is the cloud or enterprise path for LLMPerf?
LLMPerf is primarily self-hosted; teams typically run generators on their own machines, CI runners, or private cloud. Plan for generator capacity, network access to targets, and result storage yourself. As open source, commercial support or hosted siblings may exist separately. Check the official site. Pricing note: Free; Apache License 2.0.

### What are known limitations of LLMPerf?
LLMPerf appears as discontinued in this directory. Prefer GuideLLM or another active alternative for new work. Historical references remain so older docs and diagrams still resolve. Verify any remaining commercial support directly with the vendor before depending on it. Confirm details against the official site and a short proof-of-concept in your environment.

---
Curated by NaveenKumar Namachivayam (QAInsights) · methodology: https://perf.jmeter.ai/about/#methodology · corrections: https://github.com/QAInsights/Performance-Testing-Tools/issues/new?title=Tool%20submission%3A%20&body=Tool%20name%3A%20%0AOfficial%20URL%3A%20%0AWhat%20should%20be%20added%20or%20corrected%3F%20%0A