TOOL SIGNAL / DISCONTINUED

LLMPerf

Ray Project

AI/LLM Inference Open Source Self-hosted

NOT YET REVIEWED · CATALOG DATA

Official site ↗ Source repo ↗
Pricing model
Free; Apache License 2.0.
First released
2023
Last verified
Answer

LLMPerf generates concurrent requests to language-model APIs and reports throughput and latency for inference workloads. The repository is archived, so it remains useful as a historical benchmark reference rather than an actively maintained runner.

What it is

LLMPerf generates concurrent requests to language-model APIs and reports throughput and latency for inference workloads. The repository is archived, so it remains useful as a historical benchmark reference rather than an actively maintained runner.

Best for

  • Token-level latency and throughput benchmarks against LLM inference endpoints
  • Teams that prefer open-source licensing and self-hosting options
  • Groups comfortable scripting in Python

Not the best fit when

  • New greenfield programs (prefer an active successor or alternative)
  • Teams unwilling to operate generators, results storage, and scaling

Architecture and concurrency

LLMPerf is commonly operated as self-hosted software with scenario authoring in Python. Concurrency and distribution depend on how you size workers or injectors.

Fact sheet

Specs

Scripting
Python
Protocols
HTTPHTTPS
Operating systems
LinuxmacOS
Tags
aillmpythonbenchmark
Repo health
★ 1126 · last push 2024-12-09 ARCHIVED REPO

CATALOG LAST VERIFIED · 2026-08-12 · CURATED BY NAVEENKUMAR NAMACHIVAYAM

Frequently asked

ANSWER-SHAPED
Who is LLMPerf for?

LLMPerf is best for teams benchmarking LLM and generative AI endpoints on token latency and throughput. It is open source, so teams can self-host and extend it. Vendor: Ray Project. Deployment model: self-hosted. Free; Apache License 2.0. Confirm details against the official site and a short proof-of-concept in your environment.

How does LLMPerf differ from I am speed?

Compared with I am speed, deployment is self-hosted versus hybrid for I am speed; scripting centers on Python rather than None. LLMPerf lists protocols HTTP, HTTPS; I am speed lists HTTPS, SSE. Status is discontinued versus active. Use the Test Rig to compare full specs side by side before a proof-of-concept.

What scripting language and concurrency model does LLMPerf use?

LLMPerf uses Python for scripting or scenario definition. Runtime model: code-first scenarios where virtual users execute scripted behavior under load. OS support: Linux, macOS. Protocols: HTTP, HTTPS. Match these to how your team already authors tests and which systems you must drive under load.

What is the cloud or enterprise path for LLMPerf?

LLMPerf is primarily self-hosted; teams typically run generators on their own machines, CI runners, or private cloud. Plan for generator capacity, network access to targets, and result storage yourself. As open source, commercial support or hosted siblings may exist separately. Check the official site. Pricing note: Free; Apache License 2.0.

What are known limitations of LLMPerf?

LLMPerf appears as discontinued in this directory. Prefer GuideLLM or another active alternative for new work. Historical references remain so older docs and diagrams still resolve. Verify any remaining commercial support directly with the vendor before depending on it. Confirm details against the official site and a short proof-of-concept in your environment.