QPS & Latency Calculator

Frequently Asked Questions

Does Little's Law assume any specific distribution?

No - it holds for any stable queuing system under general arrival and service time distributions, with no assumptions about Poisson arrivals or exponential service times.

What does 'stable system' mean?

A stable system is one where the arrival rate does not persistently exceed the service capacity. If requests arrive faster than they can be processed, the queue grows without bound.

Why do I need headroom above the theoretical concurrency?

Traffic is bursty, so instantaneous concurrency spikes above the average. Without headroom, burst arrivals cause queueing, which raises latency, which increases concurrency - a feedback loop that can saturate the system.

How does this differ from M/M/1 queuing theory?

M/M/1 adds specific distribution assumptions to predict how latency grows as utilization rises. Little's Law is distribution-free and relates only steady-state averages - it does not predict queueing delay growth.

Provided by AllCalculators.io
Free online calculators for everyday. No registration required.

Estimates for informational purposes only.

Important Disclaimer: Estimates for informational purposes only.

This calculator provides estimates for informational purposes only. Results are based on assumptions and may not reflect actual outcomes. Consult qualified professionals in relevant fields before making important decisions based on these results.

JavaScript is required to use the interactive calculator above. The questions and answers below remain readable without JavaScript.

How It Works

Little's Law is the foundational equation for capacity planning: L = λ × W, where L is the average number of requests in the system (concurrency), λ is the arrival rate (QPS), and W is the average time each request spends in the system (latency). The law holds for any stable system regardless of internal scheduling details or traffic distributions.

This calculator solves for whichever of the three quantities you don't know. Give it QPS and average latency to find the required concurrency - and, after applying a target utilization factor, the number of worker threads or connections to provision. Give it concurrency and latency to find the max sustainable QPS. Give it concurrency and QPS to back out the implied average latency.

Use Cases

Little's Law applies to virtually every server-side capacity planning problem:

Sizing a database connection pool: how many connections does a service need given its QPS and average query latency?

Provisioning thread pool workers for an HTTP server to handle peak traffic without queueing

Understanding why a service starts dropping requests - at 100% utilization, any additional load causes unbounded queueing

Setting concurrency limits on outbound API calls to a rate-limited external service

Explaining to stakeholders why a service with only 20 concurrent requests can still handle 500 QPS

Tips

Little's Law requires the average end-to-end residence time, not a percentile. The p99 latency is much higher than the mean - using it will over-provision significantly. Use the arithmetic mean measured over the same window as your QPS measurement.

Always provision for headroom: a pool at 100% utilization has zero slack for traffic spikes, and queueing delay rises sharply as utilization approaches 100%. A 70% target utilization is a common default for steady services.

Latency and QPS must use consistent time units. If QPS is in requests/second, latency must be in seconds.

For async/event-loop servers (Node.js, Go with goroutines), concurrency maps to outstanding requests, not OS threads. Size your downstream connection pools and semaphores to the concurrency value L.

If your latency increases at high load (as it usually does), iterate: higher concurrency may push latency up, which then requires even more concurrency per Little's Law - validate with a realistic load test.

FAQ

Does Little's Law assume any specific distribution?

No - that's what makes it powerful. Little's Law holds for any stable queuing system under general arrival and service time distributions, with no assumptions about Poisson arrivals or exponential service times. It requires only that the system be in a stable state where the arrival rate does not exceed the service capacity.

What does 'stable system' mean?

A stable system is one where the arrival rate does not persistently exceed the service capacity. If requests arrive faster than they can be processed, the queue grows without bound and the law no longer gives a finite L. An overloaded system needs more capacity, not a better formula.

Why do I need headroom above the theoretical concurrency?

Little's Law gives the average concurrency. Traffic is bursty, so instantaneous concurrency can spike above the average. Without headroom, burst arrivals cause queueing, which raises the effective latency W, which increases L, creating a feedback loop that can tip a lightly overloaded system into complete saturation.

How does this differ from M/M/1 queuing theory?

M/M/1 and related queuing models add specific distribution assumptions (Poisson arrivals, exponential service times) to predict the latency distribution under load - including how much latency increases as utilization rises. Little's Law is distribution-free and just relates the steady-state averages; it does not predict how queueing delay grows as you approach capacity.

QPS & Latency Calculator

Frequently Asked Questions

Related Developer Tools Calculators