Throughput Calculator

Calculate AI model serving throughput and capacity.

Throughput Parameters

Tokens per Second (per instance)

tok/s

Average Input Tokens

Average Output Tokens

Number of Instances

Target Latency (ms)

ms

Requests per Second

0.14

12,343 requests/day

⏱️Requests/Minute

8.6

🕐Requests/Hour

514

Latency target NOT met: 7000ms > 2000ms

Capacity Details

Tokens per Request700

Time per Request7.00s

Tokens per Hour0.36M

Concurrent Capacity1.00 req