KV Cache Size Calculator
Loading model configurations...
Model:
Data Type:
float16 (FP16)
bfloat16 (BF16)
float32 (FP32)
int8 (INT8)
Number of Tokens:
Calculate KV Cache Size
Reverse Calculator: Find Maximum Tokens
Model:
Data Type:
float16 (FP16)
bfloat16 (BF16)
float32 (FP32)
int8 (INT8)
GPU RAM Size (GB):
Calculate Maximum Tokens