Model Parameter Count Calculator

Calculate transformer model parameter counts and memory requirements.

Model Architecture

Architecture Type

Hidden Size (d_model)

Number of Layers

Number of Attention Heads

Vocabulary Size

Intermediate Size (FFN)

Total Parameters

6.74B

6,738,673,664 parameters

💾FP16 Memory

12.55 GB

📦INT4 Memory

3.14 GB

Embedding Parameters131.1M

Attention Parameters2147.5M

FFN Parameters4328.5M

FP32 (Full Precision)25.10 GB

FP16 (Half Precision)12.55 GB

INT8 (8-bit Quantized)6.28 GB

INT4 (4-bit Quantized)3.14 GB