llm-memory-calculator

Interactive memory calculator for large language models with 3D visualization

HTML

public

LLM Memory Calculator

A standalone HTML/JavaScript application for calculating GPU memory requirements for large language models (LLMs). This tool assists AI practitioners in determining hardware requirements for inference, fine-tuning, and training from scratch.

Features

Calculate memory requirements for models ranging from 7B to 175B+ parameters
Support for different precision levels (32-bit, 16-bit, 8-bit, 4-bit, and Q4_K_M quantization)
Estimates for different operation modes (inference, fine-tuning, training)
Hardware recommendations for both NVIDIA GPUs and Apple Silicon
Interactive 3D visualization of memory allocation components
Single HTML file with no server requirements

Usage

Online Version

You can access the online version of this calculator at: https://deadjoe.github.io/llm-memory-calculator/

Local Usage

Simply open llm-memory-calculator.html or index.html in any modern web browser. No installation or server setup required.

Hosting Your Own Copy

To host this calculator on GitHub Pages:

Fork this repository
Go to your fork’s Settings tab
Navigate to “Pages” in the left sidebar
Under “Build and deployment”, select “Deploy from a branch”
Select “main” branch and “/ (root)” folder
Click “Save”
After a few minutes, your calculator will be available at https://yourusername.github.io/llm-memory-calculator/

Memory Calculation Algorithm

The calculator uses a comprehensive algorithm to estimate memory requirements based on real-world usage patterns rather than theoretical minimums. Below is the detailed calculation methodology:

Core Memory Components

Base Model Size (GB):
- Parameter Count (P) × Bytes per Parameter
- Bytes per parameter varies by precision:
  - 32-bit (FP32): 4 bytes/parameter
  - 16-bit (FP16/BF16): 2 bytes/parameter
  - 8-bit Quantized: 1.2 bytes/parameter (includes quantization overhead)
  - 4-bit Quantized: 0.65 bytes/parameter (includes metadata overhead)
  - Q4_K_M Quantized: 0.59 bytes/parameter (more efficient 4-bit quantization)
Framework Overhead:
- 15% additional to base model size
- Accounts for PyTorch/TensorFlow runtime memory allocation
Operation Mode Multiplier:
- Inference: 1.05× multiplier (5% overhead)
- Fine-tuning: 2.5× multiplier (150% overhead)
- Training from Scratch: 4.0× multiplier (300% overhead)
Inference-specific Components (only applied in inference mode):
- KV Cache:
  - KV Cache (GB) = (Num Layers × 2 × Hidden Size × Context Length × Bytes per Parameter) ÷ 10^9
  - Uses model-specific values for layers and hidden dimensions:
    - 7B models: 32 layers, 4096 hidden dimension
    - 13B models: 40 layers, 5120 hidden dimension
    - 24B models: 48 layers, 6144 hidden dimension
    - 34B models: 60 layers, 6656 hidden dimension
    - 70B models: 80 layers, 8192 hidden dimension
    - 175B+ models: 96 layers, 12288 hidden dimension
  - Uses 8192 as the default context length
- Activation Memory:
  - Activation Memory (GB) = (Hidden Size × Context Length × Bytes per Parameter × 2) ÷ 10^9
CUDA/Runtime Buffer:
- 8% of total memory allocation
- Accounts for CUDA workspace and system buffers
System Architecture Factor:
- PC/NVIDIA: 1.0× (standard)
- Apple Silicon: 0.9× (unified memory efficiency)
Hardware Recommendation Safety Margin:
- 10% additional buffer for recommendations
- Ensures comfortable operation without memory pressure

Complete Formula

For inference mode:

$\text{Memory}_{\text{GB}} = \left[\left(\text{Base Model Size} \times 1.15 \times 1.05\right) + \text{KV Cache} + \text{Activation Memory} + \text{Buffer}\right] \times \text{System Factor}$

Where:

$\text{KV Cache}_{\text{GB}} = \frac{\text{Num Layers} \times 2 \times \text{Hidden Size} \times \text{Context Length} \times \text{Bytes per Param}}{10^9}$

$\text{Activation Memory}_{\text{GB}} = \frac{\text{Hidden Size} \times \text{Context Length} \times \text{Bytes per Param} \times 2}{10^9}$

$\text{Buffer}_{\text{GB}} = \text{Total Memory Before Buffer} \times 0.08$

For training/fine-tuning:

$\text{Memory}_{\text{GB}} = \left[\left(\text{Base Model Size} \times 1.15 \times \text{Mode Factor}\right) + \text{Buffer}\right] \times \text{System Factor}$

Hardware Recommendations

The calculator accounts for actual usable VRAM in its recommendations by reserving:

~2-8GB for system/CUDA runtime on NVIDIA GPUs
~2-20GB for system needs on Apple Silicon (depending on total RAM)

Development

The calculator is built with vanilla JavaScript and Three.js for visualizations. All computation happens client-side with no external API dependencies.

Technologies Used

HTML5 and CSS3 for layout and styling
Vanilla JavaScript (ES6+) for calculations
Three.js for 3D memory visualization
KaTeX for formula rendering
Tween.js for smooth animations

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

Developed based on practical experience and benchmarks from running large language models
Memory estimation formulas derived from real-world LLM deployment scenarios

Find me

v0.3.3[beta]