Home / Troubleshooting / CUDA Memory Fix
🚨 Memory Error

CUDA Out of Memory Fix

Complete guide to solving memory issues when training Kontext LoRA on 8GB, 12GB, and 16GB GPUs

⚡ Quick Fix (Try This First)

  • Switch to FLUX.1 Dev FP8 or GGUF-Q8 model
  • Use t5xxl_fp8_e4m3fn text encoder instead of FP16
  • Set batch size to 1
  • Close all other applications
  • Restart ComfyUI and try again

🔍 Understanding the Problem

CUDA out of memory errors occur when your GPU's VRAM is insufficient for the training process. FLUX.1 Kontext models are large and memory-intensive, especially when training LoRAs.

GPU VRAM Requirements

GPU Model VRAM FP16 Support FP8 Support GGUF Support Recommended
RTX 3060 8GB No Limited Yes GGUF-Q8
RTX 3070 8GB No Limited Yes GGUF-Q8
RTX 3080 10GB Limited Yes Yes FP8
RTX 3090 24GB Yes Yes Yes FP16
RTX 4060 8GB No Yes Yes FP8/GGUF
RTX 4070 12GB Limited Yes Yes FP8
RTX 4080 16GB Yes Yes Yes FP16
RTX 4090 24GB Yes Yes Yes FP16

💡 Proven Solutions

🎯
Use GGUF-Q8 Models
Easy

Best solution for 8GB GPUs. GGUF-Q8 provides 99% quality of FP16 while using almost half the VRAM.

# Download GGUF-Q8 model wget https://huggingface.co/city96/FLUX.1-dev-gguf/resolve/main/flux1-dev-Q8_0.gguf
⚙️
Switch to FP8 Precision
Easy

Use Kijai's FP8 compressed models for significant memory reduction with minimal quality loss.

# FP8 models use ~6GB less VRAM flux1-dev-fp8.safetensors t5xxl_fp8_e4m3fn.safetensors
🔧
Optimize Training Parameters
Medium

Adjust training settings to reduce memory footprint without sacrificing quality.

batch_size: 1 lora_rank: 8-12 (instead of 16+) gradient_accumulation: 1 precision: fp8
💾
Enable CPU Offloading
Medium

Offload some operations to system RAM when you have 32GB+ system memory.

# In ComfyUI settings split_mode: true cpu_offload: true
🧹
Clear GPU Memory
Easy

Free up GPU memory by clearing cache and closing other applications.

# In Python/ComfyUI console import torch torch.cuda.empty_cache() torch.cuda.synchronize()
⚡
Set PyTorch Memory Config
Advanced

Configure PyTorch memory allocation to prevent fragmentation issues.

# Set environment variable PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

📋 Step-by-Step Fix Guide

Check Your Current Setup

First, identify your GPU model and VRAM amount:

# Check GPU info nvidia-smi # or in Python import torch print(f"GPU: {torch.cuda.get_device_name()}") print(f"VRAM: {torch.cuda.get_device_properties(0).total_memory / 1024**3:.1f}GB")

Download Optimized Models

Based on your VRAM, download the appropriate model:

  • 8GB VRAM: Download GGUF-Q8 or FP8 models
  • 12GB VRAM: Download FP8 models
  • 16GB+ VRAM: Can use FP16 with optimizations

Update ComfyUI Configuration

Modify your ComfyUI settings for memory optimization:

# In extra_model_paths.yaml or settings checkpoints: - path: models/checkpoints/flux1-dev-fp8.safetensors text_encoders: - path: models/text_encoders/t5xxl_fp8_e4m3fn.safetensors

Adjust Training Parameters

Use memory-efficient training settings:

learning_rate: 2e-4 batch_size: 1 lora_rank: 12 alpha: 24 precision: fp8 gradient_checkpointing: true

Test and Monitor

Start training and monitor memory usage:

# Monitor VRAM usage during training watch -n 1 nvidia-smi

⚠️ Common Mistakes to Avoid

❌ Don't mix precision types Using FP16 text encoder with FP8 model can cause compatibility issues.
❌ Don't ignore system RAM Insufficient system RAM (less than 16GB) can cause slowdowns and crashes.
❌ Don't use high batch sizes Batch sizes above 1 exponentially increase memory usage.
❌ Don't skip model cleanup Not clearing GPU cache between training sessions wastes memory.

✅ Success Indicators

✅ Training starts without errors No CUDA out of memory messages in the console.
✅ Stable memory usage VRAM usage stays below 90% of total capacity.
✅ Consistent performance Training progresses smoothly without memory spikes.

🆘 Still Having Issues?

If you're still experiencing memory problems after trying these solutions:

  • Check system requirements: Ensure you have adequate system RAM (16GB+)
  • Update drivers: Use the latest NVIDIA drivers and CUDA toolkit
  • Try cloud solutions: Consider using Google Colab or cloud GPU services
  • Join the community: Get help from other users in our Discord and Reddit communities
💡 Pro Tip Consider upgrading to a GPU with more VRAM if you frequently work with AI models. The RTX 4070 (12GB) or RTX 4080 (16GB) provide much better headroom for training.