hardware

Escape the API Mines: How Far Can One RTX 4090 Actually Go?

I bought a $1,800 GPU to stop paying OpenAI. Then I learned that VRAM doesn't care about my feelings. Here is the brutal math on running Llama-3 70B on a single card.