LLM Garage

Scaling up to five RTX 3090s, the new Minimax M2.5 as daily driver, and the refactored website

Getting the massive machine up and running: motherboard issues, RAM troubleshooting, and power solutions

Starting the engineering journal: from hobbyist hacker to building LLM inference systems with off-the-shelf components

A scalable multi-GPU inference system designed for expansion from 2 to 6 GPUs with Threadripper backbone

Technical analysis of PCIe lanes, CPU selection, and the workstation motherboard that makes it all possible

Vector db RAG for your coding agents: a cross-platform memory system built with PostgreSQL + pgvector and MCP

6-bit quantization meets 122B parameters: high-quality MoE inference at 40+ tokens/sec

1-bit quantization meets 397B parameters: the largest model we've run yet

Minimax M2.1 (IQ2_M): slumming it with lower quant precision

Performance and memory analysis of an 80B MoE model with Q5_K_XL quantization

Performance and memory analysis of a 30B MoE model with Q5_K_M quantization

Extreme quantization test: 358B model at 1-bit vs 30B model at 5-bit

MoE CPU offloading to fit a 120B parameter model on 48GB VRAM + 64GB RAM

Discovering 7x speedup with Mixture of Experts models: Nemotron-3-Nano vs Qwen3-32B benchmark

Optimizing multi-GPU setups, testing context windows, and workarounds for 65K token limits

Running MiniMax M2.5 through a power limit sweep (100W-350W per GPU) to understand efficiency and performance tradeoffs

How capping RTX 3090 power consumption to 200W cuts electricity costs by ~43%, reduces heat, and extends GPU lifespan

Quantization techniques and memory optimizations to support large context windows on limited VRAM

Real-world llama.cpp server setup, framework comparison, and troubleshooting journey

Troubleshooting memory management, port conflicts, and common server failure patterns

Lightweight tool for measuring LLM inference performance across configurations