I started this site, and my Ironhorse build with the express goal of getting Minimax M2.1 to run locally. And now we're getting somewhere. I'm more excited about local models than ever. GLM-4.7-Flash cracked things wide open, and subsequent models Qwen3-Coder-Next, and Minimax M2.5 have blown me away. We have gotten so much more.
Hardware Upgrades
I continue to upgrade my hardware, and we are now running five (5) RTX 3090s, with another Founder's Edition on the way. To get here, I bought an EVGA Supernova 1600W power supply, returned 512 GB ECC RAM ($3,500), in exchange for the GPUs. We now have enough power to light up my city block, and do some serious damage to my electrical bill.
UPDATE: We are now running six (6) RTX 3090s, giving us up to 144GB of VRAM. This allows for full context with unquantized KV cache—we're incredibly happy with Minimax M2.5. To avoid tripping the breaker, I've set the power limit to 200W per GPU.
Model Performance
So the decision to move to higher levels of VRAM was a good one. We are getting full context now with Unsloth's Q5_K_XL quant for Qwen3-Coder-Next. It's a fabulous model. Meanwhile, we jumped into the low quant sandbox, and picked up both IQ2_M quants for Minimax M2.1 and M2.5.
While I haven't explored all my options so far, I have gravitated to using Unsloth GGUFs paired with llama.cpp. The Unsloth Dynamic Quantization 2.0 creates compact models that I feel don't suffer from severe degradation in quality. Frankly, a slightly nerfed, but new model is still better than the models of last year. So, I'm a believer until somebody shows me something better.
We are currently constrained to about 100k tokens before hitting OOM errors with M2.5. We'll have to wait for the 6th GPU before we have full context (198k tokens) on M2.5.
Daily Driver
I am now using Minimax M2.5 as my daily driver, and honestly, the model is great. We updated our basic TPS benchmark script to measure prefill tok/s, generation tok/s, and time-to-first-token (TTFT), a good measure of how good or shitty the user experience will be.
The new benchmark script says we are generating about 72 tokens/sec, which I am thrilled for. With the new capabilities at our disposal, I tasked M2.5 with studying how to implement the Ralph Wiggum technique for a more hands-off experience. Learning Ralph was something I was loathe to do—the perfect type of task to set an agent to figure out.
Website Refactor
M2.5 created the PROMPT.md and took my laundry list of tasks to do for our website. Since yesterday, it's refactored our LLM Garage to use Astro (some static website framework as an alternative to React), added some basic navigation, and organized the whole damn project. I take full blame for the appearance of the website, as these choices were mine.
Cost Analysis
Cost-wise, we are approaching 256GB Mac Studio M3 territory, with total spend for the project (including the GPU I owned before) totalling about $8,000. Not cheap, but I'd say the experience is worth it. Not sure my wife would agree.
Bill of Materials
| Manufacturer | Part Number | Description | Qty | Unit Price | Ext Price |
|---|---|---|---|---|---|
| Supermicro | MBD-M12SWA-TF-O | E-ATX Motherboard sWRX8 socket 6x PCIe 4.0 x16 8-ch DDR4 | 1 | $760.52 | $760.52 |
| N/A | N/A | 64 GB PCIe 4.0 RAM 3200 MHz | 1 | $0.00 | $0.00 |
| AMD | 100-000000445 | AMD Ryzen Threadripper PRO 5955WX Processor (UNLOCKED) | 1 | $760.52 | $760.52 |
| Noctua | NH-U14S TR4-SP3 | CPU Cooler for Threadripper (sWRX8/sTR4) | 1 | $107.82 | $107.82 |
| Noctua | NF-A12x25 PWM | 120mm High-Performance Fans | 0 | $35.00 | $0.00 |
| Gigabyte | GV-N3090GAMING OC-24GD | GIGABYTE GeForce RTX 3090 GAMING OC 24GB | 2 | $783.89 | $1,567.78 |
| Nvidia | 900-1G136-2510-000 | Nvidia RTX3090 Founder's Edition | 4 | $1,023.51 | $4,094.05 |
| LINKUP | B08XN26DRZ | LINKUP PCIE 5.0 Riser Cable 40cm black v2 | 6 | $51.42 | $308.52 |
| EVGA | EVGA Supernova 1600 T2 1600w Titanium PSU | 1 | $258.28 | $258.28 | |
| C2G/Cables To Go | 10343 | NEMA 14-30P to Dual NEMA 5-15R Y-Adapter | 1 | $9.76 | $9.76 |
| EVGA | Evga Supernova 1300 G2 80 Plus Gold Modular PSU | 1 | $104.29 | $104.29 | |
| N/A | N/A | Misc VGA to PSU cables | 12 | $0.00 | $0.00 |
| Add2PSU | ADD2PSU-S | Dual PSU Adapter/Sync Cable | 1 | $17.00 | $17.00 |
| Bomeiqee | - | Mining Rig Frame for 12GPU Fans & GPU is not Included | 1 | $70.14 | $70.14 |
| Total | $8,007.27 | ||||
Here's a snapshot of the current machine. If you couldn't tell, I'm pretty proud of it.
What's Next
Future posts will pivot to something I know a lot about actually: power! I have some things cooking for measuring and optimizing system configurations to get the greatest number of tokens per kilowatt hour. I'll gladly share that when it's ready.
So, onward. There's plenty to do, infinite things to learn, and more models to try. Thanks for coming along.
—Ash