You can run the Gemma 4 and Qwen3.5 MoE models with as little as 12 GB of VRAM at 30-40 tps (Q4/Q5), and they both blow GPT-4o and DeepSeek R1 out of the water. But 64gb RAM is also not really out of scale with the cost of a shop tool in other trades. If you’re a professional that’s confident in a positive return on the investment, or just a hobbyist with the luxury budget for a “shop” that cost is well within consumer market. That’s not everybody, of course, but it’s not some inconceivable fantasy.
The key point is that local models continue to get more efficient and usable. You need high end consumer grade hardware today, but given how fast improvements are happening, it’s entirely likely that you’ll be able to get the same capability on even smaller hardware in a few months.
I really appreciate you taking the time for the reply.
From your point of view this makes sense of course and I hope you are right about the upcoming improvements. I did some experiments with a M1 Mac mini and was quickly disappointed but maybe I’ll give it another shot.
Thanks again, I’m always open to be corrected and love to learn new stuff.
You can run the Gemma 4 and Qwen3.5 MoE models with as little as 12 GB of VRAM at 30-40 tps (Q4/Q5), and they both blow GPT-4o and DeepSeek R1 out of the water. But 64gb RAM is also not really out of scale with the cost of a shop tool in other trades. If you’re a professional that’s confident in a positive return on the investment, or just a hobbyist with the luxury budget for a “shop” that cost is well within consumer market. That’s not everybody, of course, but it’s not some inconceivable fantasy.
The key point is that local models continue to get more efficient and usable. You need high end consumer grade hardware today, but given how fast improvements are happening, it’s entirely likely that you’ll be able to get the same capability on even smaller hardware in a few months.
I really appreciate you taking the time for the reply. From your point of view this makes sense of course and I hope you are right about the upcoming improvements. I did some experiments with a M1 Mac mini and was quickly disappointed but maybe I’ll give it another shot. Thanks again, I’m always open to be corrected and love to learn new stuff.