They are the way to go, but hardware pricing makes it cost-prohibitive to match frontier capabilities, which makes decomp and recomp difficult. Or rather, they require more skill and take longer, introducing some barriers against the current floodgates being flung open.
We all know the eye-watering prices GPUs and RAM cost now, to say nothing of Unified Memory devices like the Strix. Most people just don’t have a spare $10K to throw at an inference box that can run (say) a 300B model at interactive (not batch) speeds.
Coming at this from another angle tho, I note that according to Steam stats, the most popular GPUs are in the 8GB and 16GB range…which is smack bang in the VRAM range of what a good MoE (say, Qwen 3.6 35B-A3B) runs on (for 8GB) or arguably the best local dense coder (Qwen 3.8 27B) can live at. Which leaves us with a capable assistant and a capable coder.
I think there’s probably room for LLM improvement in the 8–16GB range too…and given that most consumer PCs and laptops live there, I’m sanguine the open-weight devs might make it happen.
But I also suspect the gatekeeping of high-end capacities is intentional.
We’re probably not going to have the ability to vibe code Mario into Elden Ring again…not without significantly more skill. Dunno.
Yes, quite a few, but it depends on what you’re trying to do with them. Any ideas?