☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.mlEnglish · 19 hours agoGPT-5, the world best model just 1 year ago, is today inferior to Qwen3.6 27B that you can run on your desktoplemmy.mlimagemessage-square45fedilinkarrow-up194arrow-down110
arrow-up184arrow-down1imageGPT-5, the world best model just 1 year ago, is today inferior to Qwen3.6 27B that you can run on your desktoplemmy.ml☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.mlEnglish · 19 hours agomessage-square45fedilink
minus-squareHubi@feddit.orglinkfedilinkarrow-up1·7 hours agoYou can run it on systems with as little as 32 GBs of RAM.
minus-squareelucubra@sopuli.xyzlinkfedilinkarrow-up4·7 hours agoYou don’t want to run these models in RAM. I started using them on an RTX 3060-12Gb VRAM, and quickly added another, for 24Gb RAM. I also have 64 Gb RAM. Now it works well. Not lightning fast, but quite useable. VRAM is the key.
minus-squareHubi@feddit.orglinkfedilinkarrow-up1·edit-26 hours agoSure, but with a model of that size you are unloading layers in any case. I’m just saying you need 32 GBs of RAM at the minimum to be able to run it.
You can run it on systems with as little as 32 GBs of RAM.
You don’t want to run these models in RAM. I started using them on an RTX 3060-12Gb VRAM, and quickly added another, for 24Gb RAM. I also have 64 Gb RAM. Now it works well. Not lightning fast, but quite useable. VRAM is the key.
Sure, but with a model of that size you are unloading layers in any case. I’m just saying you need 32 GBs of RAM at the minimum to be able to run it.