☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.mlEnglish · 19 hours agoGPT-5, the world best model just 1 year ago, is today inferior to Qwen3.6 27B that you can run on your desktoplemmy.mlimagemessage-square47fedilinkarrow-up1100arrow-down110
arrow-up190arrow-down1imageGPT-5, the world best model just 1 year ago, is today inferior to Qwen3.6 27B that you can run on your desktoplemmy.ml☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.mlEnglish · 19 hours agomessage-square47fedilink
minus-squarestuner@lemmy.worldlinkfedilinkarrow-up3·5 hours agoI run it using LM Studio, which defaults to Q4 quantization, I think. I was able to put about 10 layers on the GPU with 64k token context. That put me at about 9.1 GB VRAM usage, leaving some room for Video playback xD
minus-squareCameronDev@programming.devlinkfedilinkarrow-up2·4 hours agoI’ll give LM studio a go, thanks.
I run it using LM Studio, which defaults to Q4 quantization, I think. I was able to put about 10 layers on the GPU with 64k token context. That put me at about 9.1 GB VRAM usage, leaving some room for Video playback xD
I’ll give LM studio a go, thanks.