I can run both the 27B and 35B version of Qwen3.6 on my 10GB VRAM 3080, and they run ok. You don’t need to put the entire model in VRAM, even if it’s probably beneficial.
Full quantisation? I’ve only got a 8GB 3070, but I’ll give it a go
Edit: Tried the unsloth/qwen3.6 with llama.CPP, and it failed to allocate a 26GB Vulcan buffer and died. Dunno what magic your using, no luck for me though :(
I run it using LM Studio, which defaults to Q4 quantization, I think. I was able to put about 10 layers on the GPU with 64k token context. That put me at about 9.1 GB VRAM usage, leaving some room for Video playback xD
I can run both the 27B and 35B version of Qwen3.6 on my 10GB VRAM 3080, and they run ok. You don’t need to put the entire model in VRAM, even if it’s probably beneficial.
Full quantisation? I’ve only got a 8GB 3070, but I’ll give it a go
Edit: Tried the unsloth/qwen3.6 with llama.CPP, and it failed to allocate a 26GB Vulcan buffer and died. Dunno what magic your using, no luck for me though :(
I run it using LM Studio, which defaults to Q4 quantization, I think. I was able to put about 10 layers on the GPU with 64k token context. That put me at about 9.1 GB VRAM usage, leaving some room for Video playback xD
I’ll give LM studio a go, thanks.