What version do you use and how do you run Qwen3.6? I’ve played around a bit with the Q4 version in LM studio +Zed, but I was not happy with the results. It looses track very often and often enters infinite loops or just stops…
I use an exl3, with 4 bit MLPs but higher bit depth attention layers. And I force some custom sampling so I can lower the temperature a bit while keeping it out of loops.
This won’t work in LM Studio though. You have to run such a thing in TabbyAPI or some other backend that supports exllamav3.
What version do you use and how do you run Qwen3.6? I’ve played around a bit with the Q4 version in LM studio +Zed, but I was not happy with the results. It looses track very often and often enters infinite loops or just stops…
I use an exl3, with 4 bit MLPs but higher bit depth attention layers. And I force some custom sampling so I can lower the temperature a bit while keeping it out of loops.
This won’t work in LM Studio though. You have to run such a thing in TabbyAPI or some other backend that supports exllamav3.
Ok, thanks! Thaf sounds quite advanced, but I’ll have a read afterwards :)