What hardware that needs? My issue with running local models was that it’s too much of a resource hog to be able to do gamedev on the same machine, and any sensible model needs pretty expensive hardware to just get a server for it. Especially with current prices.
Mac Minis top out at 48GB and are 1.8k when configured like that. It’s going to be at least $2k to buy anything that has a hope of running it at a reasonable speed.
Running local isn’t free, but at least it’s just a single upfront payment.
Yep, while I don’t use them myself, I saw the output of the latest models at the beginning of May. While there are some “good” things in it, the vast majority of the output was unnecessary maintenance load or just wrong. And, while the person showing off the output claimed they couldn’t have written the code, I didn’t see anything particularly special.
On top of that, I don’t believe the output of Qwen (or any other coding model) can be distributed without violating a large number of copyrights, so it’s entirely inappropriate for FOSS projects.
I don’t believe the output of Qwen (or any other coding model) can be distributed without violating a large number of copyrights
I have a perfect example for that. I asked Qwen to write a simple python socket app. one for server and one for client.
While I was reading through forum posts about python socket communication, I found a post from 8 years ago. same script. same variable names. same comments. word for word. line for line. the same exact script.
I run Qwen 3.6 27B at home. For “free”. It is extremely useful.
My point being that I’m not going to be priced out of using it
Don’t worry, they want to replace your hardware with a “cloud based computing solution” as well.
When did that absurdity come back? I thought we killed the cloud computer nonsense a decade ago.
Well you see… subscriptions.
What hardware that needs? My issue with running local models was that it’s too much of a resource hog to be able to do gamedev on the same machine, and any sensible model needs pretty expensive hardware to just get a server for it. Especially with current prices.
Geforce 3090 with 24TB should be able to run a “Q5 version” of it. Maybe get a second older computer, or maybe you can run two cards in one PC.
64GB unified memory. I run it (and a lot more) on a dgx spark, but a Mac mini would suffice also.
You could prob run 4-bit version on a RTX card with 32g. Maybe even 24g. Like a 5090 or 4090 or such.
So much info out there.
Mac Minis top out at 48GB and are 1.8k when configured like that. It’s going to be at least $2k to buy anything that has a hope of running it at a reasonable speed.
Running local isn’t free, but at least it’s just a single upfront payment.
The M4 Pro Mac Mini caps out at 64GB RAM. Whether or not Apple can sell you that SKU right now is a different question with the ongoing DRAM shortage.
That (64GB) doesn’t appear on the site at the moment.
Well you aren’t a brain dead business man then.
…yet
qwen is garbage. it can’t even count the elements within an array of numbers.
to be clear though, it’s not just qwen. all code models are fucking trash.
See, this is what people say when they say “people who can code” are doing good things with these LLMs.
Why the fuck would you ask the model to count elements?
Ask it to make a python script that will do the counting, then run the script.
are these not legitimate questions? sure I could do them in-code, but is it not faster to just ask it?
first time I ever had a clanker insinuate my skill level is below their own. thanks for the chuckle.
Ok. Also I am sorry the audience of Stack Overflow dried up for folks to use as punching bags.
what are you even talking about?
Are you sure you were using the actual coding model? There are a number of them
Qwen coder 30B A3B
Yep, while I don’t use them myself, I saw the output of the latest models at the beginning of May. While there are some “good” things in it, the vast majority of the output was unnecessary maintenance load or just wrong. And, while the person showing off the output claimed they couldn’t have written the code, I didn’t see anything particularly special.
On top of that, I don’t believe the output of Qwen (or any other coding model) can be distributed without violating a large number of copyrights, so it’s entirely inappropriate for FOSS projects.
I have a perfect example for that. I asked Qwen to write a simple python socket app. one for server and one for client.
While I was reading through forum posts about python socket communication, I found a post from 8 years ago. same script. same variable names. same comments. word for word. line for line. the same exact script.
so much for AI “not stealing content”.
most people are going to destroy their home servers running these workloads
Destroy as in the fan bearings are going to wear out quicker?