Feels like a minds war waged by billionaires in this space when you’re actually playing with this stuff. All this hype is a joke as are the proprietary junk. Get a decent comp and try offline AI yourself and see what it can do. Try Llama2 70B Q4 GGML. You need a machine with more than 10-12+ cores and at least 64GB of system memory. It really helps to have a Nvidia GPU with 16GB+ but you don’t have to have that here. This model can write python snippets like you’re searching stack overflow but an order of magnitude faster. If you know basic code elements, branching, and looping, this model can code, and resolve its errors when it gets something wrong by prompting it with the error message. A 30B like WizardLM or Vicuna are almost technically useful, but the 70B is a beast.
Feels like a minds war waged by billionaires in this space when you’re actually playing with this stuff. All this hype is a joke as are the proprietary junk. Get a decent comp and try offline AI yourself and see what it can do. Try Llama2 70B Q4 GGML. You need a machine with more than 10-12+ cores and at least 64GB of system memory. It really helps to have a Nvidia GPU with 16GB+ but you don’t have to have that here. This model can write python snippets like you’re searching stack overflow but an order of magnitude faster. If you know basic code elements, branching, and looping, this model can code, and resolve its errors when it gets something wrong by prompting it with the error message. A 30B like WizardLM or Vicuna are almost technically useful, but the 70B is a beast.
That doesn’t sound simple by any measure.
What they’re trying to say is: very soon these models will run on your smartphone without internet connection and OpenAi will be no more.
The hardware is massively high end for sure. Question is, is it worth it?
In 5-10 years time these will be “recommended system requirements”