…and I still don’t get it. I paid for a month of Pro to try it out, and it is consistently and confidently producing subtly broken junk. I had tried doing this before in the past, but gave up because it didn’t work well. I thought that maybe this time it would be far along enough to be useful.
The task was relatively simple, and it involved doing some 3d math. The solutions it generated were almost write every time, but critically broken in subtle ways, and any attempt to fix the problems would either introduce new bugs, or regress with old bugs.
I spent nearly the whole day yesterday going back and forth with it, and felt like I was in a mental fog. It wasn’t until I had a full night’s sleep and reviewed the chat log this morning until I realized how much I was going in circles. I tried prompting a bit more today, but stopped when it kept doing the same crap.
The worst part of this is that, through out all of this, Claude was confidently responding. When I said there was a bug, it would “fix” the bug, and provide a confident explanation of what was wrong… Except it was clearly bullshit because it didn’t work.
I still want to keep an open mind. Is anyone having success with these tools? Is there a special way to prompt it? Would I get better results during certain hours of the day?
For reference, I used Opus 4.6 Extended.
like… it fixed it when I called it out, but it made the mistake again later on. I was only using it save time coverting, like, 11 or so files, but it made the mistake 3 or 4 times, not only with the encapsulation, but with the case-sensitivity, too. both with paths, although I couldn’t see any particular pattern to it.
just annoying, and I had to read through each compose file just to check it for errors. in the end it did save time, but much less that I thought it would.
If it gets it wrong the first time I rarely reprompt. I know I can get it to fix it, but it’s usually faster for me to do it because I already figured out where and what to do the fix. Low key think it’s just a ploy to get us to burn more tokens. Sure correcting it means it writes a few lines to the memory file, but it’s only a matter of time before it trips over that context as well.
yeah, I also wonder if it’s a ploy. that’s really the only time I’ve used for any kind of code assistance, and I really haven’t used Claude much, overall, but it generally seems more capable than chatGPT, for example. it felt a bit strange that it would make such a simple mistake.