This is just a classic case of bad use of the tools provided. Agents are notorious for making shit up Or getting something that’s just like super close, but not quite accurate.
I bet this dude also probably just uses the same session over and over and over and over again, which clogs up his context window and makes the model less accurate the longer it goes on to.
This probably could have been prevented if it had been forced to show a plan before it tried to do anything. It’s hard to know because the article is so light on details. You also shouldn’t brazenly trust the thing so much. You should run a command and walk away. You should keep an eye on what it is doing.
It’s a bit like giving a junior developer a production key and being like “don’t delete production!” and then walking away.
The way the guy was prompting this agent also leaves a lot to be desired. It’s trained to work on emulating human thoughts, speech patterns. Turns out When giving instructions, it’s really difficult to figure out what to do from a list of things to not do. If the dude just instead told the agent what to do and how he wanted it to work and when it needed to bring things to his attention, instead of telling it to not guess, instead explaining that it needed to use whatever tools to go look up a documentation to understand the context and scope of the project it’s working on It does a better job.
Giving a model the right context to do something is the difference between a model doing something like deleting your production database or your model acting like a magical machine that can get anything done.
This is just a classic case of bad use of the tools provided. Agents are notorious for making shit up Or getting something that’s just like super close, but not quite accurate.
I bet this dude also probably just uses the same session over and over and over and over again, which clogs up his context window and makes the model less accurate the longer it goes on to.
This probably could have been prevented if it had been forced to show a plan before it tried to do anything. It’s hard to know because the article is so light on details. You also shouldn’t brazenly trust the thing so much. You should run a command and walk away. You should keep an eye on what it is doing.
It’s a bit like giving a junior developer a production key and being like “don’t delete production!” and then walking away.
The way the guy was prompting this agent also leaves a lot to be desired. It’s trained to work on emulating human thoughts, speech patterns. Turns out When giving instructions, it’s really difficult to figure out what to do from a list of things to not do. If the dude just instead told the agent what to do and how he wanted it to work and when it needed to bring things to his attention, instead of telling it to not guess, instead explaining that it needed to use whatever tools to go look up a documentation to understand the context and scope of the project it’s working on It does a better job.
Giving a model the right context to do something is the difference between a model doing something like deleting your production database or your model acting like a magical machine that can get anything done.