We cut middle managers across the organization because AI allows us to have more direct reports per manager while still measuring and mentoring our teams effectively. – Matthew Prince, How I Choose…
Are you asking me to reject my professional daily reality?!
You can provide sources all day, but it won’t change my reality of this being the most productivity enhancing tool since MS introduced intellisense in 1996.
If I wanted to shit on AI I could absolutely provide data to make it look like it sucks and laugh at it. It can do some really stupid shit.
In the hands of an expert, this technology is a productivity multiplier. In the hands of a beginner, this technology is a security and code quality problem. If you’re having problems controlling it, look inward.
Are you asking me to reject my professional daily reality?!
Nobody’s asking you to do anything. If it works for you, then that’s fine.
People are talking about the tech in general and their own experiences with it, alongside relevant research they have found. You are more than welcome to disagree with each other. Nobody is forced to change their opinions or how they work over a short internet conversation.
As an aside, LLMs, like everything else in life, require nuance to evaluate. They excel at specific tasks that are built for them, and are terrible at the wide array of tasks that are not built for them. It’s entirely possible that your work primarily lies in the former while others work in the latter space.
They excel at specific tasks that are built for them
They are however widely known to be terrible at code, at least compared to an advanced coder. They introduce not only more bugs even after human review, but new kinds of more insideous bugs.
I like to say the main problems with most projects were already the code quality and the bugs, and not that we somehow needed even more low quality lines of code.
(Disclaimer: not talking about passive AI bug analysis here, just using AI to write actual code.)
They are however widely known to be terrible at code
They are for large tasks. However, for simple pattern repetition tasks, they’re generally fine, code or not. I’ve had success, for example, having them remove pointless, confusing try…except blocks surrounding imports at work. I usually find that I just rewrite anything myself if it’s anything more complex than that because the code it produces makes no sense and taught me nothing.
I like to say the main problems with most projects were already the code quality and the bugs, and not that we somehow needed even more low quality lines of code.
I’ve had success, for example, having them remove pointless, confusing try…except blocks surrounding imports at work.
And you may have introduced some dangerous hidden bug that way, which you may not have doing it manually.
(I’m not saying that makes it not worth it, this is just what the studies are saying. I personally think it’s not worth it, but I realize there is some subjectivity here.)
And you may have introduced some dangerous hidden bug that way, which you may not have doing it manually.
You act like I can’t read some import statements and see if they match the import statements on the other side of the diff lol.
There was no bug introduced. All the dependencies were required. If any of the imports did error, then that’s a bug with that package that got surfaced instead.
Yeah absolutely agree. In another thread I pointed out the difference between a pro using it and a novice using it.
Currently the loudest people seem to be the novices using it, even journalists? Maybe it’s just hatred and determination of people to make it sound bad to fulfill their fantasy of it sucking. Theres definitely an echo chamber effect going around also, a hivemind of “ai sucks”.
Anyhow, I like to add my experience with AI to discussions to counter all the negativity.
Quoting studies to actually back up one’s point is in my opinion far less of an echo chamber and a fantasy than anecdotes of “but for me it feels faster”. Especially when AI is known to slow people down while making them feel faster.
Yes, that must be wonderful to live in academic world and to throw around papers and shit but down here on the ground a lot of us actually get shit done and we aren’t the delusional ones that are pretending like somethings making us faster because we don’t have the money to waste if it’s not actually making us faster.
Are you an engineer? I’m trying to get a feel for what type of people don’t think that LLMs are useful for software engine
FYI in many countries the term “engineer” is protected. Software devs would not be allowed to call themselves engineers without some kind of certification.
All that aside, I think you’ll find that a majority of people on this instance write code regularly, whether as a hobby or day job. Also, at my software dev day job, we actually regularly discuss the academic research around LLMs primarily because it has a major impact on our work.
Personally speaking, what I’ve seen over the past few years is that it creates pretty demos really quickly that fall apart the minute you need to actually develop for real. The code becomes an unmaintainable amalgamation of random libraries used to do the same thing multiple ways, and my coworkers who rely on it heavily have learned basically nothing about the libraries or tools they use because they ask the LLM to do it all for them. This is also ignoring the complete lack of motivation I have now for PR reviews knowing that the same mistakes will be made again and again in the future because teaching a coworker a better way to do something does nothing to improve the output of a LLM, which cannot learn.
That’s not to say you can’t use it effectively. There just needs to be a balance between what you do as a developer vs what you have the LLM churn out quickly for you. It requires a lot of direction, enough so that I find it to be a waste of time as opposed to implementing things myself usually. Plus, I actually learn more doing it all myself, like upcoming library versions, changes in the tools and libraries I use since last using them, new language features, and so on.
While I’m not going to do a code review of your linked projects (nor do I believe that would be very useful), it sounds to me like you’ve found a way to make it work for you. That’s awesome. I, unfortunately, am regularly subjected to the slop emitted by it when in the hands of people who are actively destroying what experience they might have once had in favor of doing less work.
Yeah I’m familiar with some places protecting that word.
I find all the workforce productivity related academic papers in the space right now to be sensationalist and subjective. We just haven’t had enough time to let the dust fall.
Totally understand what you’re struggling with. Ppl still need to care about and understand what they’re writing and make sure things are done properly. You don’t oneshot everything.
Also, it depends on what types of systems you’re working on. Integration and glue code in backend systems is where I live most of the time. Using ai removes a lot of tedious boilerplate.
Are you asking me to reject my professional daily reality?!
Can you point me to a single field study that shows programmers become faster and not just feel faster, and that doesn’t come with some caveat like they haven’t tested AI coders vs non-AI coders, or coders without significant AI exposure before (since otherwise it won’t rule out simply becoming dependent)?
Even if you could find one, and I was unable to so far, it doesn’t change that:
you are probably faster by verbatim plagiarizing somebody’s other project at a large scale, and
by encouraging the environmental destruction brought on in particular by the training of new models.
Two caveats:
Keep in mind more lines of code is not a useful metric for faster project completion and faster maintenance task completion, especially for code bases that are already large.
I’m merely speaking about using LLM code in your project, so for example LLM auto completion or copy&pasting code from a chatbot. I’m mot talking about LLM code reviews that point out issues in natural language.
No, I don’t study or review research on this subject at the moment. My personal experience is far more reliable.
Look, I’m 50 years old. Been doing this shit forever. It’s an amazing productivity enhancer for ME. I can’t say any more really. I linked unique repos that were built by me in minutes as examples.
I understand your position and your doubt since it’s pretty common opinion in the echo chambers around here. Are you a software engineer?
LLMs seem to be inherently dumb: https://machinelearning.apple.com/research/illusion-of-thinking
And from what I can find in recent studies, no, they didn’t suddenly get smart. They just plagiarize slightly better: https://www.sciencedirect.com/science/article/pii/S2949719123000213#b7
Are you asking me to reject my professional daily reality?!
You can provide sources all day, but it won’t change my reality of this being the most productivity enhancing tool since MS introduced intellisense in 1996.
If I wanted to shit on AI I could absolutely provide data to make it look like it sucks and laugh at it. It can do some really stupid shit.
In the hands of an expert, this technology is a productivity multiplier. In the hands of a beginner, this technology is a security and code quality problem. If you’re having problems controlling it, look inward.
Nobody’s asking you to do anything. If it works for you, then that’s fine.
People are talking about the tech in general and their own experiences with it, alongside relevant research they have found. You are more than welcome to disagree with each other. Nobody is forced to change their opinions or how they work over a short internet conversation.
As an aside, LLMs, like everything else in life, require nuance to evaluate. They excel at specific tasks that are built for them, and are terrible at the wide array of tasks that are not built for them. It’s entirely possible that your work primarily lies in the former while others work in the latter space.
They are however widely known to be terrible at code, at least compared to an advanced coder. They introduce not only more bugs even after human review, but new kinds of more insideous bugs.
I like to say the main problems with most projects were already the code quality and the bugs, and not that we somehow needed even more low quality lines of code.
(Disclaimer: not talking about passive AI bug analysis here, just using AI to write actual code.)
They are for large tasks. However, for simple pattern repetition tasks, they’re generally fine, code or not. I’ve had success, for example, having them remove pointless, confusing try…except blocks surrounding imports at work. I usually find that I just rewrite anything myself if it’s anything more complex than that because the code it produces makes no sense and taught me nothing.
Tell me about it lol.
And you may have introduced some dangerous hidden bug that way, which you may not have doing it manually.
(I’m not saying that makes it not worth it, this is just what the studies are saying. I personally think it’s not worth it, but I realize there is some subjectivity here.)
You act like I can’t read some import statements and see if they match the import statements on the other side of the diff lol.
There was no bug introduced. All the dependencies were required. If any of the imports did error, then that’s a bug with that package that got surfaced instead.
Yeah absolutely agree. In another thread I pointed out the difference between a pro using it and a novice using it.
Currently the loudest people seem to be the novices using it, even journalists? Maybe it’s just hatred and determination of people to make it sound bad to fulfill their fantasy of it sucking. Theres definitely an echo chamber effect going around also, a hivemind of “ai sucks”.
Anyhow, I like to add my experience with AI to discussions to counter all the negativity.
Quoting studies to actually back up one’s point is in my opinion far less of an echo chamber and a fantasy than anecdotes of “but for me it feels faster”. Especially when AI is known to slow people down while making them feel faster.
Yes, that must be wonderful to live in academic world and to throw around papers and shit but down here on the ground a lot of us actually get shit done and we aren’t the delusional ones that are pretending like somethings making us faster because we don’t have the money to waste if it’s not actually making us faster.
Are you an engineer? I’m trying to get a feel for what type of people don’t think that LLMs are useful for software engine
FYI in many countries the term “engineer” is protected. Software devs would not be allowed to call themselves engineers without some kind of certification.
All that aside, I think you’ll find that a majority of people on this instance write code regularly, whether as a hobby or day job. Also, at my software dev day job, we actually regularly discuss the academic research around LLMs primarily because it has a major impact on our work.
Personally speaking, what I’ve seen over the past few years is that it creates pretty demos really quickly that fall apart the minute you need to actually develop for real. The code becomes an unmaintainable amalgamation of random libraries used to do the same thing multiple ways, and my coworkers who rely on it heavily have learned basically nothing about the libraries or tools they use because they ask the LLM to do it all for them. This is also ignoring the complete lack of motivation I have now for PR reviews knowing that the same mistakes will be made again and again in the future because teaching a coworker a better way to do something does nothing to improve the output of a LLM, which cannot learn.
That’s not to say you can’t use it effectively. There just needs to be a balance between what you do as a developer vs what you have the LLM churn out quickly for you. It requires a lot of direction, enough so that I find it to be a waste of time as opposed to implementing things myself usually. Plus, I actually learn more doing it all myself, like upcoming library versions, changes in the tools and libraries I use since last using them, new language features, and so on.
While I’m not going to do a code review of your linked projects (nor do I believe that would be very useful), it sounds to me like you’ve found a way to make it work for you. That’s awesome. I, unfortunately, am regularly subjected to the slop emitted by it when in the hands of people who are actively destroying what experience they might have once had in favor of doing less work.
Yeah I’m familiar with some places protecting that word.
I find all the workforce productivity related academic papers in the space right now to be sensationalist and subjective. We just haven’t had enough time to let the dust fall.
Totally understand what you’re struggling with. Ppl still need to care about and understand what they’re writing and make sure things are done properly. You don’t oneshot everything.
Also, it depends on what types of systems you’re working on. Integration and glue code in backend systems is where I live most of the time. Using ai removes a lot of tedious boilerplate.
Can you point me to a single field study that shows programmers become faster and not just feel faster, and that doesn’t come with some caveat like they haven’t tested AI coders vs non-AI coders, or coders without significant AI exposure before (since otherwise it won’t rule out simply becoming dependent)?
Even if you could find one, and I was unable to so far, it doesn’t change that:
you are probably faster by verbatim plagiarizing somebody’s other project at a large scale, and
by making yourself addicted and reliant on the AI where your own skill is eroding: https://www.404media.co/software-developers-say-ai-is-rotting-their-brains/ (if you get a paywall: https://archive.is/tHq80 ) and
by having a higher rate of bugs in your code no matter how carefully you review it https://www.coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report which especially for security sensitive projects may have dire long term consequences, and
by encouraging the environmental destruction brought on in particular by the training of new models.
Two caveats:
Keep in mind more lines of code is not a useful metric for faster project completion and faster maintenance task completion, especially for code bases that are already large.
I’m merely speaking about using LLM code in your project, so for example LLM auto completion or copy&pasting code from a chatbot. I’m mot talking about LLM code reviews that point out issues in natural language.
No, I don’t study or review research on this subject at the moment. My personal experience is far more reliable.
Look, I’m 50 years old. Been doing this shit forever. It’s an amazing productivity enhancer for ME. I can’t say any more really. I linked unique repos that were built by me in minutes as examples.
I understand your position and your doubt since it’s pretty common opinion in the echo chambers around here. Are you a software engineer?