This is a garbage article, I read the whole thing and they give no arguments except that it’s unlikely because they think without embodiment it’s not possible. They don’t even give a coherent argument for why it’s not possible without embodiment and claim cortisol is necessary to feel stress for example but neurotransmitters aren’t actually necessary, one can easily imagine a purely electrical brain that functions in the same way neurologically. This is just an evolutionary artifact.
i’m not saying llm’s are conscious, they almost definitely are not, but this article is garbage and won’t convince anyone of anything because it doesn’t actually engage with any counter arguments. The genghis khan section is a particularly egregious strawman
“Now let’s replace the prompt to read “The following is a conversation between a helpful AI chatbot and a user.” The LLM will produce a coherent dialogue just as it did before; the user character might ask for recipe suggestions or sightseeing recommendations, and the helpful AI-chatbot character will provide responses. Has anything fundamentally changed between the first example and the second? Did changing the names of the characters from historical figures to generic roles cause the LLM to conjure up conscious entities who possess subjective experience? Of course not. Both the user and the helpful AI chatbot are fictional characters.”
yeah of course it doesn’t, wtaf is this argument, nobody thinks the llm pretending to be julius ceasar is conjuring him and not doing the same thing a writer would do, same for pretending to be a helpful assistant, the point is the neural net generating the text, obviously not what it is pretending to be.
His argument is that generating statistically plausible text should not be treated as proof of consciousness. The reason why embodiment tends to be brought up is because it creates a basis for a system to have self awareness. You end up with a feedback loop where the system has to model the world and itself within it, and taking actions feeds back into the system so it has to be able to recognize itself as it interacts with its environment. Ted Chiang wrote a great novella where he discusses this idea https://en.wikipedia.org/wiki/The_Lifecycle_of_Software_Objects
Of course, it is possible that you could have some other type of feedback loop that produces self awareness and consciousness, but from what we understand of how LLMs work, it seems highly unlikely that statistical token generation is sufficient for that.
I do agree that he fails to really make the argument for why a disembodies intelligence could not be conscious. In my opinion, the strongest part of the article is at the end where he shows how the whole constitution kabuki theatre that Anthropic came up with clearly wouldn’t afford any protections to an entity that was conscious, so they don’t really believe what they’re saying.
This is a garbage article, I read the whole thing and they give no arguments except that it’s unlikely because they think without embodiment it’s not possible. They don’t even give a coherent argument for why it’s not possible without embodiment and claim cortisol is necessary to feel stress for example but neurotransmitters aren’t actually necessary, one can easily imagine a purely electrical brain that functions in the same way neurologically. This is just an evolutionary artifact.
i’m not saying llm’s are conscious, they almost definitely are not, but this article is garbage and won’t convince anyone of anything because it doesn’t actually engage with any counter arguments. The genghis khan section is a particularly egregious strawman
“Now let’s replace the prompt to read “The following is a conversation between a helpful AI chatbot and a user.” The LLM will produce a coherent dialogue just as it did before; the user character might ask for recipe suggestions or sightseeing recommendations, and the helpful AI-chatbot character will provide responses. Has anything fundamentally changed between the first example and the second? Did changing the names of the characters from historical figures to generic roles cause the LLM to conjure up conscious entities who possess subjective experience? Of course not. Both the user and the helpful AI chatbot are fictional characters.”
yeah of course it doesn’t, wtaf is this argument, nobody thinks the llm pretending to be julius ceasar is conjuring him and not doing the same thing a writer would do, same for pretending to be a helpful assistant, the point is the neural net generating the text, obviously not what it is pretending to be.
His argument is that generating statistically plausible text should not be treated as proof of consciousness. The reason why embodiment tends to be brought up is because it creates a basis for a system to have self awareness. You end up with a feedback loop where the system has to model the world and itself within it, and taking actions feeds back into the system so it has to be able to recognize itself as it interacts with its environment. Ted Chiang wrote a great novella where he discusses this idea https://en.wikipedia.org/wiki/The_Lifecycle_of_Software_Objects
Of course, it is possible that you could have some other type of feedback loop that produces self awareness and consciousness, but from what we understand of how LLMs work, it seems highly unlikely that statistical token generation is sufficient for that.
I do agree that he fails to really make the argument for why a disembodies intelligence could not be conscious. In my opinion, the strongest part of the article is at the end where he shows how the whole constitution kabuki theatre that Anthropic came up with clearly wouldn’t afford any protections to an entity that was conscious, so they don’t really believe what they’re saying.