Seems like he’s been pushed into using LLMs as a way to cope with the deluge of LLM-generated security reports.

  • Tiresia@slrpnk.net
    link
    fedilink
    arrow-up
    28
    arrow-down
    5
    ·
    2 days ago

    Most LLM implementations to have come out in the past year have had introspection - a section of text where they’re prompted to think1 about the problem at a meta level which isn’t shown to the users. LLM engineers are actively working on expanding this into a more persistent, consistent, and functional world model - a bunch of text statements that other parts of the implementation are trained to treat1 as probably factually true, which it is regularly prompted to curate1 based on its interpretation1 of user input and other data.

    For example, an LLM might have a world model statement that says “As an LLM I may be running at different times. Before stating the current time with confidence, check the current time with an external source such as the UTC API.” so an introspection scratchpad it generates might be “To answer that question accurately I need to know the time. I will refer to the UTC API. Ah, it returned 12:17 on June 3rd 2026. Since Britain is currently at UTC+1 I can confidently say the sun is up in Britain”, and then the text the user sees is “Thank you for asking, the sun is currently up in Britain”.

    As for the lack of thought behind LLM backlash, that’s a factor of human psychology. In order to free up limited mental capacity, the human brain automatically simplifies rules it has learned consciously, imperfectly archiving the conscious method of learning it to long-term memory. People made up their minds about LLMs, and now the reasons are archived and no longer necessary for people’s response to LLMs. So now when people see LLMs, they don’t use the thought, they can just do the behavior they decided on and move on with their life.

    Re-litigating LLMs feels like going to an old archive and digging through dusty tomes. It can absolutely be worth it, but it’s an effort you’re not going to put in just because you see someone using it or praising it.

    Personally, my opposition to non-local LLMs is enshittification. Every habit you let become dependent on LLMs will be used to exploit you. Your habits before LLMs will be archived and too much effort to relearn, so you’ll pay out your ass for a worse service than what you used to be able to do yourself. My opposition to all LLMs is veganism, but that’s a story for a different comment.


    1: LLM instruction text anthropomorphises LLMs. LLMs don’t do these cognitive tasks the same way a human would.

    • prole@lemmy.blahaj.zone
      link
      fedilink
      arrow-up
      1
      arrow-down
      2
      ·
      1 day ago

      As for the lack of thought behind LLM backlash, that’s a factor of human psychology. In order to free up limited mental capacity, the human brain automatically simplifies rules it has learned consciously, imperfectly archiving the conscious method of learning it to long-term memory. People made up their minds about LLMs, and now the reasons are archived and no longer necessary for people’s response to LLMs. So now when people see LLMs, they don’t use the thought, they can just do the behavior they decided on and move on with their life.

      What an absolute crock of shit. Did your local LLM model tell you this?

      How convenient it must be for you to be able to disregard all criticism as simply “a lack of thought,” and that people have already made up their minds and don’t ever think about it again.

      Honestly, it’s fucking insulting. Just because you’ve given up basic critical thinking doesn’t mean everyone else has.

      • Tiresia@slrpnk.net
        link
        fedilink
        arrow-up
        1
        arrow-down
        1
        ·
        1 day ago

        Cached responses are healthy and natural, and they aren’t any more likely to be wrong than well-reasoned arguments.

        Like, suppose you were planning a dinner party with friends. One of them goes “person X? Ugh, can we not invite them?”, to which another friend goes into a long argument about why person X is totally better now. The first friend can’t really articulate what’s wrong, they just have a shitty feeling, and the arguments they use to explain that feeling are weak and full of holes. Which friend’s judgment would you be more inclined to trust?

        I for one would trust the friend with the feeling rather than the friend with the clearly reasoned argument 9 times out of 10.

        And so it is with LLMs/genAI. The reflexive repulsion towards them and towards people supporting them is well-earned. It’s healthy for people to set boundaries conservatively when so many genAI proponents are trying to weasel their way into acceptability with bad-faith comparisons, deliberate violation of “no genAI” boundaries as a form of gotcha, and a systematic lack of integration of critiques of genAI when trying to find new implementations. The Luddites were right, and so are anti-AI movements.

        All of which is to say, I think you got a false positive here, but you’ll get 'em next time.

        (You are correct that people do keep thinking about stuff and adding the impressions of those thoughts to the cached response. The analogy of a archive isn’t quite correct, it’s more like a giant pile of complaints that keeps getting added on to, that you need to shovel through to get back to that good point you heard 524 days ago, if it hasn’t disintegrated into abstract impression. I don’t think this changes the fundamental point, though, that usually the best reasons people have for believing something are not stored in their brain in a legible, rational format, and so the best actions - such as opposition to LLMs - are driven by emotional impression rather than thought).