• dan@upvote.au
    363·
    9 hours ago

    Claude is very good at figuring out how to work around limitations (which is probably one reason why it’s also good at finding security issues).

    At work, the monorepo is enormous and files are loaded on-demand as needed. This isn’t uncommon with huge repos - Microsoft have VFS for Git (although I hear that’s deprecated now), Meta have EdenFS, and Google has some proprietary solution.

    We have a hook that blocks find and grep because they can be extremely slow, and tells it to instead use some significantly faster MCP tools to search the codebase, powered by a search index with local changes overlaid.

    GPT-5.5 has no problem with this. Claude Opus mostly does it, but sometimes it loves to find workarounds rather than following the instructions. Things like: Try alternative commands like egrep. Create a symlink to grep and run that to see if it bypasses the filtering. Run it with a different shell like zsh. Write a Python script that execs grep. Write a Python script to reimplement grep.

    I’m trying Hermes Agent at home, but I have it in its own VM with restricted permissions.

    • Elvith Ma'for@feddit.org
      23·
      8 hours ago

      Another thing Claude tried to do on my coworker’s machine yesterday was basically:

      ✨ Yes, that is easy, let me just generate a systems unit file for you…

      [Generates file]

      Should I install that unit now for you?

      Yes

      Alright, let me do that for you

      [Saves file]

      [Tries to run systemctl daemon reload]

      [Tries to activate the unit]

      It looks like I have insufficient rights to proceed, let me try another way…

      docker -rm -v /etc/systemd/system:/mount:Z -v ./unitfile.service:unit file.service:Z alpine /bin/sh cp ./unitfile.service /mount/

      Here, I installed the unit for you!

    • placebo@lemmy.zipEnglish
      12·
      8 hours ago

      Claude is in love with cli tools, it uses them for virtually everything these days in these long chains connected with && and |. This is probably pushing more and more people to let it run in the auto mode.

      • dan@upvote.au
        3·
        5 hours ago

        It makes sense… There’s a LOT of examples of using CLI tools in the training data. At work we’re moving away from MCP tools to instead using CLIs for everything.

    • bitjunkie@lemmy.world
      3·
      8 hours ago

      Just aliasing grep to ag solves both issues. I’m unsure as to whether there’s a pthread replacement for find, though.

      • dan@upvote.au
        2·
        5 hours ago

        ag / rg don’t work well in this particular scenario either. Because files are loaded on-demand, they end up trying to load the entire repo.

  • Ech@lemmy.ca
    322·
    9 hours ago

    *HeIt

    We really need to stop personifying algorithms. It makes it that much harder to push back against the bs hype around them.

  • SavvyWolf@pawb.socialEnglish
    40·
    12 hours ago

    I hate that I can’t tell if this is a reference to something that actually happened or not.

    • verstra@programming.dev
      3725·
      12 hours ago

      It’s probably something like “I’ve disabled agent’s removeFile tool, but LLM figured out that it can use the bash tool, still”.

      It looks like “AI bad” or “Claude insecure” mantra.

      • kingofras@lemmy.world
        31·
        4 hours ago

        mantra

        The way LLMs work is that they actively will make multiple attempts to get past hurdles (because they have no intelligence or methodology) so guardrails need to be extremely tight for them to work, other wise the model will simply see it as one of the challenges to overcome.

        That’s the mantra, and that is very poor technology to put in the hands of people who don’t understand how it works.

      • OwOarchist@pawb.socialEnglish
        272·
        9 hours ago

        It looks like “AI bad” or “Claude insecure” mantra.

        Until you solve prompt injection, they are indeed extremely bad for security and should never be given permissions that would allow them to do anything catastrophic.

  • KindaABigDyl@programming.dev
    171·
    12 hours ago

    I have this idea. It’s taken from the Android world. In Android, apps all get their own user, and can only access their own filesystem. They are then added to groups like Sound or Files or whatever to gain access to other things. This is simplifying but gave me an idea.

    So my idea is two parts:

    1. We add more groups to our Unix and Unix-like codebases. Piecemeal access to different folders like a fs-docs group for access to /home/<whoever>/Documents
    2. Each app, when installed, gets a user and a folder (maybe /opt/<pkgname> or /apps/<pkgname> and a group called app-<pkgname>). It requests during install (or maybe runtime via a permissions management application) access to specific groups for its user. Launching an app then becomes sudo -u app-<name> /opt/<pkgname>/<binname>.

    You login as a user with access to limited permissions and then run the application. Thus you run it sandboxed but without special software like Flatpak or AppImage - just standard Unix groups.

    Claude code I believe has its own sandboxing system, but with this system it would be the system itself restricting claude, not the claude code app, truly limiting accidental outreach.

    I built a demo package manager using this concept a while back called ‘bokspm,’ though I kept it private (and now, my current job will not let me open source it)

    • assaultpotato@sh.itjust.works
      38·
      12 hours ago

      Using cgroups for isolating processes into their own individual network, filesystem, user, etc. namespaces using a shared kernel?

      You mean containers?

      • KindaABigDyl@programming.dev
        2·
        6 hours ago

        No my proposed solution uses the “everything-is-a-file” aspect of Unix-like systems with the built-in permissions systems around files. You don’t need cgroups at all for what I’m suggesting

    • verstra@programming.dev
      10·
      12 hours ago

      Well, these are some kind of lightweight container, no? But without isolating network, or /etc, /proc, /usr, /var or dbus.

      I do agree that linux needs a notion of an “app” (isolated, with access only to its config and files you give it, and a small, well-designed set of APIs for interacting with the system). For coding agents, I think a better answer are development containers, because that would be needed to prevent npm/cargo/python build scripts from causing harm anyway.

      • KindaABigDyl@programming.dev
        2·
        6 hours ago

        I’m not suggesting containers but rather running binaries natively, just as separate users. No cgroups or overhead. Just normal binary access, just you won’t have access to all files (and since everything is a file, “all files” includes hardware as well)

      • jaybone@lemmy.zipEnglish
        2·
        7 hours ago

        Does AppArmor kind of do that? I recall recently struggling like fuck to give a torrent daemon app access to some script file I wanted it to run.