☆ Yσɠƚԋσʂ ☆

  • 12.8K Posts
  • 13.1K Comments
Joined 6 years ago
cake
Cake day: January 18th, 2020

help-circle



























  • Yes, these are absolutely things humans struggle to do. And finding more exploits faster is literally better.

    Again, you just keep ignoring what I write here and you clearly don’t understand how these tools are actually used. You’re not just having LLM come up with some hypothesis at random here. You use the tool to do the attack. I don’t know why this bit of information is so hard for you to process.

    Also, it should be obvious why it’s hard to find correlations in a large set of data than in a small one. Go think about why where’s waldo is hard for humans.

    Or not. Maybe for you it would be, but not for a trained researcher.

    Maybe you should stop trying to debate a topic you’re very clearly not qualified to have an opinion on. It doesn’t matter if there are intermediate steps which are necessary to make or not. The discussion is about exploits. Either you get unauthorized access or you don’t. Either you have a hole in your system or you don’t.

    And as I’ve repeatedly explained to you, and you studiously ignored, finding and exploiting these vulnerabilities is part of the same process. The LLM tests what it does against a live system, and it builds the exploit step by step.

    Also, here’s what Linus has to say on the subject since you’re just going to ignore anything I say. https://www.theregister.com/security/2026/05/18/linus-torvalds-says-ai-powered-bug-hunters-have-made-linux-security-mailing-list-almost-entirely-unmanageable/5241633



  • What I’m saying here is that the way you actually use LLMs is by having them go through the steps of the exploit. It makes a hypothesis and then it tries it, and then you see the result. There’s nothing to be fooled by here because the steps it takes either work or they don’t.

    The reason LLMs are much better at finding these vulnerabilities is because a human can’t keep a large codebase in their head all at once. If you look at a project like Lemmy for example, there’s a ton of code in it. You have to be an expert in what that code is doing, how the moving pieces relate to each other, and the domain itself to find the exploit. The LLM can zero in on the problems much easier, and actually take the steps to try the exploit. For example, for the case I mentioned with piefed, the issue was very subtle way the oauth token was being misused. It wasn’t localized in one place where auth was done, but manifested in a different part of the codebase that relied on it. Something like that would take a lot of dedicated work to find manually.




  • There’s a bigger problem here which is that models themselves are general commodities and there’s just not enough difference between them for any one player to differentiate themselves. A company can get ahead of others by a few months, but then the rest quickly close the gap. It’s a really low margin business because you constantly have to burn a ton of money just to stay a bit ahead, and you have diminishing returns the longer you do it.

    The only rational approach is to treat models as shared infrastructure akin to Linux because the money is going to be in customization niches. Companies will charge to tune models for specific use cases and charge support for that. There’s also going to be money at the bottom for hardware vendors making chips and memory. But the middle tier of generic LLMs is just relentless involution driving profits towards the bottom.







  • The whole trope that LLMs need absurd levels of energy use has not been true for a while now. People latched on to this idea because early models were hideously inefficient, as is the case with pretty much any new technology. Today, you can run local coding models on your laptop that surpass the capabilities of frontier models needing whole data centres to run just a year ago. You no longer need an inordinate amount of computing power to run any of this stuff, and performance gains haven’t stopped. There’s no indication that we’re close to any sort of a limit here.

    Also, nowhere did I say that a socialist world would have developed it in the same fashion. I’m merely pointing out that it would have been developed, and there would have been many existing use cases which I listed which have little to do with commercial incentives. I get the impression that you’re conflating hype with the actual legitimate use of which there are plenty already.

    Finally, there is really nothing stopping people from developing this technology in open source fashion. And that’s the way to decouple this tech from commercial incentives going forward. There are already open models to build on, and that should be leveraged to develop completely open alternatives which are community driven.