It only took nine seconds for an AI coding agent gone rogue to delete a company’s entire production database and its backups, according to its founder. PocketOS, which sells software that car rental businesses rely on, descended into chaos after its databases were wiped, the company’s founder Jeremy Crane said.

The culprit was Cursor, an AI agent powered by Anthropic’s Claude Opus 4.6 model, which is one of the AI industry’s flagship models. As more industries embrace AI in an attempt to automate tasks and even replace workers, the chaos at PocketOS is a reminder of what could go wrong.

Crane said customers of PocketOS’s car rental clients were left in a lurch when they arrived to pick up vehicles from businesses that no longer had access to software that managed reservations and vehicle assignments.

  • LukeZaz@beehaw.org
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 day ago

    I think this kind of rhetoric is best saved for when AI is not currently one of the most harmful things in society today. Argue it’s a hammer all you like; people aren’t going to be receptive when that hammer is currently being used to beat their faces in, and making that argument at such a time isn’t exactly sympathetic.

    • t3rmit3@beehaw.org
      link
      fedilink
      arrow-up
      2
      ·
      edit-2
      14 hours ago

      I think that “stop being mad the hammer exists, start being mad at the group of people who are beating your face in” is a very important message. Getting rid of AI (which isn’t even something we can do; you can’t put the genie back in the bottle with this) won’t fix the issue, they’ll just make another hammer. The hammer is both a weapon in this case, and a distraction.