With surveillance capitalism reaching it’s peak in 2026/2027, is getting out the Web the only way for the privacy inclined user?

Will Gemini (or cousins) go the way of Linux? Or is it foverer condemned to go the way of Hurd

  • macniel@feddit.org
    link
    fedilink
    arrow-up
    1
    arrow-down
    1
    ·
    1 day ago

    Build search engines that only index pages that contain the smolweb headers defined in the standard

    and how do you propose to impose that? Heck robots.txt doesn’t even seem to be honored!

    • FineCoatMummy@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      3
      ·
      19 hours ago

      Heck robots.txt doesn’t even seem to be honored!

      Eh. I make web pages for friends and family. We don’t want them indexed. In my server logs, most of the big scrapers are pretty good about honoring robots.txt. Google does. Meta does. At least some the AI scrapers do. Pretty sure Bytedance does too but I’d have to double check.

      I’ll see a hit from let’s say G’s bot, fetching robots.txt, and then no other activity. For all the thigns I can shit on G and Meta for, this isn’t one of them.

      Is it universal to heed it? Nah, ofc not. But it’s still more effective than you’d think, given it is voluntary.