A year of fighting scrapers on my 1.5 million-page website. What I tried, how I failed, and what’s working now.

    • undefinedTruth@lemmy.zip
      link
      fedilink
      English
      arrow-up
      3
      ·
      edit-2
      2 days ago

      Why would a website owner use this over Anubis. Not bashing it or anything, just curious how it differentiates?

      • melroy@kbin.melroy.org
        link
        fedilink
        arrow-up
        2
        ·
        2 days ago

        I wanted to have high performance. See my benchmarks. I’m able to reach 150k-180k reqs/sec allowing me to handle ddos attacks as well. Even under very heavy load. Anubis will fall apart.

        Great question BTW!

    • BrikoX@lemmy.zipOPM
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      3 days ago

      I don’t run PatronView, just sharing the blog post from them.

      I tried to access angieguardian.org, but was met with “JavaScript is required to access this site” which basically guaranteed I will never visit it again.

      • melroy@kbin.melroy.org
        link
        fedilink
        arrow-up
        2
        ·
        3 days ago

        OK fine. That is due to the pow. The whole point of Angie Guardian. I also don’t like it. But the internet is fked up!