Lemmy newb here, not sure if this is right for this /c.

An article I found from someone who hosts their own website and micro-social network, and their experience with web-scraping robots who refuse to respect robots.txt, and how they deal with them.

  • El Barto@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    3 天前

    You had me until the “ethically sound position” part.

    You’re saying that Joe Blogger is acting unethically because he doesn’t allow VPN users to visit his site. C’mon, brother.