GitHub - TecharoHQ/anubis: Weighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlers
github.com
Weighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlers - TecharoHQ/anubis

I just started using this myself, seems pretty great so far!

Clearly doesn’t stop all AI crawlers, but a significantly large chunk of them.

    • DrewEnglish
      4•3 days ago

      If your browser doesn’t have a Mozilla user agent (I.e. like chrome or Firefox) it will pass directly. Most AI crawlers use these user agents to pretend to be human users

      • @swelter_spark@reddthat.comEnglish
        1•2 days ago

        What I’m thinking about is more that in Linux, it’s common to access URLs directly from the terminal for various purposes, instead of using a browser.

        • DrewEnglish
          1•2 days ago

          If you’re talking about something like curl, that also uses its own User agent unless asked to impersonate some other UA. If not, then maybe I can’t help.