GitHub - TecharoHQ/anubis: Weighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlers
github.com
Weighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlers - TecharoHQ/anubis

I just started using this myself, seems pretty great so far!

Clearly doesn’t stop all AI crawlers, but a significantly large chunk of them.

  • DrewEnglish
    4•3 days ago

    If your browser doesn’t have a Mozilla user agent (I.e. like chrome or Firefox) it will pass directly. Most AI crawlers use these user agents to pretend to be human users

    • @swelter_spark@reddthat.comEnglish
      1•2 days ago

      What I’m thinking about is more that in Linux, it’s common to access URLs directly from the terminal for various purposes, instead of using a browser.

      • DrewEnglish
        1•2 days ago

        If you’re talking about something like curl, that also uses its own User agent unless asked to impersonate some other UA. If not, then maybe I can’t help.