Admin on the slrpnk.net Lemmy instance.

He/Him or what ever you feel like.

XMPP: povoq@slrpnk.net

Lemmy alt: @kris@feddit.org

Avatar is an image of a baby octopus.

  • 44 Posts
  • 975 Comments
Joined 4 years ago
cake
Cake day: September 19th, 2022

help-circle










  • This is the botPolicy.yaml that we use on slrpnk.net :

    bots:
      - name: known-crawler
        action: CHALLENGE
        expression:
          # https://anubis.techaro.lol/docs/admin/configuration/expressions
          all:
            # Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36
            - userAgent.contains("Macintosh; Intel Mac") && userAgent.contains("Chrome/125.0.0.0") # very old chrome?
            - missingHeader(headers, "Sec-Ch-Ua") # a valid chrome has this header
        challenge:
          difficulty: 6
          algorithm: slow
    
        # Assert behaviour that only genuine browsers display.
        # This ensures that Chrome or Firefox versions
      - name: realistic-browser-catchall
        expression:
          all:
            - '"User-Agent" in headers'
            - '( userAgent.contains("Firefox") ) || ( userAgent.contains("Chrome") ) || ( userAgent.contains("Safari") )'
            - '"Accept" in headers'
            - '"Sec-Fetch-Dest" in headers'
            - '"Sec-Fetch-Mode" in headers'
            - '"Sec-Fetch-Site" in headers'
            - '"Accept-Encoding" in headers'
            - '( headers["Accept-Encoding"].contains("zstd") || headers["Accept-Encoding"].contains("br") )'
            - '"Accept-Language" in headers'
        action: CHALLENGE
        challenge:
          difficulty: 2
          algorithm: fast
    
      - name: generic-browser
        user_agent_regex: (?i:mozilla|opera)
        action: CHALLENGE
        challenge:
          difficulty: 4
          algorithm: fast
    
    status_codes:
      CHALLENGE: 202
      DENY: 406
    
    dnsbl: false
    
    #store:
    #  backend: valkey
    #  parameters:
    #    url: redis://valkey-primary:6379/0
    

    I think I just took it over from Codeberg.org back from when they still used Anubis. Nothing really relevant to Lemmy specifically and it is only in front of the frontends, not the s2s federation API.

    It seems though like there are some crawlers that use 3rd party hosted alternative frontends to crawl (unintentionally?) through the federation API, so something in front of that would be useful I guess.









  • poVoq@slrpnk.nettoSelfhosted@lemmy.worldPower efficiency
    link
    fedilink
    English
    arrow-up
    19
    ·
    edit-2
    14 days ago

    Sadly this. I have a graveyard of nice server boards that I got cheap before realizing how power hungry they are.

    For CPUs basically anything older than gen 6 intel is too power hungry (although be careful with Xeon and xeon derived cpus, that are sometimes older gens rebadged as gen 6).