Pro@programming.dev to Technology@programming.devEnglish · edit-24 months agoCodeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.i.imgur.comimagemessage-square114linkfedilinkarrow-up1707file-textcross-posted to: [email protected]
arrow-up1707imageCodeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.i.imgur.comPro@programming.dev to Technology@programming.devEnglish · edit-24 months agomessage-square114linkfedilinkfile-textcross-posted to: [email protected]
minus-squarewitten@lemmy.worldlinkfedilinkEnglisharrow-up12·4 months agoBelieve me, these AI corporations have way too many IPs to make this feasible. I’ve tried per-IP rate limiting. It doesn’t work on these crawlers.
Believe me, these AI corporations have way too many IPs to make this feasible. I’ve tried per-IP rate limiting. It doesn’t work on these crawlers.