Andy Reid@lemmy.world to Technology@lemmy.worldEnglish · 1 year agoAI companies are violating a basic social contract of the web and and ignoring robots.txtwww.theverge.comexternal-linkmessage-square67fedilinkarrow-up1782arrow-down112
arrow-up1770arrow-down1external-linkAI companies are violating a basic social contract of the web and and ignoring robots.txtwww.theverge.comAndy Reid@lemmy.world to Technology@lemmy.worldEnglish · 1 year agomessage-square67fedilink
minus-squarertxn@lemmy.worldlinkfedilinkEnglisharrow-up58·edit-21 year agoI would be shocked if any big corpo actually gave a shit about it, AI or no AI. if exists("/robots.txt"): no it fucking doesn't
minus-squareBargsimBoyz@lemmy.worldlinkfedilinkEnglisharrow-up1·1 year agoYeah I always found it surprising that everyone just agreed to follow a text file on a website on how to act. It’s one of the worst thought out/significant issues with browsing still out there from the beginning pretty much.
I would be shocked if any big corpo actually gave a shit about it, AI or no AI.
if exists("/robots.txt"): no it fucking doesn'tYeah I always found it surprising that everyone just agreed to follow a text file on a website on how to act. It’s one of the worst thought out/significant issues with browsing still out there from the beginning pretty much.