Lee Duna@lemmy.nz to Reddit@lemmy.worldEnglish · 2 days agoReddit sues Perplexity for allegedly ripping its content to feed AIwww.theverge.comexternal-linkmessage-square17fedilinkarrow-up195arrow-down10cross-posted to: [email protected]
arrow-up195arrow-down1external-linkReddit sues Perplexity for allegedly ripping its content to feed AIwww.theverge.comLee Duna@lemmy.nz to Reddit@lemmy.worldEnglish · 2 days agomessage-square17fedilinkcross-posted to: [email protected]
minus-squareDamage@feddit.itlinkfedilinkarrow-up13·2 days agoSince multiple Lemmy instances show the same, federated, content, I wonder if our posts will have more weight in the model, for a normal scraper it would be as if many people repeated the same thing over and over on different sites.
minus-squareTangent5280@lemmy.worldlinkfedilinkarrow-up3·2 days agoIt would be trivial to remove duplicates. With a little bit of foresight they could just as easily avoid the duplicates in the first place.
minus-squareDamage@feddit.itlinkfedilinkarrow-up8·2 days agoThey could also avoid to re-crawl the whole internet every day, but here we are, so who knows.
Since multiple Lemmy instances show the same, federated, content, I wonder if our posts will have more weight in the model, for a normal scraper it would be as if many people repeated the same thing over and over on different sites.
It would be trivial to remove duplicates. With a little bit of foresight they could just as easily avoid the duplicates in the first place.
They could also avoid to re-crawl the whole internet every day, but here we are, so who knows.