• breadsmasher@lemmy.world
    link
    fedilink
    English
    arrow-up
    7
    ·
    11 months ago

    It isn’t an enforceable solution. robots.txt and similar are just please bots dont index these pages. Doesn’t mean any bots will respect it