lemmy.yachts
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
GnuLinuxDude@lemmy.ml
cake
to lemmy.ml meta@lemmy.ml · 2 年前

Should lemmy.ml block chatgpt scraping in robots.txt?

message-square
message-square
8
fedilink
24
message-square

Should lemmy.ml block chatgpt scraping in robots.txt?

GnuLinuxDude@lemmy.ml
cake
to lemmy.ml meta@lemmy.ml · 2 年前
message-square
8
fedilink

Some context about this here: https://arstechnica.com/information-technology/2023/08/openai-details-how-to-keep-chatgpt-from-gobbling-up-website-data/

the robots.txt would be updated with this entry

User-agent: GPTBot
Disallow: /

Obviously this is meaningless against non-openai scrapers or anyone who just doesn’t give a shit.

  • totallynotarobot@lemmy.world
    link
    fedilink
    arrow-up
    2
    arrow-down
    1
    ·
    2 年前

    If they’ll pay us when they scrape our content, sure.

    • Geist_@lemmy.world
      link
      fedilink
      arrow-up
      1
      ·
      2 年前

      … Is that like a non-argument? How do you suppose they would pay sites, let alone site users to scrape their content?

      • totallynotarobot@lemmy.world
        link
        fedilink
        arrow-up
        1
        ·
        2 年前

        Yes that’s the point

lemmy.ml meta@lemmy.ml

meta@lemmy.ml

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: [email protected]

Anything about the lemmy.ml instance and its moderation.

For discussion about the Lemmy software project, go to [email protected].

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 7 users / day
  • 7 users / week
  • 9 users / month
  • 288 users / 6 months
  • 1 local subscriber
  • 1.51K subscribers
  • 87 Posts
  • 181 Comments
  • Modlog
  • mods:
  • Nutomic@lemmy.ml
  • BE: 0.19.7
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org