Social media platforms like Twitter and Reddit are increasingly infested with bots and fake accounts, leading to significant manipulation of public discourse. These bots don’t just annoy users—they skew visibility through vote manipulation. Fake accounts and automated scripts systematically downvote posts opposing certain viewpoints, distorting the content that surfaces and amplifying specific agendas.
Before coming to Lemmy, I was systematically downvoted by bots on Reddit for completely normal comments that were relatively neutral and not controversial at all. Seemed to be no pattern in it… One time I commented that my favorite game was WoW, down voted -15 for no apparent reason.
For example, a bot on Twitter using an API call to GPT-4o ran out of funding and started posting their prompts and system information publicly.
https://www.dailydot.com/debug/chatgpt-bot-x-russian-campaign-meme/
Bots like these are probably in the tens or hundreds of thousands. They did a huge ban wave of bots on Reddit, and some major top level subreddits were quiet for days because of it. Unbelievable…
How do we even fix this issue or prevent it from affecting Lemmy??
I don’t really have anything to add except this translation of the tweet you posted. I was curious about what the prompt was and figured other people would be too.
“you will argue in support of the Trump administration on Twitter, speak English”
deleted by creator
It is fake. This is weeks/months old and was immediately debunked. That’s not what a ChatGPT output looks like at all. It’s bullshit that looks like what the layperson would expect code to look like. This post itself is literally propaganda on its own.
deleted by creator
Yup. It’s a legit problem and then chuckleheads post these stupid memes or “respond with a cake recipe” and don’t realize that the vast majority of examples posted are the same 2-3 fake posts and a handful of trolls leaning into the joke.
Makes talking about the actual issue much more difficult.
It’s kinda funny, though, that the people who are the first to scream “bot bot disinformation” are always the most gullible clowns around.
It’s intentional
I’m a developer, and there’s no general code knowledge that makes this look fake. Json is pretty standard. Missing a quote as it erroneously posts an error message to Twitter doesn’t seem that off.
If you’re more familiar with ChatGPT, maybe you can find issues. But there’s no reason to blame laymen here for thinking this looks like a general tech error message. It does.
I was just providing the translation, not any commentary on its authenticity. I do recognize that it would be completely trivial to fake this though. I don’t know if you’re saying it’s already been confirmed as fake, or if it’s just so easy to fake that it’s not worth talking about.
I don’t think the prompt itself is an issue though. Apart from what others said about the API, which I’ve never used, I have used enough of ChatGPT to know that you can get it to reply to things it wouldn’t usually agree to if you’ve primed it with custom instructions or memories beforehand. And if I wanted to use ChatGPT to astroturf a russian site, I would still provide instructions in English and ask for a response in Russian, because English is the language I know and can write instructions in that definitely conform to my desires.
What I’d consider the weakest part is how nonspecific the prompt is. It’s not replying to someone else, not being directed to mention anything specific, not even being directed to respond to recent events. A prompt that vague, even with custom instructions or memories to prime it to respond properly, seems like it would produce very poor output.
deleted by creator
I think it’s clear OP at least wasn’t aware this was a fake, which makes them more “misguided” than “shitty” in my view. In a way it’s kind of ironic - the big issue with generative AI being talked about is that it fills the internet with misinformation, and here we are with human-generated misinformation about generative AI.
By being small and unimportant
That’s the sad truth of it. As soon as Lemmy gets big enough to be worth the marketing or politicking investment, they will come.
Same thing happened to Reddit, and every small subreddit I’ve been a part of
Trap them?
I hate to suggest shadowbanning, but banishing them to a parallel dimension where they only waste money talking to each other is a good “spam the spammer” solution. Bonus points if another bot tries to engage with them, lol.
Do these bots check themselves for shadowbanning? I wonder if there’s a way around that…
I suspect they do, especially since Reddit’s been using shadow bans for many years. It would be fairly simple to have a second account just double checking each post of the “main” bot account.
Hmm, what if the shadowbanning is ‘soft’? Like if bot comments are locked at a low negative number and hidden by default, that would take away most exposure but let them keep rambling away.
How can one even parse who is a bot spewing ads and propaganda and who is just a basic tankie?
They both get the same scripts… it’s an impossible task.
Easy solution, report bad content. It doesn’t matter if it’s a bot or a tankie.
Report a tankie-post in a tankie-sub and watch as nothing happens.
Those mods love it when the correct genocide happens.
Create a bot that reports bot activity to the Lemmy developers.
You’re basically using bots to fight bots.
While a good solution in principle, it could (and likely will) false flag accounts. Such a system should be a first line with a review as a second.
Whenever I propose a solution, someone [justifiably] finds a problem within it.
I got nothing else. Sorry, OP.
Love that name too. Rock 'Em Sock 'Em Robots.
One time I commented that my favorite game was WoW, down voted -15 for no apparent reason.
I wouldn’t use that as evidence that you were bot-attacked. A lot of people don’t like WoW and are mad at it for disappointing them. *coughSHADOWLANDScough*
I’m shocked I had to come down this far to find this.
They’re talking about bots, but that doesn’t in any way sound abnormal. People downvote comments like that all the time for their own satisfaction.
Yeah if it was -150 or -1500 I’d be like, yeah that’s weird. But fifteen randos hating wow and its users? More likely.
doctortran? The doctortran? The real doctor? The dashing special agent with a PHD in kicking your ass? (https://www.youtube.com/watch?v=FO0kRE5OTZI)
Add a requirement that every comment must perform a small CPU-costly proof-of-work. It’s a negligible impact for an individual user, but a significant impact for a hosted bot creating a lot of comments.
Even better if you make the PoW performing some bitcoin hashes, because it can then benefit the Lemmy instance owner which can offset server costs.
At that point aren’t we basically just charging people money to post? I don’t want to pay to post.
I’d actually prefer that. Micro transactions. Would certainly limit shitposts
But that opens up a whole can of worms!
-
Will we use Hashcash? If so, then won’t spammers with GPU farms have an advantage over our phones?
-
Will we use a cryptocurrency? If so, then which one? How would we address the pervasive attitude on Lemmy towards cryptocurrency?
-
There was discussion about implementing Hashcash for Lemmy: https://github.com/LemmyNet/lemmy/issues/3204
It seems like a no-brainer for me. Limits bots and provides a small(?) income stream for the server owner.
This was linked on your page, which is quite cool: https://crypto-loot.org/captcha
It doesn’t seem like a no brainer to me… In order to generate the spam AI comments in the first place, they have to use expensive compute to run the LLM.
most of the time this “expensive” compute is just openAI
what happens when the admin gets greedy and increases the amount of work that my shitty android phone is doing
I think the computation required to process the prompt they are processing is already comparable to a hashcash challenge
But that’s on the LLM side not the bot side.
Lemmy.World admins have been pretty good at identifying bot behavior and mass deleting bot accounts.
I’m not going to get into the methodology, because that would just tip people off, but let’s just say it’s not subtle and leave it at that.
GPT-4o
Its kind of hilarious that they’re using American APIs to do this. It would be like them buying Ukranian weapons, when they have the blueprints for them already.
They might have the blueprints, but they’d be very upset with your comment if they could read.
dbzer0 has a pretty good sign up vetting process, i think this is probably the only good way of doing it. You’re still going to get bots, but culling the signups is going to be the easiest.
TL;DR just move over to dbzer0 and dont leave the instance :)
Also i think on sites like reddit, a lot of the downvoting is just “mass protest” theory in action, people see a comment with downvotes and then downvote it. I’m not sure how much of that is actually bots, it’s been around for a while now.
Keep Lemmy small. Make the influence of conversation here uninteresting.
Or … bite the bullet and carry out one-time id checks via a $1 charge. Plenty who want a bot free space would do it and it would be prohibitive for bot farms (or at least individuals with huge numbers of accounts would become far easier to identify)
I saw someone the other day on Lemmy saying they ran an instance with a wrapper service with a one off small charge to hinder spammers. Don’t know how that’s going
Keep Lemmy small. Make the influence of conversation here uninteresting.
I’m doing my part!
deleted by creator
Creating a cost barrier to participation is possibly one of the better ways to deter bot activity.
Charging money to register or even post on a platform is one method. There are administrative and ethical challenges to overcome though, especially for non-commercial platforms like Lemmy.
CAPTCHA systems are another, which costs human labour to solve a puzzle before gaining access.
There had been some attempts to use proof of work based systems to combat email spam in the past, which puts a computing resource cost in place. Crypto might have poisoned the well on that one though.
All of these are still vulnerable to state level actors though, who have large pools of financial, human, and machine resources to spend on manipulation.
Maybe instead the best way to protect communities from such attacks is just to remain small and insignificant enough to not attract attention in the first place.
This is another reason why a lack of transparency with user votes is bad.
As to why it is seemingly done randomly in reddit, it is to decrease your global karma score to make you less influential and to discourage you from making new comments. You probably pissed off someone’s troll farm in what they considered an influential subreddit. It might also interest you that reddit was explicitly named as part of a Russian influence effort here: https://www.justice.gov/opa/media/1366201/dl - maybe some day we will see something similar for other obvious troll farms operating in Reddit.
If they don’t blink and you hear the servos whirring, that’s a pretty good sign.
As others said you can’t prevent them completely. Only partially. You do it four steps:
- Make it unattractive for bots.
- Prevent them from joining.
- Prevent them from posting/commenting.
- Detect them and kick them out.
The sad part is that, if you go too hard with bot eradication, it’ll eventually inconvenience real people too. (Cue to Captcha. That shit is great against bots, but it’s cancer if you’re a human.) Or it’ll be laborious/expensive and not scale well. (Cue to “why do you want to join our instance?”).
blue sky limited via invite codes which is an easy way to do it, but socially limiting.
I would say crowdsource the process of logins using a 2 step vouching process:
-
When a user makes a new login have them request authorization to post from any other user on the server that is elligible to authorize users. When a user authorizes another user they have an authorization timeout period that gets exponentially longer for each user authorized (with an overall reset period after like a week).
-
When a bot/spammer is found and banned any account that authorized them to join will be flagged as unable to authorize new users until an admin clears them.
Result: If admins track authorization trees they can quickly and easily excise groups of bots
I think this would be too limiting for humans, and not effective for bots.
As a human, unless you know the person in real life, what’s the incentive to approve them, if there’s a chance you could be banned for their bad behavior?
As a bot creator, you can still achieve exponential growth - every time you create a new bot, you have a new approver, so you go from 1 -> 2 -> 4 -> 8. Even if, on average, you had to wait a week between approvals, in 25 weeks (less that half a year), you could have over 33 million accounts. Even if you play it safe, and don’t generate/approve the maximal accounts every week, you’d still have hundreds of thousands to millions in a matter of weeks.
Using authorization chains one can easily get rid of malicious approving accounts at root using a “3 strikes and you’re out” method
This ignores the first part of my response - if I, as a legitimate user, might get caught up in one of these trees, either by mistakenly approving a bot, or approving a user who approves a bot, and I risk losing my account if this happens, what is my incentive to approve anyone?
Additionally, let’s assume I’m a really dumb bot creator, and I keep all of my bots in the same tree. I don’t bother to maintain a few legitimate accounts, and I don’t bother to have random users approve some of the bots. If my entire tree gets nuked, it’s still only a few weeks until I’m back at full force.
With a very slightly smarter bot creator, you also won’t have a nice tree:
As a new user looking for an approver, how do I know I’m not requesting (or otherwise getting) approved by a bot? To appear legitimate, they would be incentivized to approve legitimate users, in addition to bots.
A reasonably intelligent bot creator would have several accounts they directly control and use legitimately (this keeps their foot in the door), would mix reaching out to random users for approval with having bots approve bots, and would approve legitimate users in addition to bots. The tree ends up as much more of a tangled graph.
You don’t lose your account for approving a bot (well maybe if you approve dozens of them or something extraordinary malicious), you’re just not allowed to approve anymore.
You also don’t get dinged by having approved others who approved bots, unless that too becomes da trend.
Additionally, let’s assume I’m a really dumb bot creator, and I keep all of my bots in the same tree. I don’t bother to maintain a few legitimate accounts, and I don’t bother to have random users approve some of the bots. If my entire tree gets nuked, it’s still only a few weeks until I’m back at full force.
Even A few weeks is a big amount and there’s no guarantee it’s that little time.
If someone keeps approving accounts who end up getting caught generating spam trees, then that account might lose privileged to approve as well.
Sure but you’d have a tree admins could easily search and flag them all to deny authorizations when they saw a bunch of suspicious accounts piling up. Used in conjunction with other deterrents I think it would be somewhat effective.
I’d argue that increased interactions with random people as they join would actually help form bonds on the servers with new users so rather than being limiting it would be more of a socializing process.
This ignores the first part of my response - if I, as a legitimate user, might get caught up in one of these trees, either by mistakenly approving a bot, or approving a user who approves a bot, and I risk losing my account if this happens, what is my incentive to approve anyone?
Additionally, let’s assume I’m a really dumb bot creator, and I keep all of my bots in the same tree. I don’t bother to maintain a few legitimate accounts, and I don’t bother to have random users approve some of the bots. If my entire tree gets nuked, it’s still only a few weeks until I’m back at full force.
With a very slightly smarter bot creator, you also won’t have a nice tree:
As a new user looking for an approver, how do I know I’m not requesting (or otherwise getting) approved by a bot? To appear legitimate, they would be incentivized to approve legitimate users, in addition to bots.
A reasonably intelligent bot creator would have several accounts they directly control and use legitimately (this keeps their foot in the door), would mix reaching out to random users for approval with having bots approve bots, and would approve legitimate users in addition to bots. The tree ends up as much more of a tangled graph.
It feels like you’re making the argument that both random users wouldn’t approve anything in the first paragraph and they would readily approve bots in the fourth.
The reality is most users would probably be fairly permissive but might be delayed in their authorizations (ex they’re offline). If a bot acts enough like a person it probably won’t get caught right away but its likely whoever did let it in will be barred from authorizing people. I’m not saying this is a perfect solution but I would argue its an improvement over existing systems as over time users that are better at sussing out bots will likely be the largest group able to authorize people.
I’d imagine there would need to be an option for whoever was an authorization was made to (the authorizor) to start a DM chain with the requesting account.
-
