Hi all!
As many of you have noticed, many Lemmy.World communities introduced a bot: @[email protected]. This bot was introduced because modding can be pretty tough work at times and we are all just volunteers with regular lives. It has been helpful and we would like to keep it around in one form or another.
The [email protected] mods want to give the community a chance to voice their thoughts on some potential changes to the MBFC bot. We have heard concerns that tend to fall into a few buckets. The most common concern we’ve heard is that the bot’s comment is too long. To address this, we’ve implemented a spoiler tag so that users need to click to see more information. We’ve also cut wording about donations that people argued made the bot feel like an ad.
Another common concern people have is with MBFC’s definition of “left” and “right,” which tend to be influenced by the American Overton window. Similarly, some have expressed that they feel MBFC’s process of rating reliability and credibility is opaque and/or subjective. To address this, we have discussed creating our own open source system of scoring news sources. We would essentially start with third-party ratings, including MBFC, and create an aggregate rating. We could also open a path for users to vote, so that any rating would reflect our instance’s opinions of a source. We would love to hear your thoughts on this, as well as suggestions for sources that rate news outlets’ bias, reliability, and/or credibility. Feel free to use this thread to share other constructive criticism about the bot too.
One problem I’ve noticed is that the bot doesn’t differentiate between news articles and opinion pieces. One of the most egregious examples is the NYT. Opinion pieces aren’t held to the same journalistic standards as news articles and shouldn’t be judged for bias and accuracy in the same way as news content.
I believe most major news organizations include the word “Opinion” in titles and URLs, so perhaps that could be something keyed off of to have the bot label these appropriately. I don’t expect you to judge the bias and accuracy of each opinion writer, but simply labeling them as “Opinion pieces are not required to meet accepted journalistic standards and bias is expected.” would go a long way.
Thanks for this. As a mod of /c/news, I hadn’t really thought about that. We don’t allow opinion pieces, but this is very relevant if we roll out a new bot for all the communities that currently use the MBFC bot.
No problem. Specifically came to my attention about a week ago on this post where the bot reported on an opinion piece as if it was straight news.
BTW, I actually do appreciate the bot and think it’s doing about as well as it can given the technical limitations of the platform.
This contributes significantly to the noise issue most people complain about
Interesting that people say that opinion pieces should not be held to the same standard. I personally see such pieces contribute to fake news going around. Shouldn’t a platform with reach, held accountable for wrong information, they hide behind an opinion piece?
It’s not a question of “should” - an opinion piece is rhetoric, not reporting. You can fact check some of it sometimes but functionally can’t hold it to the same standards as a regular news article. I agree that this can sometimes lead to “alternative facts” and disingenuous arguments, but the only other option is to forbid the publication of them which is obviously an infringement of first amendment rights. It’s messy, and it can lead to people being misinformed, but it’s what we’re stuck with.
This. Otherwise op-eds get a free pass to launder opinions the paper wants to publish, but can’t.
My personal view is to remove the bot. I don’t think we should be promoting one organisations particular views as an authority. My suggestion would be to replace it with a pinned post linking to useful resources for critical thinking and analysing news. Teaching to fish vs giving a fish kind of thing.
If we are determined to have a bot like this as a community then I would strongly suggest at the very least removing the bias rating. The factuality is based on an objective measure of failed fact checks which you can click through to see. Although this still has problems, sometimes corrections or retractions by the publisher are taken note of and sometimes not, leaving the reader with potentially a false impression of the reliability of the source.
For the bias rating, however, it is completely subjective and sometimes the claimed reasons for the rating actually contradict themselves or other 3rd party analysis. I made a thread on this in the support community but TLDR, see if you can tell the specific reason for the BBC’s bias rating of left-centre. I personally can’t. Is it because they posted a negative sounding headline about Trump once or is it biased story selection? What does biased story selection mean and how is it measured? This is troubling because in my view it casts doubt on the reliability of the whole system.
I can’t see how this can help advance the goal (and it is a good goal) of being aware of source bias when in effect, we are simply adding another bias to contend with. I suspect it’s actually an intractable problem which is why I suggest linking to educational resources instead. In my home country critical analysis of news is a required course but it’s probably not the case everywhere and honestly I could probably use a refresher myself if some good sources exist for that.
Thanks for those involved in the bot though for their work and for being open to feedback. I think the goal is a good one, I just don’t think this solution really helps but I’m sure others have different views.
One issue with poor media literacy is that I don’t think people are going to go out of their way to improve their literacy on their own just from a pinned post. We could include a link in the bot’s comment to a resource like that though.
Do you think that the bias rating would be improved by aggregating multiple factors checkers’ opinions into one score?
Yeah it’s definitely a good point, although I would argue people not interested in improving their media literacy should not be exposed to a questionable bias rating as they are the most likely to take it at face value and be misled.
The idea of multiple bias sources is an interesting one. It’s less about quantity than quality though I think. If there are two organisations that use thorough and consistent rating systems it could be useful to have both. I’m still not convinced that it’s even a solvable problem though but maybe I’m just being too pessimistic and someone out there has come up with a good solution.
Either way I appreciate that it’s a really tough job to come up with a solution here so best of luck to you and thanks for reading the feedback.
I think it should be removed
To clarify what MBFC considers “MIXED” factual reporting (the same rating they give known disinformation factory Breitbart):
Further, while The Guardian has failed several fact checks, they also produce an incredible amount of content; therefore, most stories are accurate, but the reader must beware, and hence why we assign them a Mixed rating for factual reporting.
They list like five fact checks, while The Guardian puts out basically quintuple that every day. And moreover, this is the sort of asinine nitpick that they classify as a “fact check”.
“Private renting is making people ill.” “Private renting is making people ill, but maybe this happens with other housing situations too, we don’t know, so we rate this as false.”
MBFC’s ratings for “factual reporting” are a joke.
This is my problem with MBFC, and which seems to consistently get ignored by the admins and mods pushing for the bot.
MBFC seems to rate every even slightly “left wing” news source as “mixed factual reporting” for absolutely any excuse whatsoever. The fact that they deem The Guardian as reliable as Breitbart should really tell you something.
Remove it.
No need for a bot. Obvious misinformation should be removed by the mods. Bias is too subjective to be adjudicated by the mods. Just drop it already. It’s consistently downvoted into oblivion for a reason. The feedback has been petty damn obvious. This whole thread is just because the mods are so sure they’re right that they can’t listen to the feedback they already got. Just kill the bot.
The bot is basically a spammer saying “THIS ARTICLE SUCKS EVEN THOUGH I DIDN’T READ IT” on every damn post. If that was a normal user account you’d ban it.
In literally every thread I’ve seen it post in, it gets downvoted to hell.
The bot is basically loud as fuck in a way that disrupts the comment feed.
Imagine how comments should create and add to a conversation. Imagine how various lemmy clients feed or service that conversation….
Now imagine how a double dropdown big as fuck post says “fuck you” to that conversation.
Just please consider how the form of your shit can be just as imposing as the content, which I really appreciate.
Yet somehow your posts always have me thinking “shut the fuck up” which seems antithetical to building a community.
Get rid of it entirely. In another one of your comments you acknowledged that it “seemed” like the bot is an extension of the mods telling everyone else what to think. You are close. It doesn’t seem that way, it is that way.
Also, bot is annoying AF. If you really are in love with so much, make it an opt-in service and it can DM all the psychos who want to be spammed by it.
It has been helpful and we would like to keep it around in one form or another.
Bull fucking shit. The majority of feedback has been negative. I can’t recall a single person arguing in its favor, but I can think of many, myself included, arguing against it. I hope you can find my report of one particular egregious example, because Lemmy doesn’t let me see a history of things I reported. I recall that MBFC rated a particular source poorly because they dared to use the word “genocide” to describe what’s going on in Gaza. Trusting one person, who clearly starts from an American point of view, and has a clearly biased view of world events, to be the arbiter of what is liberal or conservative, or factual or fictional, is actively harmful.
No community, neither reddit nor Lemmy nor any other, has suffered for lack of such a bot. I strongly recommend removing it. Non-credible sources, misinformation, and propaganda are already prohibited under rule 8. If a particular source is so objectionable, it should be blacklisted entirely. And what is and is not acceptable should be determined in concert with the community, not unilaterally.
Edit: And another thing! It’s obnoxious for bot comments to count toward the number of comments as shown in the post list. Nobody likes seeing it and thinking “I wonder what people are saying about this” and it’s just the damn bot again. But that’s really a shortcoming in Lemmy.
Yes! The mods starting out the discussion with their preferred outcome is so incredibly telling. This is a tool to reinforce the mods bias, deliberately or not
.
I will start by saying that I feel like we are trying to address the criticism in your first paragraph with these changes. That being said, thanks for your feedback. I particularly like the comment you shared under the “edit,” because I hadn’t seen that sentiment shared before (not saying nobody else had that issue, just appreciating you for contributing that and challenging me to think more about how we execute things).
I also would like it not add to the comment count. I am now getting inured to comment counts of “1”.
I generally like the bot and its intentions, but feel it inaccurate with my perception too often.
Just as a point of clarification, there is certainly not a community consensus among the feedback.
While you are absolutely correct in stating that there are vocal members of the community opposed to it in any form, there is also a significant portion of the community that would prefer to keep or modify how it works. The mod team will be taking all of these perspectives into account. We hope that you will be respectful of community members with whom you disagree.
I haven’t seen any strong arguments for keeping it up.
Edit: clearly there are none.
I’m gonna be Left-Center on this with reliable credibility that the bot is useless at best.
It is reporting on the source, not the content, of what is posted which is already going to be a problem for discourse.
If there are media sources that are known or proven to be a problem, I would find it preferable the bot just alert that and ignore anything else.
I appreciate the joke lol. But on a serious note, it sounds like you’re saying it’s not actually 100% useless, just that it’s being deployed too widely. Any specific suggestions on what the bot should say on those questionable sources?
Current ownership and governance of the media outlet, generally speaking. Noting if an outlet is state owned or public traded, etc might help.
Does the bot even tell the difference between an opinion piece and investigative journalism?
If a source is a proven misinformation generator then noting the proof with direct links to evidence, cases, rulings, etc. However those sources tend to disappear quickly and are constantly being generated. It is whack a mole and generates an endlessly outdated list.
The problem is it likely isn’t any information a bot can just scoop up and relay, and instead requires research and human effort.
MBFC does link to articles that are examples of misinformation. And no, the bot cannot tell if something is an opinion piece or not.
Interesting suggestion about state-owned media, hadn’t heard that before. Thanks for that
I blocked it straight away so I don’t have a dog in this fight but I’m instantly skeptical of any organization that claims to be the arbiter of what is biased and to what degree.
How much more feedback do you need to gather on the subject to understand that a bot with a garbage datasource is no use to anyone? Even opening this thread is an insult and a sign of how little you recognize and care for your community. Remove the shit bot already instead of fishing for excuses to keep it active.
Exactly! What a waste. They already got their feedback
Honestly, have a look through the whole thread. There are comments from those who, like yourself, oppose it in any form. However, please also be respectful of the many community members who are saying that it is useful or could be made useful.
The mods will be taking all of these comments into account.
The mods aren’t neutral though. They already start with a pretty strong “the bot is here to stay” which is borderline insulting to the community. Then they’re asking for ideas to make it better, which already presumes the idea is feasible or a good idea in the first place. Sure, I would make it less spammy, put the details behind a link, etc etc, but they’re already committed to the bot as a solution to their stated problem of overloaded mods. Well that could be solved in much better ways. All the energy going to this controversial bot is adding to the mod overburden!
I wound up blocking your shitty bot because it spammed pretty much every post.
Okay. This post is an attempt to solicit constructive criticism/feedback. Do you have anything more concrete to share?
Yes, maybe don’t have the bot be the first and only response on every single post. Let them gain the tiniest bit of traction first. It’s beyond annoying to see an article, go to the comments, and your bot be the only response.
I wish the comment count on a post didn’t include MBFC (or maybe bots in general).
I’ll be honest, that’s probably outside of the scope of what we can do for now. It’s definitely valuable feedback in general and I wish I could offer some kind of solution but that’s probably even outside the control of the instance admins.
Someone can feel free to correct me if I’m wrong!
On this topic the information would probably be ideally delivered by flairs/post tags which lemmy doesn’t support yet (AFAICT).
Simply having (bias:left) (factuality: high) would be much better than a whole comment.
