They noticed that some ssh sessions took 0.5 seconds too long under certain circumstances. 😲
Holy hell that’s good QA.
Microsoft employee.
Definitely not from the team working on search on Windows then.
Don’t see why you’re being downvoted, the person in question who discovered this is a postgres maintainer employed by Microsoft.
Probably people think this is a troll or something.
I wrote it because I was surprised, especially since I’m not a fan of microsoft and their policies. Lately, I have the feeling Microsoft is better than Google (relative terms) when it comes to oss.
What is additionally surprising is the breaches of Microsoft services in the last year. There is one every few weeks or so… And then they pick up a backdoor because login took 0.5 instead of 0.1s.
Anyway, his findings are amazing.
This isn’t the same thing, but I’m reminded of Minecraft.
Minecraft is a massively popular game. Notch once said he planned to make it open source when its popularity died down. But now Microsoft owns it.
Not only that, but Mojang accounts don’t work anymore. You have to have a Microsoft account to play it now. Even trying to download and play an older version of the game offline requires Microsoft to approve it. Microsoft is actively tightening the leash on the game because it makes them money. Open sourcing the game will likely never happen now. The best we can hope for it for versions to fall into public domain after 70-ish years.
That’s how I see Microsoft. They only care about what its beneficial for them to drive profits. Working on open source projects, and open sourcing a few of their tools to get the benefits of community adoption and code review is great, sure. But they’d sooner try to incorporate Linux into Windows to keep people in their surveillance ecosystem, than to open source Windows.
Remember when Windows 10 was the last version, until they changed their minds? Remember when they floated the idea of charging a recurring subscription to use Windows, before they silently dropped the idea? Remember when there was credible talk about the next version of Windows being cloud-based where they controlled all your data and you had no privacy? Hell, you have basically no privacy on Windows 10. Trying to reclaim some involves registry edits, special third party tools, and a constant battle with automatic updates reverting your changes.
I’ll say it again. Microsoft doesn’t care about OSS. It’s just currently beneficial for them to pretend they do.
Goggle seemed to care a lot about OSS, then started making everything in Android depend on their proprietary ecosystem to function. Now Google is using the dominant position they got by taking advantage of OSS adoption, and have been pushing privacy-invading standards and trying to get rid of ad blockers online, among many other things.
For these huge companies, OSS is just a tool to get more control and power. The moment it’s no longer useful, they’ll find ways to work around the license and enshitify everything again.
It keeps happening. I refuse to keep trusting bad actors every time they dangle a shiny trinket over our heads.
I do appreciate the work this person did in finding the bug. It’s not all doom and gloom.
I agree with you sentiment here. That’s why I wrote ‘relative terms’ in my comment.
Since Nadela took over, Microsoft did some open thing which benefited community. So, Microsoft opened somewhat.
During the same time, under Pichai, google went the other way: they focus more on monetization and try to control stuff the apple way. Manifest v3? Google also didn’t do anything really worth mentioning in the last 10y in terms of products. Well, except ‘attention’ article. And even this they didn’t believe in and they cannot deliver a decent product. I just tried google advanced Gemini and it’s, to put it politely, shit. Google also had some positive actions like mainlining a lot of stuff in Linux Kernel to more easily upgrade android.
So, while google is closing down and making mistakes, Microsoft is opening a bit up.
If you look the state from the last year and the state now. Microsoft improved. Google went the other way.
Microsoft doesn’t care about open source, they care about the money Cloud Services using open source bring them. I don’t think google cares as well. For reason read this: https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
The person who found the backdoor : https://mastodon.social/@AndresFreundTec/112180083704606941
Time to bring back the reproducible build hype
Probally more SLSA which reproducible builds is apart of
Reading the comments here https://news.ycombinator.com/item?id=39865810 it appears that libarchive may be tainted as well.
Lzma balls
Damn fine work all around.
I know this is an issue fraught with potential legal and political BS, and it’s impossible to check everything without automation these days, but is there an organization that trains and pays people to work as security researchers or QA for open source projects?
Basically, a watchdog group that finds exploitable security vulnerabilities, and works with individuals or vendors to patch them? Maybe make it a publicly owned and operated group with mandatory reporting of some kind. An international project funded by multiple governments, where it’s harder for a single point of influence to hide exploits, abuse secrets, or interfere with the researchers? They don’t own or control any code, just find security issues and advise.
I don’t know.
Just thinking that modern security is getting pretty complicated, with so many moving parts and all.
Here is the official statement from OpenSUSE: https://news.opensuse.org/2024/03/29/xz-backdoor/
t y for sharing.
#showerthoughts The problem is in upstream and has only entered Debian Sid/unstable. Does this mean that for example bleeding edge Arch (btw) sshd users are compromised already ?
Arch is on 5.6.1 as of now: https://archlinux.org/packages/core/x86_64/xz/
We at Nixpkgs have barely evaded having it go to a channel used by users and we don’t seem to be affected by the backdoor.
Arch had a patch rolled out yesterday [1][2][3] that switches to the git repo. On top of that the logic in the runtime shim and build script modifier was orchestrated to target Debian and RPM build systems and environments [4].
[2] https://gitlab.archlinux.org/archlinux/packaging/packages/xz/-/issues/2
[3] https://security.archlinux.org/CVE-2024-3094
[4] https://www.openwall.com/lists/oss-security/2024/03/29/4
Homebrew rolled back the release after finding out
Here’s a link to the PR for anyone who’s interested
It seems like a RCE, rather an auth bypass once though. https://bsky.app/profile/filippo.abyssdomain.expert/post/3kowjkx2njy2b
Some no-name came and without any problems asked to become a maintainer in a project used in almost any distro, took it over, put a backdoor in there and no one had any questions? In this case, everything turned out thanks to pure chance. Noname screwed up his backdoor, which attracted the attention of a guy from Microsoft, and out of boredom, he dug up what was what. And if I hadn’t messed up, or that guy from Microsoft decided to go drink beer instead of poking around in the xz code, then no one would have discovered anything. It’s scary to imagine how many of these nonames are sitting in all these thousands of open source projects, waiting in the wings to roll out a malicious patch.
Since the actual operation of the liblzma SSH backdoor payload is still unknown, there’s a protocol for securing your impacted systems:
• Consider all data, including key material and secrets on the impacted system as compromised. Expand the impact to other systems, as needed (for example: if a local SSH key is used to access a remote system then the remote system must be considered impacted as well, within the scope the key provides).
• Wipe the impacted host and reinstall it from scratch. Use known good install that does not contain the malicious payload. Generate new keys and passwords. Do not reuse any from the impacted systems.
• Restore configuration and data from backups, but from before the time the malicious liblzma package was installed. However, be careful not to allow potentially leaked credentials or keys to have access to the newly installed system (for example via $HOME/.ssh/authorized_keys).
This handles the systems themselves. Unfortunately any passwords and other credentials stored, accessed or processed with the impacted systems must be considered compromised as well. Change passwords on web sites and other services as needed. Consider the fact that the attacker may have accessed the services and added ways to restore access via for example email address or phone number in their control. Check all information stored on the services for correctness.
This is a lot of work, certainly much more than just upgrading the liblzma package. This is the price you have to pay to stay safe. Just upgrading your liblzma package and hoping for the best is always an option, too. It’s up to you to decide if this is a risk worth taking.
This recovery protocol might change somewhat once the actual operation of the payload is figured out. There might be situations where the impact could be more limited.
As an example: If it turns out that the payload is fully contained and only allows unauthorized remote access via the tampered sshd, and the host is not directly accessible from the internet (the SSH port is not open to internet) this would mean that it might be possible to clean up the system locally without full reinstall.
However, do note that the information stored on the system might have still been leaked to outside world. For example leaked ssh keys without a passphrase could still afford the attacker access to remote systems.
This is a long con, and honestly the only people at fault are the bad actors themselves. Assuming Jia Tan’s GitHub identity and pgp key weren’t compromised by someone else, this backdoor appears to be the culmination of three years of work.
I will laugh out loud if the “fixed” binary contains a second backdoor, but one of better quality. It’s reminiscent of a poorly hidden small joint, which is naturally found, and then bargaining, apologizing and making amends begin. Although now it is generally not clear where the code is more proven.
Is this only happened with SSH, or other network facing services using liblzma too?
We know that sshd is targeted but we don’t know the full extent of the attack yet.
