OK, I think I understand what this is about: the vulnerability that they reported (and Microsoft fixed) is that there was a trick you could use to run your own code with root privileges inside the container - when the system was designed to have you only execute code as a non-root user.
It turned out not to really matter, because the container itself was still secured - you couldn't make network requests from it and you couldn't break out of it, so really all you could do with root was mess up a container that only you had access to anyway.
It's crazy to me that someone can write a post called "How We Rooted Copilot" when in reality they got root in an ephemeral python sandbox container that was locked down so much that they couldn't do anything.
I read "rooted copilot" and I think they got root on a vm that is core to copilot itself.
A much more accurate title would be "How We Rooted the Copilot Python Sandbox"
I read this as them breaking out of a Python sandbox into a container. That also squares with MSFT scoring this "moderate" severity.
So am I just missing something or could you create a network connection to the "outside" world (clearly by finding your way around the local network? Start fuzzing the router endpoint, Etc. Or is Microsoft able to provide these containers where their customers can get root access to them without them having any risk of exfiltration or exploitation?
Back when openai released python interpretation it was trivial to do what they did there. There was no open network access, the only thing of interest was a little insight in to how their developers program. A couple of internal configuration files.
This is literally the same.
How does he know that the response isn't just hallucinations?
I'm telling it because I work there and I don't recognize any of those processes.
In fact I found one script named keepAliveJupyterSvc.sh in a public repo: https://github.com/shivamkm07/code-interpreter/blob/load-tes...
That repo, and its contributors are MS/Azure employees working on the service for running python code in a container. I don't know why it's under a personal account. Though it says it's a fork from an Office repo that I can't find.
It may not be a hallucination. Perhaps the Copilot code was generated from the GitHub training set?
Oh boy, this really seems to be hallucination.
Guys, chatbots are mostly token generators, they don't run programs e give you responses...it's not a simple shell program, it computes things in GPU and return tokens, in which are translated back to English.
This is very out of date. They now often trigger tooling and return the outputs of the tooling.
Not really. You're referring to agents, but the model doesn't always require agents, and the public chatbot is not connected to a shell freely evaluating arbitrary commands.
Earlier LLMs used to be a goldmine for company secrets (when it learned documents that shouldn't be on public internet). Most of it seem to be scrubbed now.
> Earlier LLMs used to be a goldmine for company secrets (when it learned documents that shouldn't be on public internet).
Sounds fake. LLMs don't usually memorize things that appear once in their training set anyway, nor have I heard about major issues accidentally training on a bunch of non-public data.
I can see how someone would believe it to be true though, since LLMs can easily hallucinate in a way that looks like this is true.
In my humble experience company secrets are mostly useless for other companies.
This reminds me of that one time after working at a company for 4 months they informed me they were in a middle of an IP lawsuit which is part of the reason they hired me to rewrite the front end without knowing that was going on. That was f*(ked for reasons.
Whatever the case, the only time people look at your social media history is to look for attacks and the only reason they will look at a company's slack messages and emails are to look for attacks during discovery.
I would argue that company secrets are mostly useless for the company but very, very useful to other companies. For this reason, there should be retention policy of a day or two for almost all communication unless it is important, required by law, or documentation. And, definitely do not share that information with the public without good reason.
The bigger issue is around "material non‑public information" in stock market terms - things like unreported sales figures which someone could use to make trading decisions.
Using that information for trading is illegal, but so is exposing that information outside of approved channels.
Then why are they secret?
Because its hard to define the parts that are really sensitive. At our work people must classify every document but a lot of people choose public for everything because it doesn't enforce any restrictions. So they can just dump it in a folder and share it with the whole company. This is not what we want them to do obviously but people are lazy, don't like to create access lists. But anyway it means we can't rely on the classification. And indicator detection like credit card and social security numbers is far from perfect. A lot of sensitive info will just be text, like about new products being developed. 3D models, code, strategy emails.
Also, if people start rooting around in everything they can take things out of context. If I send a message to my boss that I think that something we're doing is stupid, if that were public it could make some waves even though internally it's inconsequential because I'm a nobody. Also, many documents might have one or two bits that hint to really important information and having them can help finding those
As you probably know, there's tons of information in a multinational and the hardest part is finding the right stuff. This is one of the main tasks I use Copilot for. Also because outlook and SharePoint search are really terrible though. If those actually worked I wouldn't need copilot so much.
Because "mostly" does a lot of work in that sentence. Companies, like militaries, keep secret a lot of information that would be safe to release because they don't know which bits are highly unsafe.
Paranoia and not knowing which ones fall into "mostly" category :)
At most of the companies I've worked, low-grade managers love to hoard secrets. It makes them feel powerful. Someone gets promoted from Lower Level Manager Grade 4 to Lower Level Manager Grade 5 and they feel all "Oooh! Look at the new things I know!"
My mother-in-law is like this with knowing what various relatives are doing. Being the gatekeeper of knowledge gives her imagined power. I guess it's just part of the human condition.
Being such a person that fixes lots of stuff for other people nothing I do is secret but learning to do it seems too hard for most. What I do is try to delegate if I find people that do want to learn.
If someone shows me they are good at something they are going to have to expect being sent trickier problems.
Sometimes it might seem like I keep things a secret. I am probably just having a bad day.
That has an awful lot to do with what "the thing" is. I'm sure there are a few people out there doing it just to feel more important, but often there's a good reason for denying someone access - either it's just a terrible idea to begin with or they don't know you well enough to trust you without someone else (i.e. their boss) specifically requesting it.
I could be off base here about your experience, but I know that some people made the same comments about me when I pushed back on sharing dangerous credentials with inexperienced coworkers. Damned if you do, damned if you don't.
It may depend on what the script is for or the system being used. Segregation of duties is a risk mitigation principle of ISO 27001 to reduce fraud, waste, and error.
That's why corporate espionage is a really lucrative industry?
Of course it depends what secrets. 99% will just be internal process drivel and inter departmental bickering but there's some real important stuff in there too.
Do you have any concrete examples of this? I have not seen any myself.
I looked for an alleged case of an LLM apparently reproducing email signatures—but couldn’t find it exactly, and of course many email signatures have been published over the years, especially on newsgroups. (Maybe it was conspiratorial kind of thinking from web commenters assuming ChatGPT was training on emails users were feeding it, which as mentioned certainly doesn’t need to be the case.)
Something like the top screenshot here, though:
https://www.zdnet.com/article/chatgpt-can-leak-source-data-v...
(not parent commenter but) tl;dr no
When companies (non-"tech") started adopting them they also had no "guardrails" for content outside what the intent of such products were (dunno what the standard term for this is).
There was a boba tea company that had a free, no-sign-in required LLM that I used to generate a few bash scripts before ChatGPT free-tier started.
Source?
Don't really seam to be a vulnerability?
The safety in the system is that the code is executed in a container.
Assuming the container was isolated. Which I'd assume it was.
Seems like they could have taken a shortcut by giving copilot a sudo binary to use as base64.
You would need to change ownership of the file to root also.
[dead]
> We reported the vulnerability to Microsoft in April and they have since fixed it as a moderate severity vulnerability. As only important and critical vulnerabilities qualify for a bounty award, we did not receive anything, except for an acknowledgement on the Security Researcher Acknowledgments for Microsoft Online Services webpage.
The important part:
> Now what have we gained with root access to the container?
> Absolutely nothing!
> We can now use this access to explore parts of the container that were previously inaccessible to us. We explored the filesystem, but there were no files in /root, no interesting logging to find, and a container breakout looked out of the question as every possible known breakout had been patched.
> a container breakout looked out of the question as every possible known breakout had been patched
Are you not concerned about all the other platforms that rely on containers as security boundaries between tenants? There are a lot of them.
It is hard to answer that since the stack is so convoluted. Some parts are forced on the user. Copilot is built into Microsoft Office workplace applications.
If you break out of a container, do you have access to the same system that serves these applications? Who knows, it looks like a gigantic mess.
I expect that they run their containers more isolated as virtual machines. So they have bigger problems of there is a breakout possible.
Severity is based on impact. What was the impact here beyond single container and that specific user instance? Feels like moderate was okay, or even too high.
Maybe this was their honeypot container.
I'll never understand why people do free dev work for multinational trillion dollar conglomerates.
It's still good for reputation. This is by a researcher at a company, so a benefit for both of them. Plus if we didn't have bug bounty programs, they'd have to willingly work at Microsoft to do this research.
This could have turned badly in terms of reputation if they had tried to complain that the vulnerability should be critical, e.g. or using other ways to seek attention for not getting bounty, but current way was rather neutral way.
Could say the same thing about open source software.
It's why I don't understand why people believe in "open source". Why would I contribute free dev work to a billion dollar corporation? I do believe in "Free Software" which is contributing free dev work to my fellow man for the benefit of all man mankind.
Free software and open source are two ideologies for the same thing. Free Software is the ideology of developing the software for the benefit of mankind (it's sometimes termed a "political" stance but I see it as an ethical stance). Open source is the ideology of saving money at a corporation by not paying the developers. Sure open source can benefit mankind but will only develop corporate software for money. When developing on my own time, I will focus on software that either personally benefits me or benefits other regular people.
at least under some licenses like GPL/AGPL you get some code back.
Why do basic science which benefits everyone else for free?
Meh it depends whether you use those things of course. There's other IDEs, other languages. And Microsoft isn't doing this out of charity. A lot of the really useful plugins are not working on the open source version, so people that use them provide telemetry which is probably valuable. Or they use it as a gateway to their services like GitHub Copilot.
If a mega corporation gives you something for free it's always more beneficial to them otherwise they wouldn't do it in the first place.
I think the argument is that when big companies make use of stuff, it gets more scrutiny and occasionally they contribute back improvements, and the occasional unicorn gets actual man hours paid for improving it. So if your project gets big enough, it's beneficial. But you have to have a MIT/BSD license usually, because companies will normally stay away from GPL.
I know maintainers of projects have been hired directly by companies using their code as it is the most expedient way forward. Others might just offer up enough money to get the maintainer to take up a few of their specific issues/requests in a way that makes it worth their while. Just because someone is working on a project that is open source does not mean that money cannot be involved in the development. The company paying that money knows that the updates released as a normal part of the project will be available to anyone else using it as well.
No, we can't say. I'm not an asshole, it helps people, and companies shun GPL licenses. That's not a valid comparison. Microsoft can go fuck itself, people around me love my software and it improves their lives.
It's... 100% a valid comparison? The point is that doing free vulnerability research isn't irrational, not that doing open source work is bad. You're twisting yourself into a pretzel trying to keep the original argument alive.
It's called "I use the software, I already want to improve the software I'm using, so after I improve it I'll contribute the improvements I've already made to the broader community."
Granted, I myself have been guilty of not giving back to the open source community this way in the past, but I won't pretend that was reasonable or ethical of me!
edit: after reading some commemnts, i realize i may have meant to say "free software" instead of "open source"
Well a lot of people do this kind of work to be able to commit crimes.
It mostly pays in career benefits. Same reason why plenty intern for free.
Who is interning for free as a software engineer?
Me
People people who did bootcamps and thus are too risky to hire for most roles and cannot get into the standard CS hiring pipeline. Especially now that junior roles are drying up.
In professions like fashion, virtually everyone seems to at some point.
i don't think they did the work for them. they just reported it to them.
M$: If you're not going to send any money, send some swag. Make it cool and hackers will wear it, and now you have them advertising for you and possibly even want to work for you. Culture is a tool, and hackers have culture, so learn how to use it.
As you’ll see elsewhere, “root” got them literally nothing. They tried but there was nothing to be had.
They didn't find anything they could do with it but that container isn't there for no reason. I agree with the rating but it's nonetheless worrying. You don't leave the house you bought unlocked because there's nothing in it to steal yet.
More like leaving your front gate unlocked.
There was a time in programming that tried to avoid monstrosities like the Python scientific data stack combined with Copilot integration hacks.
That time produced qmail and postfix. We are back to the early 1990s.
It's wild how easy this was. I feel like we're really in the wild west era of security with these AI tools -- reminds me of early Web 2.0 days, like when "samy is my hero" hit and Myspace didn't even have a security team. I anticipate many high-profile incidents before they figure out how to tame this beast.
I don't think there's really much "AI" involved in this; this is basically like breaking any hosted code IDE. I get that an LLM was the direct vector, but the underlying security issue is common to everything that runs remote code.
I have to give Microsoft props here. Most companies don't bother to lock things down well enough, but they were thorough.
I would give the one engineer the credit for doing things better, not Microsoft. Microsoft overall culture of security is terrible. Look at the CISA report.
Okay, so I give the team that put this together credit. Hopefully the parent company sees based on this that it's worth letting teams invest more in quality and security work, over features.
Microsoft has islands of security excellence in what these days is a sea of mediocrity.
What CISA report?
Not OP, but guessing they were referencing this one:
https://www.cisa.gov/resources-tools/resources/CSRB-Review-S...
I’m guessing they mean this one:
https://www.cisa.gov/news-events/bulletins/sb25-167
> Microsoft--Microsoft 365 Copilot
> Description Ai command injection in M365 Copilot allows an unauthorized attacker to disclose information over a network.
> Published 2025-06-11
> CVSS Score 9.3
> Source Info CVE-2025-32711
https://www.cve.org/CVERecord?id=CVE-2025-32711
And maybe they are referring to this engineer from the linked advisory notes?
https://msrc.microsoft.com/update-guide/vulnerability/CVE-20...
> Acknowledgements
> Arantes (@es7evam on X) with Microsoft Aim Labs (Part of Aim Security)
I don’t know specifically how this container was implemented, but Microsoft has a standard way to do isolated Python sandboxes: https://learn.microsoft.com/en-us/azure/container-apps/sessi... Hopefully this feature is using that or something similar.
It seems weird to me that copilot sometimes refuses to execute code but sometimes allows it. What exactly are they aiming for?
They're not. It's better to think of Copilot as a collaborative storytelling session with a text autocomplete system, which some other program is rudely hijacking to insert the result of running certain commands.
Sometimes the (completion randomly selected from the outputs of the) predictive text model goes "yes, and". Other times, it goes "no, because". As observed in the article, if it's autocompleting the result of many "yes, and"s, the story is probably going to have another "yes, and" next, but if a story starts off with a certain kind of demand, it's probably going to continue with a refusal.
funny how it sounds kind of the opposite of how people might work. Get enough 'no's from someone and they might finally cave in. get enough 'yes'es and they might get sick of doing everything you ask.
In the modern world vulnerabilities are stacks. Asserting that "the container itself was still secured" is just a statement that the attackers didn't find anything there. But container breakouts and VM breakouts are known things. All it takes is a few mistakes in configuration or a bug in a virtio driver or whatever. This is a real and notable result.
If they had found and reported a container breakout I expect they would've got a bug bounty from it!
Are there any known unfixed container breakouts at the moment in the kind of systems Microsoft are likely to be using here?
The problem is that you're encouraging people to keep stuff like this to themselves until they can use it to perform an exploit that they'd get paid for, which is the opposite of what Microsoft wants - they'd much rather you report it now so that if an exploit does get found that requires root they would potentially be protected.
The simple question for Microsoft to answer is - does it matter to them if attackers have root access on the container? If the answer is yes then the bug bounty for root access should at least pay something to encourage reporting. If the answer is no then this shouldn't have been marked as a vulnerability because root access is not considered a security issue.
Presumably someone with mal-intent would sit on the root vulnerability waiting for a container breakout bug to come around.
A container root exploit isn't a critical security vulnerability either, describing it as moderate seems fair, but it's a reasonable step towards one.
That is exactly what it is.
Propper security I depth means that when trusted actors betray the system, the damage is limited.
Not really the right metaphor. A $5 wrench isn't a "vulnerability" because it's $5! Tools that are accessible to everyone are part of the threat model, not something you can eliminate or avoid. This trick is novel and new.
Like, consider your personal cult was built around an "unopenable" bolt-tighted box. Then someone invents the wrench in an attempt to open it. That would be a clear "security vulnerability", right?
Almost certainly yes, since at that point all you're looking for is a Linux kernel LPE.
> they would've got a bug bounty from it!
Why do you think that, rather than get sued? I am curious
Microsoft have a bug bounty program which is credible and well run.
Suing people who responsibly disclose security issues to you is a disastrous thing to do. Word spreads instantly and now you won't get any responsibly disclosed bug reports in the future.
Microsoft are way too smart to make that mistake.