> Should we assume that all software is more compromised than the kernel?
Absolutely, the front door has always been unlocked. We were previously protected by the economics of exploit detection favoring the developers, but now we have entered a new world.
The issues arise almost everywhere we look. It's frustrating because the user experience in edge systems has barely changed in the last fifteen years. We could have rode the advances in semiconductor manufacturing to embed more layers of protection such as CHERI while slightly slowing down the rate of performance improvements. Instead we have only marginally upgraded security and largely added features which the users scarcely notice.
I am hopeful about the long term prospects of formal verification, but having humans reviewing the proof statements and confirming they match our desired properties will take many years. Meanwhile, AI assisted hacking is an urgent issue. So we will have to delegate implementation, proof specification, and proof development to the AIs in the short term. Maybe that goes well, but who knows. Interesting times.
I recall reading that every IBM mainframe OS release had 1000 bugs in it. (This was in the 1970s/80s.) Sorry I can't supply a source, after 4-5 decades. It was probably an article in Datamation magazine.
Now, I don't know how many of those bugs were privilege escalations, or could be leveraged to be. I'm not sure that people tried to exploit in the same way back then (or, if they did, if they said anything publicly about the results). Still, even if the majority of those bugs were not exploitable, OSes were much smaller then.
So that's my conclusion of how we got here: We've always been here.
Thanks. I'm not a programmer so I can't understand what is happening in the kernel by looking at the changelog, so I'll ask: Are the problems remediated structurally so that nothing like it can occur in the future, rather than just patching it up? Are there "lessons learned" that are being applied and documented here (like, "this problem looks like it's shape transports to other subsystems as well, let's fix them all in a disciplined way")?
As models get more advanced and generate an increasing throughput of increasingly higher quality results, how are you determining when to trust its results and when human review is needed on what?
E.g., 948 results is a high number to review in a timely manner unless your team is very large or spending several months combing through the results
EDIT: nvm i shouldve kept reading, answered
> At this scale, manually investigating every report is impractical for our research team.
We therefore focused on validating findings that a) were reachable by a low-privileged user, b) required no special capabilities, and c) produced measurable evidence of memory corruption.
It's a bittersweet result, like finding asbestos in your house. I'd rather know than not, but I would much prefer not having it in the first place.
I understand this work was based on a dated snapshot. Will there be any continuous effort, scanning new PRs as they land, to prevent new weaknesses from being added?
(I am the author the pipeline, just created an account to answer some questions :D)
My goal is to work towards a vuln-free Internet. What I observed is that stronger models can find many more vulnerabilities that weaker models cannot. If compute permits (there are other projects to secure besides Linux), I plan to scan Linux again with every newer generation of cyber models.
I've always considered local escalations to be possible in Linux (i.e. having user access almost certainly means you can get root). However, 948 vulnerabilities is kind of shocking. Hopefully we are getting to the end of the part where it "gets worse before it gets better".
> We've developed an autonomous vulnerability discovery, exploitation, and patching pipeline. In one run with our cyber model, it validated and generated patches for 948 local privilege escalation vulnerabilities in Linux at an API cost of $1.2 million.
Imagine if open-source organizations other than the Linux Foundation (and Mozilla I guess) could throw $1.2M of API-rate LLM usage at security.
> For the near and medium term, having access to a low-privileged account on an operating system, that isn’t up-to-date on its latest available patches should be considered functionally equivalent to root access for a well-resourced threat actor.
These are becoming more and more marketing materials than actual findings, because OAI probably hit the wall in making their models smarter (just like how google try to specialize their models in legal or whatever), but they need to remain relevant in the news. Sol 6.1 sucked so bad in reasoning that either it’s actually a bad model or users behind the scenes get degraded outcome while using all the resources for these marketing stunts. Stop, it’s embarrassing, make better models or be gone, at least we will get cheaper RAM to advance the development of open weights models.
Is the kernel simply too complex to reason about? Is it an underlying flaw in the security model or language implementation?
Should we assume that all software is more compromised than the kernel?
Absolutely, the front door has always been unlocked. We were previously protected by the economics of exploit detection favoring the developers, but now we have entered a new world.
The issues arise almost everywhere we look. It's frustrating because the user experience in edge systems has barely changed in the last fifteen years. We could have rode the advances in semiconductor manufacturing to embed more layers of protection such as CHERI while slightly slowing down the rate of performance improvements. Instead we have only marginally upgraded security and largely added features which the users scarcely notice.
I am hopeful about the long term prospects of formal verification, but having humans reviewing the proof statements and confirming they match our desired properties will take many years. Meanwhile, AI assisted hacking is an urgent issue. So we will have to delegate implementation, proof specification, and proof development to the AIs in the short term. Maybe that goes well, but who knows. Interesting times.
Now, I don't know how many of those bugs were privilege escalations, or could be leveraged to be. I'm not sure that people tried to exploit in the same way back then (or, if they did, if they said anything publicly about the results). Still, even if the majority of those bugs were not exploitable, OSes were much smaller then.
So that's my conclusion of how we got here: We've always been here.
We developed AIs that can find them.
And we're hiring! https://openai.com/careers/search/?q=cyber
E.g., 948 results is a high number to review in a timely manner unless your team is very large or spending several months combing through the results
EDIT: nvm i shouldve kept reading, answered
> At this scale, manually investigating every report is impractical for our research team. We therefore focused on validating findings that a) were reachable by a low-privileged user, b) required no special capabilities, and c) produced measurable evidence of memory corruption.
I understand this work was based on a dated snapshot. Will there be any continuous effort, scanning new PRs as they land, to prevent new weaknesses from being added?
A system that admits THOUSANDS of such vulnerabilities is certainly flawed in several fundamental aspects
Imagine if open-source organizations other than the Linux Foundation (and Mozilla I guess) could throw $1.2M of API-rate LLM usage at security.
*buries head in sand*
*cries*