Google has paid out nearly 10 million dollars in bounties just for Chromium? Just one reason nobody else is able to compete. The money supply is infinite.
Google has paid out nearly 10 million dollars in bounties just for Chromium? Just one reason nobody else is able to compete. The money supply is infinite.
It's interesting to browse the large collection under chrome/browser/ui and ponder over how many of them are use-after-free for data where the performance of manual memory management really doesn't matter. Like [1] which is around the lifecycle of a "choose a file" dialog.
It feels like in the big picture sense it would be better to just always using some sort of smarter/slower pointers in this kind of code just for extra defense. I saw in [2] there is some sort of `raw_ptr<T>` type [3] that seems to intend to help, so maybe the crash in [2] was actually successfully defended against?
It's too bad that there's not a good way to have a broader way to switch between dialects within a project where in one place it's "this portion of the code is perf-critical and carefully reviewed" and in another "this portion of the code is perf-oblivious and has lots of async state that's easy to get wrong". I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear.
[Disclaimer: worked on this code many years ago, wouldn't be surprised if I caused >0 of these bugs...]
[1] https://bugs.chromium.org/p/chromium/issues/detail?id=120103... [2] https://bugs.chromium.org/p/chromium/issues/detail?id=132323... [3] https://source.chromium.org/chromium/chromium/src/+/main:bas...
I had wanted to use Oilpan GC in the browser process for this reason, but the browser side folks were very opposed to using the blink libraries back then.
Most of the Chrome UI code is written with Web UI at least. These days I think they should consider typescript for more orchestration things within the browser too. It's a proven strategy by Electron.
I think the momentum is actually around MiraclePtr now though (as you found).
Oilpan isn't without issues though: finalization causes quite a few headaches, implementation details like concurrent marking make it hard to support things like std::variant, and the interface between Oilpan and non-Oilpan types often introduces opportunities for unsafety.
Indeed. It's tradeoffs, but they've been sufficient for much of the codebase for a very long time. Taking no major action (Oilpan or memory safe language) for nearly a decade was also a tradeoff. I don't think the long list of security issues there was worth it.
Hmm. The blink dirs seem to have a very large number of security issues relevant too, despite oilpan. Which is what we'd expect, honestly; oilpan does not solve all uaf problems, uafs are not all (or even the majority of) security problems, etc. The combo of "sufficient for much of the codebase" and "the long list of security issues" paints a picture of most of the codebase being secure due to oilpan, while UI code is riddled with holes due to its lack. The reality is dramatically more nuanced (as you know, but readers of your comment might not).
As a views maintainer, I'm familiar with some of the security bugs on the UI side. Clickjacking-type issues are more common there than uafs. Uafs are an issue, but the problems are not so much due to problematic use of unsafe c++ types and idioms as problematic API designs -- resulting in, for example, cases where it's not clear whether an object is expected to be able to respond safely to calls at all, whether or not it's technically alive. Oilpan, MiraclePtr, proper use of smart pointers would all help bandaid the uafs, but are in many cases difficult to apply correctly without understanding (and often fixing) the underlying systemic design problems. Which is happening, but slowly.
There are also more dimensions of tradeoffs involved, but this is long winded enough as it is. The tldr is that at this point I would consider a couple other options better uses of effort for tackling this specific problem compared to converting browser types to oilpan.
Possibly another way of expressing your point: after writing my above comment, I found myself wondering how much a system that kept these UAF pointers alive longer (to eliminate the UAFs, e.g a GC) would actually reduce the attack surface -- the UAF-ish bugs are still bugs, and code poking at a GC-preserved object that the rest of the code doesn't really expect to still be alive might itself be pretty fraught.
the UAF-ish bugs are still bugs, and code poking at a GC-preserved object that the rest of the code doesn't really expect to still be alive might itself be pretty fraugh
For the LayoutObject heirarchy - the team doing that conversion added a NOT_DESTROYED() macro for this reason. It's gross, but was the least worst option.
As an aside - the performance of oilpan is broadly net positive now if you avoid some of the pitfalls. (The largest being a write into a Member<> requires a write-barrier). E.g. Things become trivially destructible, and no ref incrementing/decrementing, etc.
raw_ptr<T> is in fact a smart pointer wrapper that mitigates exploitation of most use-after-frees https://security.googleblog.com/2022/09/use-after-freedom-mi...
It might be better to say raw_ptr<T> is a dumb pointer wrapper. raw_ptr is discouraged in favor of unique_ptr/WeakPtr/RefCounted; but it is used in places where you'd have a T* at rest.
It's too bad that there's not a good way to have a broader way to switch between dialects within a project where in one place it's "this portion of the code is perf-critical and carefully reviewed" and in another "this portion of the code is perf-oblivious and has lots of async state that's easy to get wrong". I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear.
You are describing Rust's "unsafe" keyword.
And this kind of code is very literally the original impetus of it. The language was sort of originally designed to implement a browser, after all.
I have written a lot of Rust, and I am describing something different. Rust lets you write code that cares a lot about memory (with lifetimes etc.) with either compiler assistance or without. What I am describing is code that doesn't much care about memory and would be happy to sacrifice performance for it.
I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear.
Python, with perf critical sections written in C or Rust is pretty much that. From what I hear the Rust-Python bindings are especially good, and make the correctness part easier even in the performance critical parts.
Or you can go the opposite way and call out to a scripting language from your fast language. Today everyone is hyped about wasm for that, but we also have about two decades of computer games using lua for that (and games are probably the single biggest category of performance sensitive software)
This is actually really similar to something I've been wanting to build for a long time. In my case I've thought it would be useful to have a way to calculate the likelihood for a given change to break things based on the history of breaking changes in the same file or area of the file. Basically a riskiness score for each change. The risk score could be associated with each PR and would provide a signal for reviewers about which code should get a bit of extra attention as well as highlighting the risky changes when they are being deployed.
The tricky part would be tracking the same part of the code as it moves up and down because of insertions/deletions above it which would cause problems for a naive algorithm based on line numbers.
Just doing it at the file level, like this does, might be good enough to be useful though.
Not just the code itself, but the author. I worked with a guy that wrote at least one bug every time he created a PR.
> wrote at least one bug every time he created a PR.
An economic hero. A man of the people. Creating job security for QA departments!
A friend working in office of a big-tech company located in Denmark said "one bad engineer like me working in Copenhagen can put food on the table for 20 Bulgarians working in customer support".
Since that day I always wanted to get into FAANG-type companies, writing buggy code is basically philantropy.
This thread made some people question their no-bugs-allowed ideal, which is apparently a misanthropy…
"If debugging is the process of removing software bugs, then programming must be the process of putting them in."
This has been my job for over 2 years now!
We do it on a symbol level after statically analyzing each change, and everything in the monorepo daily. Our remedy to high risk changes is to run more tests, client tests not unit tests. Sometimes there are 100k client tests to pick, so we rank them and run a small subset.
It is a hard problem. One interesting observation is that there is a culprit symbol or two in the culprit change, but its connectivity is very similar to non culprits in the same change.
Another observation is that the transitively modified callgraph after a change is pretty big, a depth of 50 is not unusual. It is hard to get many useful signals out of it beyond amount of overlap in transitively affected symbols between change and test.
We found file level and build target level to be too coarse, but AST symbols are working.
100k client tests. Sounds like a lot. Is it integration tests or UI tests? How many tests overall have you got? I’m just curious
We add support by project, and the prototypical project we started with had 1M test reverse dependencies, a quarter of that was eligible test targets that we could recommend (based on language written in). This is probably the biggest project that we would ever find to support in the monorepo.
Some are UI tests, but we don't recommend those, because we found they don't catch breakages as often so we don't support the language they're written in. The tests we recommend are often integration type tests in that they call very higher order functions and often many of them.
Code location (+) provenance/authorship (+) data flow analysis for adjacent sensitive code.
I would add it to my review tools fwiw.
I'm currently reading a book about that topic! https://pragprog.com/titles/atcrime/your-code-as-a-crime-sce...
Impressive visualization! Also kind of keeps the vibe of the internal Google tooling, minimalistic and down-to-earth pragmatic
Google tooling of 2010, before everything needed to load 10 megabytes of javascript and had loading spinners for everything.
Hey, but at least we have rounded corners now, which is nowhere to be seen in your 2010s <table>-heavy borgmaster summary /s
No, back then we would use images to make rounded corners. It was… as awful as it sounds…
Triggered!
From the submission CSS file:
details {
margin: 5px 5px 5px 12px;
border: 1px solid #aaa;
border-radius: 4px;
padding: 4px;
}
(the border-radius CSS property was first proposed in 2002 [!] as far as I can tell: https://www.w3.org/TR/2002/WD-css3-border-20021107/)
I try to keep my projects simple, client-sided, and self-contained. The way this page is set up isn't perhaps the best practices for the modern web, but you can download just the html file and it'll probably still work in 30 years time.
How about a version where the monetary amounts are normalized by the number of lines of code?
Why do you ask? Genuinely curious, because lines of code is a moot point in software and security.
I wouldn't say that it's a moot point in every context. The metric we'd get here would amount to "given that this developer made a loc change, what is the monetary risk involved".
Developers that are high on this metric might want to allow down and think twice next time they commit.
Or not. Metrics are likely not very useful in general.
It's the same as the idea of per-capita normalization.
Because it's interesting. If "X" is 1000 lines of code and has cost $20k in bounties, and "Y" is 1 000 000 lines of code and has also cost $20k in bounties, it's interesting to see that feature X has relatively more high profile bugs when it probably does much less.
It’s a valid measure of the amount of code. 10 bugs in 100k LOC speaks of a very different quality than 10 bugs in 1k LOC.
Or normalize by number of word spilt on the bug (as a proxy for complexity)
I feel like this is 1 step away from running "blame" and associating dollar values with the security bugs caused by each engineer...
... and I bet the top 1 engineer didn't actually add any bug to the code base, they just did a few repo-wide refactor.
"refactor, no functional changes"
Narrator: There were functional changes
Time to get the svn praise back and do it sarcasticly.
Nice one, given that this probably went down to the diff level it would be interesting to see it weighted by LOC changed, e.g. 10 lines in file A and 1 line in file B would mean file A gets 1/11th of the money assigned because it was the majority of the bug.?
Or perhaps a LOC changed / file LOC based distribution. That would be how buggy each file is, with a $$$ tag.
This would show the effect you want. Code like tests are likely to be very verbose, while often the actual vulnerability will come down to a handful of characters.
Depends on which effect you are after.
I was thinking of ROI for reading code. Regularly having a one line issue in a 100 LOC file is very different from a two line issue in a 10'000 LOC file.
And yes, tests need to be excluded, of course. But looks like that's done?
Very cool! I think it's missing some entries though. I'm pretty sure we've had at least one in third_party/ffmpeg. Those fixes often land upstream first which might make tracking difficult.
It's using whatever Git Watcher comments on the monorail bugs.
IIRC, that bot used to be called bugdroid. I forget when it switched over, probably somewhere in 2020.
Nitpick: don't include DEPS, AUTHORS, or BUILD.gn files.
I ported this to a treemap visualization[1]: https://vrp-treemap.surge.sh/
The treemap library was authored by evmar, chrome OG who's also in this thread.
It would be nice to also display the average reward per file on each node.
This is a very neat visualisation. Even if a little CPU intensive expanding areas! I'd hope the Chrome team has something similar internally, it looks really useful to understand the attack surface, as it were.
Such a cool idea, and great execution.
Edit: Is the raw data somewhere? A sunburst or tree map would be worth trying out
This is a really cool view on where to aim efforts
It's open source. You're free to base your own browser on Chromium.
Then it's not much of a competition. It's just more chrome market share, with a different name.
Open source is what you make of it. If you have no plans to change Chromium, then sure, "It's just more chrome market share".
I would argue that you probably shouldn't be making a web browser unless you have plans to change the status quo for web browsers.
in practice it's just chrome and firefox, just like it was america and the soviets.
it's irrelevant what flag your fork (nation state) flies when its marching orders come from above.
That's the difference between theory and practice.
In theory, sure.
In practice, there are lany chrome forks and they all chrome with a few features on top.
In theory you can beat usain bolt by just running faster than him.
In practice nobody does.
This comment is like saying that all Unreal Engine 5 games are the same game just with different names. This is not the case. The engine is what powers and helps you build the product you want. 2 different companies can use the same engine and have their end products compete against each other.
But if you fork it you cannot paid that amount to keep it bug "free".
You can merge in the paid for fixes though. Depends how divergent you want to be.
Apple could but they don’t want
Eh? Apple do want, I'm not sure what you're basing your claim off of; https://security.apple.com/blog/apple-security-bounty-upgrad... says they paid out $20 million in bug bounties in the 2.5 years before that post was made.
I meant the competition to Chrome
On iOS they are winning :-)
Apple will have to. If EU forces browser choice to iOS, there is a considerable possibility of the last bastion of browser diversity to be lost.
I know, it's kind of hilarious considering Apple, but it's chrome 70%, Safari 20% and the rest is various Chrome-based browsers (e.g. Edge) and Firefox with 3.35%.
So Apple has a choice of losing control of the most important app or improving their product, so there is no reason to switch. I am pretty sure they love control more than anything.
That's nothing compared to the amount they spend in employee salaries for folks working on chrome.
one is hard to receive, the other any internet entity with a bank account can acquire
This seems like a tiny amount given the importantance of Chrome. Surely it would be rational for Google to pay 10x higher bounties?
These are just the bounties. I am sure the total compensation for Google's in-house security team is larger than this amount.
Apple and Microsoft would both consider $10m a rounding error