return to table of content

Chromium bug bounty money tree browser

perryizgr8
18 replies
1d7h

Google has paid out nearly 10 million dollars in bounties just for Chromium? Just one reason nobody else is able to compete. The money supply is infinite.

Sephr
7 replies
1d7h

It's open source. You're free to base your own browser on Chromium.

BiteCode_dev
4 replies
1d7h

Then it's not much of a competition. It's just more chrome market share, with a different name.

Sephr
2 replies
1d6h

Open source is what you make of it. If you have no plans to change Chromium, then sure, "It's just more chrome market share".

I would argue that you probably shouldn't be making a web browser unless you have plans to change the status quo for web browsers.

dontupvoteme
0 replies
17h9m

in practice it's just chrome and firefox, just like it was america and the soviets.

it's irrelevant what flag your fork (nation state) flies when its marching orders come from above.

BiteCode_dev
0 replies
1d4h

That's the difference between theory and practice.

In theory, sure.

In practice, there are lany chrome forks and they all chrome with a few features on top.

In theory you can beat usain bolt by just running faster than him.

In practice nobody does.

charcircuit
0 replies
23h11m

This comment is like saying that all Unreal Engine 5 games are the same game just with different names. This is not the case. The engine is what powers and helps you build the product you want. 2 different companies can use the same engine and have their end products compete against each other.

wslh
1 replies
1d7h

But if you fork it you cannot paid that amount to keep it bug "free".

quickthrower2
0 replies
1d5h

You can merge in the paid for fixes though. Depends how divergent you want to be.

haunter
4 replies
1d6h

Apple could but they don’t want

Cthulhu_
2 replies
1d6h

Eh? Apple do want, I'm not sure what you're basing your claim off of; https://security.apple.com/blog/apple-security-bounty-upgrad... says they paid out $20 million in bug bounties in the 2.5 years before that post was made.

haunter
1 replies
1d6h

I meant the competition to Chrome

quickthrower2
0 replies
1d5h

On iOS they are winning :-)

mnau
0 replies
18h42m

Apple will have to. If EU forces browser choice to iOS, there is a considerable possibility of the last bastion of browser diversity to be lost.

I know, it's kind of hilarious considering Apple, but it's chrome 70%, Safari 20% and the rest is various Chrome-based browsers (e.g. Edge) and Firefox with 3.35%.

So Apple has a choice of losing control of the most important app or improving their product, so there is no reason to switch. I am pretty sure they love control more than anything.

surajrmal
1 replies
1d4h

That's nothing compared to the amount they spend in employee salaries for folks working on chrome.

dontupvoteme
0 replies
17h10m

one is hard to receive, the other any internet entity with a bank account can acquire

josephmiller
1 replies
1d6h

This seems like a tiny amount given the importantance of Chrome. Surely it would be rational for Google to pay 10x higher bounties?

omoikane
0 replies
23h18m

These are just the bounties. I am sure the total compensation for Google's in-house security team is larger than this amount.

petesergeant
0 replies
1d5h

nobody else is able to compete. The money supply is infinite

Apple and Microsoft would both consider $10m a rounding error

evmar
11 replies
1d3h

It's interesting to browse the large collection under chrome/browser/ui and ponder over how many of them are use-after-free for data where the performance of manual memory management really doesn't matter. Like [1] which is around the lifecycle of a "choose a file" dialog.

It feels like in the big picture sense it would be better to just always using some sort of smarter/slower pointers in this kind of code just for extra defense. I saw in [2] there is some sort of `raw_ptr<T>` type [3] that seems to intend to help, so maybe the crash in [2] was actually successfully defended against?

It's too bad that there's not a good way to have a broader way to switch between dialects within a project where in one place it's "this portion of the code is perf-critical and carefully reviewed" and in another "this portion of the code is perf-oblivious and has lots of async state that's easy to get wrong". I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear.

[Disclaimer: worked on this code many years ago, wouldn't be surprised if I caused >0 of these bugs...]

[1] https://bugs.chromium.org/p/chromium/issues/detail?id=120103... [2] https://bugs.chromium.org/p/chromium/issues/detail?id=132323... [3] https://source.chromium.org/chromium/chromium/src/+/main:bas...

esprehn
5 replies
23h51m

I had wanted to use Oilpan GC in the browser process for this reason, but the browser side folks were very opposed to using the blink libraries back then.

Most of the Chrome UI code is written with Web UI at least. These days I think they should consider typescript for more orchestration things within the browser too. It's a proven strategy by Electron.

I think the momentum is actually around MiraclePtr now though (as you found).

zetafunction
4 replies
23h19m

Oilpan isn't without issues though: finalization causes quite a few headaches, implementation details like concurrent marking make it hard to support things like std::variant, and the interface between Oilpan and non-Oilpan types often introduces opportunities for unsafety.

esprehn
3 replies
22h55m

Indeed. It's tradeoffs, but they've been sufficient for much of the codebase for a very long time. Taking no major action (Oilpan or memory safe language) for nearly a decade was also a tradeoff. I don't think the long list of security issues there was worth it.

pkasting
2 replies
16h54m

Hmm. The blink dirs seem to have a very large number of security issues relevant too, despite oilpan. Which is what we'd expect, honestly; oilpan does not solve all uaf problems, uafs are not all (or even the majority of) security problems, etc. The combo of "sufficient for much of the codebase" and "the long list of security issues" paints a picture of most of the codebase being secure due to oilpan, while UI code is riddled with holes due to its lack. The reality is dramatically more nuanced (as you know, but readers of your comment might not).

As a views maintainer, I'm familiar with some of the security bugs on the UI side. Clickjacking-type issues are more common there than uafs. Uafs are an issue, but the problems are not so much due to problematic use of unsafe c++ types and idioms as problematic API designs -- resulting in, for example, cases where it's not clear whether an object is expected to be able to respond safely to calls at all, whether or not it's technically alive. Oilpan, MiraclePtr, proper use of smart pointers would all help bandaid the uafs, but are in many cases difficult to apply correctly without understanding (and often fixing) the underlying systemic design problems. Which is happening, but slowly.

There are also more dimensions of tradeoffs involved, but this is long winded enough as it is. The tldr is that at this point I would consider a couple other options better uses of effort for tackling this specific problem compared to converting browser types to oilpan.

evmar
1 replies
16h5m

Possibly another way of expressing your point: after writing my above comment, I found myself wondering how much a system that kept these UAF pointers alive longer (to eliminate the UAFs, e.g a GC) would actually reduce the attack surface -- the UAF-ish bugs are still bugs, and code poking at a GC-preserved object that the rest of the code doesn't really expect to still be alive might itself be pretty fraught.

bfgeek
0 replies
15h29m

the UAF-ish bugs are still bugs, and code poking at a GC-preserved object that the rest of the code doesn't really expect to still be alive might itself be pretty fraugh

For the LayoutObject heirarchy - the team doing that conversion added a NOT_DESTROYED() macro for this reason. It's gross, but was the least worst option.

As an aside - the performance of oilpan is broadly net positive now if you avoid some of the pitfalls. (The largest being a write into a Member<> requires a write-barrier). E.g. Things become trivially destructible, and no ref incrementing/decrementing, etc.

chc4
1 replies
1d2h

raw_ptr<T> is in fact a smart pointer wrapper that mitigates exploitation of most use-after-frees https://security.googleblog.com/2022/09/use-after-freedom-mi...

fractaled
0 replies
1d2h

It might be better to say raw_ptr<T> is a dumb pointer wrapper. raw_ptr is discouraged in favor of unique_ptr/WeakPtr/RefCounted; but it is used in places where you'd have a T* at rest.

Tuna-Fish
1 replies
1d3h

It's too bad that there's not a good way to have a broader way to switch between dialects within a project where in one place it's "this portion of the code is perf-critical and carefully reviewed" and in another "this portion of the code is perf-oblivious and has lots of async state that's easy to get wrong". I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear.

You are describing Rust's "unsafe" keyword.

And this kind of code is very literally the original impetus of it. The language was sort of originally designed to implement a browser, after all.

evmar
0 replies
16h8m

I have written a lot of Rust, and I am describing something different. Rust lets you write code that cares a lot about memory (with lifetimes etc.) with either compiler assistance or without. What I am describing is code that doesn't much care about memory and would be happy to sacrifice performance for it.

wongarsu
0 replies
1d3h

I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear.

Python, with perf critical sections written in C or Rust is pretty much that. From what I hear the Rust-Python bindings are especially good, and make the correctness part easier even in the performance critical parts.

Or you can go the opposite way and call out to a scripting language from your fast language. Today everyone is hyped about wasm for that, but we also have about two decades of computer games using lua for that (and games are probably the single biggest category of performance sensitive software)

20after4
10 replies
1d4h

This is actually really similar to something I've been wanting to build for a long time. In my case I've thought it would be useful to have a way to calculate the likelihood for a given change to break things based on the history of breaking changes in the same file or area of the file. Basically a riskiness score for each change. The risk score could be associated with each PR and would provide a signal for reviewers about which code should get a bit of extra attention as well as highlighting the risky changes when they are being deployed.

The tricky part would be tracking the same part of the code as it moves up and down because of insertions/deletions above it which would cause problems for a naive algorithm based on line numbers.

Just doing it at the file level, like this does, might be good enough to be useful though.

withinboredom
4 replies
1d4h

Not just the code itself, but the author. I worked with a guy that wrote at least one bug every time he created a PR.

dspillett
2 replies
1d3h

> wrote at least one bug every time he created a PR.

An economic hero. A man of the people. Creating job security for QA departments!

lpapez
1 replies
1d

A friend working in office of a big-tech company located in Denmark said "one bad engineer like me working in Copenhagen can put food on the table for 20 Bulgarians working in customer support".

Since that day I always wanted to get into FAANG-type companies, writing buggy code is basically philantropy.

hypothesis
0 replies
22h7m

This thread made some people question their no-bugs-allowed ideal, which is apparently a misanthropy…

jnwatson
0 replies
1d2h

"If debugging is the process of removing software bugs, then programming must be the process of putting them in."

debbiedowner
2 replies
1d3h

This has been my job for over 2 years now!

We do it on a symbol level after statically analyzing each change, and everything in the monorepo daily. Our remedy to high risk changes is to run more tests, client tests not unit tests. Sometimes there are 100k client tests to pick, so we rank them and run a small subset.

It is a hard problem. One interesting observation is that there is a culprit symbol or two in the culprit change, but its connectivity is very similar to non culprits in the same change.

Another observation is that the transitively modified callgraph after a change is pretty big, a depth of 50 is not unusual. It is hard to get many useful signals out of it beyond amount of overlap in transitively affected symbols between change and test.

We found file level and build target level to be too coarse, but AST symbols are working.

kosolam
1 replies
23h21m

100k client tests. Sounds like a lot. Is it integration tests or UI tests? How many tests overall have you got? I’m just curious

debbiedowner
0 replies
16h19m

We add support by project, and the prototypical project we started with had 1M test reverse dependencies, a quarter of that was eligible test targets that we could recommend (based on language written in). This is probably the biggest project that we would ever find to support in the monorepo.

Some are UI tests, but we don't recommend those, because we found they don't catch breakages as often so we don't support the language they're written in. The tests we recommend are often integration type tests in that they call very higher order functions and often many of them.

kevindamm
0 replies
1d3h

Code location (+) provenance/authorship (+) data flow analysis for adjacent sensitive code.

I would add it to my review tools fwiw.

bostonvaulter2
0 replies
21h46m

I'm currently reading a book about that topic! https://pragprog.com/titles/atcrime/your-code-as-a-crime-sce...

wizzard0
6 replies
1d8h

Impressive visualization! Also kind of keeps the vibe of the internal Google tooling, minimalistic and down-to-earth pragmatic

londons_explore
5 replies
1d8h

Google tooling of 2010, before everything needed to load 10 megabytes of javascript and had loading spinners for everything.

rfoo
3 replies
1d5h

Hey, but at least we have rounded corners now, which is nowhere to be seen in your 2010s <table>-heavy borgmaster summary /s

twodave
1 replies
1d4h

No, back then we would use images to make rounded corners. It was… as awful as it sounds…

ramses0
0 replies
1d3h

Triggered!

diggan
0 replies
1d5h

From the submission CSS file:

    details {
      margin: 5px 5px 5px 12px;
      border: 1px solid #aaa;
      border-radius: 4px;
      padding: 4px;
    }
Not even "designs from 2010" are safe from rounded corners.

(the border-radius CSS property was first proposed in 2002 [!] as far as I can tell: https://www.w3.org/TR/2002/WD-css3-border-20021107/)

rebane2001
0 replies
1d6h

I try to keep my projects simple, client-sided, and self-contained. The way this page is set up isn't perhaps the best practices for the modern web, but you can download just the html file and it'll probably still work in 30 years time.

keketi
6 replies
1d6h

How about a version where the monetary amounts are normalized by the number of lines of code?

Cthulhu_
4 replies
1d6h

Why do you ask? Genuinely curious, because lines of code is a moot point in software and security.

worldsayshi
0 replies
1d4h

I wouldn't say that it's a moot point in every context. The metric we'd get here would amount to "given that this developer made a loc change, what is the monetary risk involved".

Developers that are high on this metric might want to allow down and think twice next time they commit.

Or not. Metrics are likely not very useful in general.

phyzome
0 replies
1d

It's the same as the idea of per-capita normalization.

matsemann
0 replies
1d5h

Because it's interesting. If "X" is 1000 lines of code and has cost $20k in bounties, and "Y" is 1 000 000 lines of code and has also cost $20k in bounties, it's interesting to see that feature X has relatively more high profile bugs when it probably does much less.

dsabanin
0 replies
1d4h

It’s a valid measure of the amount of code. 10 bugs in 100k LOC speaks of a very different quality than 10 bugs in 1k LOC.

quickthrower2
0 replies
1d5h

Or normalize by number of word spilt on the bug (as a proxy for complexity)

londons_explore
3 replies
1d8h

I feel like this is 1 step away from running "blame" and associating dollar values with the security bugs caused by each engineer...

rfoo
1 replies
1d5h

... and I bet the top 1 engineer didn't actually add any bug to the code base, they just did a few repo-wide refactor.

arghwhat
0 replies
1d4h

"refactor, no functional changes"

Narrator: There were functional changes

consp
0 replies
1d6h

Time to get the svn praise back and do it sarcasticly.

treffer
2 replies
1d7h

Nice one, given that this probably went down to the diff level it would be interesting to see it weighted by LOC changed, e.g. 10 lines in file A and 1 line in file B would mean file A gets 1/11th of the money assigned because it was the majority of the bug.?

Or perhaps a LOC changed / file LOC based distribution. That would be how buggy each file is, with a $$$ tag.

initplus
1 replies
1d6h

This would show the effect you want. Code like tests are likely to be very verbose, while often the actual vulnerability will come down to a handful of characters.

treffer
0 replies
1d6h

Depends on which effect you are after.

I was thinking of ROI for reading code. Regularly having a one line issue in a 100 LOC file is very different from a two line issue in a 10'000 LOC file.

And yes, tests need to be excluded, of course. But looks like that's done?

DaleCurtis
2 replies
1d1h

Very cool! I think it's missing some entries though. I'm pretty sure we've had at least one in third_party/ffmpeg. Those fixes often land upstream first which might make tracking difficult.

rebane2001
1 replies
20h38m

It's using whatever Git Watcher comments on the monorail bugs.

DaleCurtis
0 replies
16h50m

IIRC, that bot used to be called bugdroid. I forget when it switched over, probably somewhere in 2020.

rei5
0 replies
21h27m

Nitpick: don't include DEPS, AUTHORS, or BUILD.gn files.

paulirish
0 replies
15h18m

I ported this to a treemap visualization[1]: https://vrp-treemap.surge.sh/

The treemap library was authored by evmar, chrome OG who's also in this thread.

layer8
0 replies
1d6h

It would be nice to also display the average reward per file on each node.

cr3ative
0 replies
1d8h

This is a very neat visualisation. Even if a little CPU intensive expanding areas! I'd hope the Chrome team has something similar internally, it looks really useful to understand the attack surface, as it were.

captn3m0
0 replies
1d7h

Such a cool idea, and great execution.

Edit: Is the raw data somewhere? A sunburst or tree map would be worth trying out

Sarkie
0 replies
1d8h

This is a really cool view on where to aim efforts