Nvidia solved a hard problem and their competitors all failed. I spent years with an AMD card growing progressively more annoyed at their self-inflicted apparent inability to multiply matrices on demand. Nvidia had nothing to do with their failure, unless they had some sort of high-level mole in AMD's driver teams.
Hitting the only successful company in a difficult field with legal assaults is not the obvious path to success. Nvidia has nothing close to a monopoly; there is no moat and they operate in a commodity space that will behave like a normal competitive market. It just happens that their competitors were wall-to-wall complacent about how quickly the hardware world can change and they stupidly thought graphics cards were for displaying graphics. It turns out graphics cards are for building AI and crypto markets.
The thing with monopoly laws is it doesn’t matter how you got into pole position. The point is that you’re illegally utilizing tangential benefits of the pole position to maintain that position as well as enter new markets.
The lock-in is probably a losing argument on technical merits unless NVIDIA is doing something nefarious there (which the lawsuit will potentially reveal). If nVidia is distributing chips in preferred ways to partners it wants to succeed (e.g. those partnered with MSFT which it has a very cozy and tight-knit relationship), that could be an anti trust thing.
So there is an argument to be made that MSFT is acutely self-aware its behavior could be viewed as engaging in antitrust. If there’s a law against structuring financial transactions, why isn’t similar behavior in the business world viewed similarly as trying to bypass the spirit of the law?
Just on that last paragraph, because the frame annoys me - that is bullying. It isn't fair to insinuate that a company did something wrong by attempting to obey the law.
MS knows all about antitrust. They've hired a bunch of lawyers and told them to do what needs to be done to be compliant with the regulations. That shouldn't then be implied as evidence that they are guilty of antitrust activities! What are the lawyers supposed to do, structure the company to maximise the likelihood of an antitrust lawsuit being bought? Nobody should want that, regulator, company or customer.
You ignore “the spirit of the law” with fancy acrobatics around “the letter of the law”… at your own peril.
I can’t comment on the claims here, but if MSFT is skirting the intent brazenly enough, the predictable regulatory reaction is no bullying. Everyone involved is well aware of that dynamic. No need to pity MSFT.
Is spirit of the law ever a concern in the USA? I’m asking because it is in the EU and every US company seem to struggle with the concept if I am to judge by how they engage GDPR, DMA and similar regulations in the EU.
The US Constitution has largely been interpreted in a “spirit of the law” manner since the founding. Originalism has taken significant durable gains in terms of SCOTUS appointments since the 1980s; prior to that originalists had a good chance of changing their POV once appointed. I suspect similar viewpoints are reflected in the judge appointments as well. Letter of the law benefits the powerful because they can always stay ahead of legislative attempts to fix the letter of the law (or lobby to change the letter). Spirit of the law is much harder to corrupt.
Spirit of the law isn’t perfect itself obviously because it is legitimate to point out that it becomes hard to know what the law actually is, especially since governments have gotten into legislating a lot of regulatory nuance and it’s hard to distinguish malicious compliance from good faith effort. It’s also corruptible from overzealous prosecution looking to make a name for itself by taking on unpopular entities that aren’t actually doing anything wrong.
This is precisely the opposite. "Spirit of the law" makes the rules squishy and indeterminate, providing opportunities for fancy lawyers to bend the result to their own interests.
"Letter of the law" often leads to harsh results when the law is drafted poorly, because if they wrote something dumb then you get something dumb instead of a judge rewriting the law to make people happy. But the people they're making happy are usually the powerful, so pick your poison.
No law can ever be written to capture every possible application of the underlying spirit. Thus your ability to escape the spirit is directly correlated with how many lawyers you can hire to find loopholes in the text (or just flat out lie).
It’s also important to remember that societies naturally undergo shifts over time. It’s impossible to continuously update a codified set of laws when the underlying moirés of the time have shifted; you’ll just be constantly arguing over the updates to add. Any law written perfectly today becomes imperfect simply through the passage of time. That’s why the Bible and any prescriptive religious text feels so outdated on many recommendations - it’s a snapshot in time of the values of a culture but those values change. There was even a fantastic sci-fi short story on this exact point of cultural shift [1] that’s worth a read.
[1] https://qntm.org/mmacevedo
The assumption here is that the rules would be complicated and provide lots of opportunities for gamesmanship. Now suppose the rule is "no company shall have more than 30% market share in any market, any that does shall be broken into no fewer than twelve independent pieces."
No loopholes, if you exceed 30% market share you get broken up. And if they find a loophole then you amend the law and take it out.
That's just politics. Somehow you need a process to decide what the law should be. The output of that process is the new law. If the output sucks then get a new process. But whether people can agree on what the law should be is a separate issue than whether we should even know what the law as enacted is supposed to mean.
That's fine, nobody is saying that you can't change the law if a case comes out in a bad way. But it should be the legislature rather than the courts to do it, and the new understanding shouldn't be applied to past behavior ex post facto.
Please clearly define the meaning of “you” in your simple law.
Literally me? Okay, I don’t own 50% of the market—my company does.
Oh you mean my company? Okay, my company doesn’t own 50% of the market—each of my companies only control 25%.
Oh you mean me and my companies? Okay, well I only own one company and my wife owns the other one.
Oh you mean… I think you get the point…
Now move on to all the other words you used: what defines a “market”? What does “broken” mean here? What does “independent” mean here? I’m sure it’s quite clear to you—that you know it when you see it. I’m also quite sure others have different interpretations.
I agree with the spirit of your comment otherwise, but simple laws rapidly become complex laws because people are complicated and language is flawed.
That isn't a loophole, it is an interpretation. It isn't possible to have laws without an assumption that a reasonable person is interpreting them (which is 95% of the reasons why judges are involved).
But we know that the interpretation phase has a lot of wiggle room (the US constitution is a regular parade of this sort of thing - the Roe v. Wade fight or the abuse of the commerce clause for example). When they go bad these things aren't loopholes as much as they are just ignoring the written law with a polite fiction and it is up to different interest groups to work out where the power lies to get what they want. Political reality vs. the theoretical rule of law ideal.
But that is necessarily independent of what is written in the law itself. If a group has enough power to overrule the written law then what you write in the law won't be able to stop them.
"no company shall have more than 30% market share in any market, any that does shall be broken into no fewer than twelve independent pieces."
It doesn't contain the word "you".
Then your company would be violating the law. "Company" means a set of entities that share a common ownership.
A set of products or services that serve as fungible substitutes for one another.
It means they no longer share common ownership. This is also what independent means.
But all of those things are just their ordinary meaning. Writing them down would make the law more explicit but it doesn't make it any more complicated. The definitions aren't each a separate set of criteria that have to be complied with separately, they're just a clarification to reduce possible ambiguity.
In particular, what you're doing is resolving edge cases. But the basic law has already addressed 99% of cases, because they're not ambiguous. An independent restaurant in a major city does not have >30% market share for food because there are many, many competitors. Microsoft has >30% market share for desktop operating systems because Microsoft has ~70% market share for desktop operating systems.
And we have to distinguish between two things here. One is, you see the word "company" and the dictionary says one of the meanings is a military unit, and then Microsoft claims that they aren't a company because they aren't a military unit. But if something has two meanings and one of them doesn't make sense in context, that's not the intended meaning. Using the "spirit of the law" for this kind of resolution is inherently necessary.
The other is, the law isn't ambiguous, but the unambiguous result is undesirable and the only way for a judge to fix it is to disregard the text and make something up. They shouldn't do this.
Seems like either system is exploitable by the rich.
The difference is this: If you use the letter of the law, an ordinary person has the same chance to find a loophole as the rich. Which will tend to cause them to get closed, because the rich have more money but ordinary people are more numerous, and the ability for ordinary people to exploit them would pressure the government to fix them.
Sometimes this would make the law more complicated, because the situation has intrinsic complexity and you have to enumerate the edge cases. Sometimes it would make the law simpler, because the existing complexity is extraneous and only an opportunity for gamesmanship. But either way it creates an evolutionary pressure for improvement.
Spirit of the law is more important in the US.
In common law systems (US, UK), there is greater emphasis on the spirit or intent behind the law. Judges have more flexibility to interpret laws and use general principles.
Civil law systems place more importance on the letter of the law. The spirit behind the law is secondary. The written legal codes are more important.
EU law is mixture of both, but most continental European countries have civil law system (Ireland does not, Nordic countries have mix of civil and common law)
Since the 1980s letter of the law has seen a remarkable upswing in power. I wouldn’t say spirit of the law applies anymore to SCOTUS rulings & I suspect federal appointments followed similar biases so it’s up & down the federal court. Not sure about state judges. Do you agree?
This is a confusing argument to me. Are you saying that structuring and smurfing should then be legal? All structuring and smurfing is is an attempt to make sure that financial transactions stay below the statutory reporting threshold.
Also, I think you’re conflating my statement which was meant to be prescriptive with a descriptive statement. I’m saying if structuring and smurfing are illegal in a financial context, MSFT doing so to skirt antitrust laws should be similarly illegal - if the laws don’t prohibit it, the laws should be updated. But I’m not a lawyer so it’s possible laws already prohibit it or MSFT violated some laws. Commenting in either direction on an information free article isn’t wise.
As for the spirit of the law vs letter of the law, that’s an ongoing debate as old as history. It’s personally weird to me to encounter letter of the law people given that the spirit of the law has a much richer history behind it to my view and seems more defensible. Some amount of rule lawyering is required, but violating the spirit of the law is a more robust legal principle that can withhold participants trying to find creative workarounds around the spirit of the law.
Ah I see, I was in two minds about including the quote but decided against it. I was talking about "Microsoft structured its minority stake in OpenAI in part to avoid antitrust scrutiny, The Times has reported. Microsoft also reached a deal in March to hire most of the staff of Inflection AI, another A.I. start-up, and license its technology. Because the deal was not a standard acquisition, it may be harder for regulators to scrutinize.", not what you wrote. Obviously they structure the deal to involve scrutiny; they aren't going to structure it to invite scrutiny; the deal is designed to avoid anything that would allow a random regulator to get involved. The Times is insinuating without cause.
I don't think you're in a position to bully Microsoft. I doubt they care about your opinion.
If you need to intentionally structure a deal to avoid scrutiny and have it described that way, it implies what you really wanted was much worse.
“We packed the goods in nondescript boxes to avoid scrutiny by the border guards…”
Scrutiny isn’t something you need to avoid as a big company if you’re not fucking around. The lawyers are there to handle the scrutiny, not to hide stuff from it.
It doesn't. That is like saying that if someone structures their affairs to pay less taxes it implies that wanted to do something illegal. They had to choose some structure, they picked the safest and easiest one. In both the hypothetical tax case and the real MS case, I suspect.
The incentive is to choose structures that minimise regulatory engagement. The fact that MS is following incentives isn't a red flag. Not a green one either, it isn't interesting except as something that the NYTimes can hook insinuations on.
This is a funny discussion, because "structuring" has a very specific meaning when it comes to finance. It's about executing your financial transactions in a way that avoids regulatory scrutiny (for example sizing transactions such that they fall just under reporting thresholds).
Structuring is explicitly illegal in the US.
This is the point that vlovich123 was making, and given you totally missed it, it seems like you might not be familiar with what the term means in this context.
Sorry for being unclear on this, I should have quoted. But I'm not, and never was, talking about vlovich123's point in any comment in this thread. I didn't have an opinion on what vlovich123 said.
It does. You can't replace the X and Y in "Structuring X to avoid Y" and still end up with the same implication.
Structuring your business to avoid breaking the law is good.
Structuring your business to avoid some otherwise regular audits that would reveal whether you are breaking the law is obviously suspicious. One might think that there is a reason why getting audited might result in negative consequences for your business.
This is an easy asnwer.
With banking, structuring laws mean banks need to report any transaction over $10,000.
So if I need to send you $100,000 and s split it up into 11 payments so each payment comes under the $10,000 reporting limit that breaks the law because my payment was for $100,000 but i structured it to get around the $10,000 reporting limit.
If Microsoft approaches anit trust limits if it owns more than 50% of a company and buys 49% then no strucuturing took place as they only bought 49%. Now if they bough 10% in each of 6 different shell companies that they own such that they'd control 60% of hte company, then this is illegal as it works around the limits.
But owning up to the limit is perfectly fine as it doesn't in anyway break the law or work around it. That's the very reason for having a limit, to say you can go up to this limit, but not over.
I'm guessing you don't deal in finance at all as we deal with this all the time. You'll see funds owning up to 9.9% of companies to avoid the 10% reporting threshold. Again, nothing illegal here as the government has said they are perfectly fine with funds owning up to <10% of a company.
I think the issue here is the overloaded use of the term "structuring."
If I think I might be doing something that antitrust regulators won't like, so I hire a lawyer to tell me what I can do, and they tell me "you can buy up to 49.9% of this company," and I buy 49.9% of the company, that's fine. It's neither illegal nor unethical. It's also not "structuring" in the technical, financial sense, but a layman might say "you structured the deal to avoid falling afoul of regulation," even though it's the wrong use of the word. I think that's what's happening in this conversation (I have no idea about what MSFT actually did or didn't do, just the direction the conversation here is going).
You picked a particularly bad example. Structuring is a notoriously bad law because innocent people make transactions that would be considered structuring all the time, and making it illegal has no legitimate purpose when the government could just use the aggregate amount of deposits over some period of time for its reporting threshold instead of the amount of each individual deposit. It's like creating a rule with a loophole that gets exercised through ordinary behavior, and then instead of closing the loophole they impose criminal penalties on the ordinary behavior.
But then you consider that it's used for civil asset forfeiture (i.e. law enforcement stealing money from innocent people without ever proving a real crime) and it suddenly makes sense:
https://www.washingtonpost.com/news/the-watch/wp/2014/03/24/...
Whereas with MSFT it's a different situation. There is nothing illegal or wrongful about depositing a large amount of money, there is just a heinous law that makes it illegal to do it in a common way. Whereas monopolizing a market is intended to be illegal regardless of how you do it, so there shouldn't be a way to structure things to monopolize the market without breaking the law. The way you're intended to comply with the law is by not monopolizing the market.
Because laws are always flawless and never contain any loopholes, and it has never happened ever that lawyers were specifically instructed by a company to find those loopholes and exploit them as hard as possible.
The idea of law as having loopholes is relevant for politicians, who are tasked with ensuring that the written law reflects their intent. It's not relevant for courts, regulators, people or companies, who are all tasked with following the law as actually written. For those people there is no "spirit" or "true law".
The reason this is important is it's a sword that cuts both ways. It's always tempting to argue that whatever you'd personally like to happen is what politicians really intended, and any gap between reality and their preferred outcome is therefore a "loophole". But once you get into saying people should follow intent, not written law, others can easily argue that politicians never intended the law to be interpreted like that against them, and so therefore they are morally justified in ignoring it. It can be used against you as easily as you can use it against them.
A common example of this problem is income vs capital gains taxes. One ideological tribe is very fond of arguing that people who have income mostly from investments rather than wages are exploiting a "loophole" in tax law, but of course the reason there are different rates to begin with is exactly because politicians wanted to encourage investment. There's plenty of cases where this intent is discussed in written literature and there's no other reason to distinguish between income sources then set differing rates. There is no "loophole" and nor is the "spirit" of the law being violated. But you hear such claims all the time.
The other reason it's problematic is because you can't really know what the intent of lawmakers was. The law was their best collective effort at writing down what they wanted, as a result of numerous compromises and disagreements between different people. If they didn't write it down properly or the resulting compromise was a mess, that's on them, but a working legal system doesn't allow people to just blow off their written instructions and assume they know what was really meant.
Actually, there is. Many judges take various interpretations of the 'spirit' into account. See eg https://en.wikipedia.org/wiki/Originalism for one example.
And if the courts use some 'spirit' guide them, companies and their lawyers better try and predict what that 'spirit' recommends.
Well, then by that definition the real life legal system of the US ain't working?
Originalism is the opposite of that, no? Originalists are all about following what was actually written down, using an understanding of English as it was used at the time. It rejects modern re-interpretations of the law and avoids speculating about the 'spirit' of the constitution, even if that would be more convenient. The competing view is that of a 'living' body of law in which the meaning continually changes without the wording itself changing.
Well, either way: than the 'living body of law' theory is an example of such a 'spirit'.
It is and you'll notice a lot of people really, really hate it. Trump just broke all fundraising records by miles, because a lot of people who previously didn't support him feel that New York has ignored the written letter of the law in favor of a tribalistic "spirit". There are similar society-rending controversies happening in Europe due to the activist lawmaking of the ECHR.
These two perspectives aren't both equally valid: the courts are not allowed under any system of civics to simply do whatever the judge feels like. They are only given leeway to interpret the law when they have no other choice because the statutes are unclear. It's a last resort, and often the result will be people "getting away with it" because there's no law against what they did.
Yes, I'm aware of that distinction. However I read the comment I was replying to so that they suggested the antitrust case is unfounded because Nvidia didn't break any laws. It's precisely why the antitrust case is valid/needed, because if Nvidia were breaking any laws, you wouldn't need antitrust, you'd just take them to court.
Antitrust is law and court.
The following is for US law.
Your first sentence is not really needed because US monopoly law really doesn't care if you are actually in pole position. It's more about monopolization than it is about monopoly.
To a first approximation think of it as being about fair competition. You could have a complete 100% monopoly in some particular market but if you got that monopoly by simply outcompeting everyone else by making a better product and you were not trying to use that to expand into other markets by doing things like tying you would probably not have an antitrust problem.
This depends on jurisdiction, and what judges you get.
Eg Standard Oil was slapped with anti-trust law, despite neither having a monopoly nor exploiting its market share like a monopoly. See eg https://en.wikipedia.org/wiki/Standard_Oil#Legacy_and_critic... or https://fee.org/articles/the-myth-that-standard-oil-was-a-pr...
Cory Doctorow argues well why it doesn't matter how you got into a monopoly position: https://pluralistic.net/2024/05/18/market-discipline/#too-bi...
Still have no idea how this dude is relevant other than he wrote some scifi books one time. What are his technical accomplishments. Oh, nothing?
Anyone can talk, especially nowdays because of AI, ironically.
I think he is relevant because he writes stuff that resonates. You might disagree, but I am judging him based on his very real accomplishments.
If his ideas resonate and are so profound, tell us what they are and why. Don't point to his blog and worship him like some minor deity.
How does one worship a minor deity? You're not obliged to read what I post any more than I'm obliged to persuade you. Feel free to ignore the links, or feel free to read them. You might gain something from them as I have, but I'm not going to expend any efforts on trying to enlighten you or any such nonsense.
I didnt read what you said because you didn't say anything. You just linked to his blog while carefully making sure to point out it was CD who wrote it.
That's literally dictionary definition of idolatry.
Idolatry: noun; the worship of idols.
It has a decent number of up votes, so clearly some people think it worth posting. As I said, you don't have to read it, and I don't really care if you do. I'm not sure how linking to something with a reference is worship, but ok.
You seem very angry.
I'm not angry, I'm simply pointing out that you place undue value on a name the vast majority of people don't care about.
What you're doing, explicitly, is placing more value on name than idea. A fallacy. I'm obviously not saying "worship of a minor deity" literally, but colloquially.
You're right, I can read it or not. Same with you, you can take valid criticism or ignore it and be all the poorer for it.
You can also continue admitting you have no rebuttal as you hyper-focus on the _diction of the conversation_ instead of the _conversation_, like how to define worship and idolatry in a casual conversation.
You have no criticism. You object to the fact that I linked to an article I liked. You think I did that because I assign undue weight to the author. I did not. I referenced the author because he is well known and it allows others to know what they are linking to. The fact you assert I did it to invoke some kind of authority is flat out wrong. I didn't summarise because the linked article is, in my opinion, a very well argued article - a summary is only going to be a poor facsimile, so better to link to the article itself.
Generally discourse would be better if we linked to well argued and reasoned articles than have a load of blow hards that like the sound of their own voice rather too much.
If you're going to say things like "literally the dictionary definition of idolatry" when it's nothing of the sort, expect people to pick up on it. Your reasoning appears to be very much in the domain of a cheese.
I shall no longer be replying to or monitoring this thread. Take care.
Interesting, I get it now. Just one question, how many times a day do you pray to your doctorow shrine?
Writing well is a rare and valuable skill.
And this article, especially, really put into words what I had been thinking for 2 decades:
https://www.wired.com/story/tiktok-platforms-cory-doctorow/
Many people like his writing, and his political stances.
Many of us appreciate the work he’s put in with the EFF. When you are right about things over-and-over for 20 or 30 years, people tend to listen to you.
Why are you attacking the author instead of addressing the message?
why is any journalist, blogger, or critic important? They are popular the vast majority of the time, and have earned it via their blog/podcast/magazine articles. It's not because they're the best SME in the field or even ever practiced in the field, it's "can they bring things to light?" for the masses, and he often does.
Honestly I don't get how the current situation could develop.
Video games have used compute shaders forever, and those work just fine on other cards.
I remember doing some quite involved stuff for my thesis using OpenCL a decade ago, and it worked just fine. Nowadays OpenCL is dead for some reason...
I just don't get what is there in CUDA that makes it so special. As far as I remember, a GPGPU API consists of a shader language, a way to copy stuff to and from the GPU, a way of scheduling and synchronizing work, atomics, groupshared variables, and some interop stuff.
Is the vendor lock-in due to CUDA libs? In that case, the problem is not CUDA, but the libraries themselves. Not sure about today, but performance-portability basically didn't exist back then. You needed to do specialize your code for the GPU arch at hand. It didn't even exist between different generations of cards by the same vendor. Even if you could run CUDA code on AMD, it would be slow, so you need to do a rewrite anyways.
There probably isn't anything in CUDA that makes it special. They are well optimised math libraries and the math for most of the important stuff is somewhat trivial. AI seems to be >80% matrix multiplication - well optimised BLAS is tricky to implement, but even a bad implementation would see all the major libraries support AMD.
The vendor "lock in" is because it takes a few years for decisions to be expressed in marketable silicon and literally only Nvidia was trying to be in the market 5 years ago. I've seen a lot of AMD cards that just crashed when used for anything outside OpenGL. I had a bunch of AI related projects die back in 2019 because initialising OpenCL crashed the drivers. If you believe the official docs everything would work fine. Great card except for the fact that compute didn't work.
At the time I thought it was maybe just me. After seeing geohotz's saga trying to make tinygrad work on AMD cards and having a feel for how badly unsupported AMD hardware is by the machine learning community, it makes a lot of sense to me that it is a systemic issue and AMD didn't have any corporate sense of urgency about fixing those problems.
Maybe there is something magic in CUDA, but if there is it is probably either their memory management model or something quite technical like that. Not the API.
What do you think about SYCL as a viable cross-platform GPU API?
I'd have been happy to use OpenBLAS if it worked on a GPU. Any API is good enough for me. I have yet to see anything in the machine learning world that required real complexity, the pain seems to be in figuring out black box data, models and decyphering what people actually did to get their research results.
The problem I had with my AMD card was that SYCL, like every other API, will involve making calls to AMD's kernel drivers and firmware that would crash the program or the computer (the crash was inevitable, but how it happened depended on circumstances).
The AMD drivers themselves are actually pretty good overall, if you want a desktop graphics card for linux I recommend AMD. Open source drivers have a noticeably higher average quality than the binary stuff Nvidia puts out. Rock solid most of the time. But for anything involving OpenCL, ROCm or friends I had a very rough experience. It didn't matter what, because the calls eventually end up going through the kernel and whatever the root problem is lives somewhere around there.
The biggest problem with SyCL is that AMD doesn’t want to back a horse that they don’t control (same reason they opposed streamline) so they won’t support it. When the #2 player in a 2-player market won’t play ball, you don’t have a standard.
Beyond that, AMD’s implementation is broken.
Same as Vulkan Compute - SPIR-V could be cool but it’s broken on AMD hardware, and AMD institutionally opposes hitching their horse to a wagon they didn't invent themselves.
This is why people keep saying that NVIDIA isn't acting anticompetitively. They're not, it's the Steam/Valve situation where their opponents are just intent on constantly shooting themselves in the head while NVIDIA carries merrily on along getting their work done.
It's crazy, because even 10 years ago it was already obvious that machine learning was big and is only going to become more important. AlphaGo vs Lee Sedol happened in 2016. Computer vision was making big strides.
5 years ago, large language model hadn't really arrived on the scene yet, at least as impressively as today, but I think eg Google was already using machine learning for Google Translate?
Goes to show how difficult and important execution is. There was also Hangouts vs Zoom.
Or Skype vs Zoom. Skype had a lot of mindshare, it was basically synonymous with calling people via the internet for a while.
But somehow Zoom overtook them during the pandemic.
The magic is that CUDA actually works well. There is no reason to pick OpenCL, ROCm, Sycl or others if you get a 10x better developer experience with CUDA.
Writing and porting kernels between different GPU paradigms is relatively trivial, that's not the issue (although I find the code much clunkier in everything other than CUDA). The problem is that the compiler toolchains and GPU accelerated libraries for FFT, BLAS, DNN, etc. which come bundled with CUDA are pretty terrible or non-existent for everything else, and the competitors are so far away from having a good answer to this. Intel have perhaps come closest with OneAPI but that can't target anything other than NVidia cards anyway, so it's a moot point.
I'm not very familiar with this area, so this is a useful insight. Otherwise I was really confused what's so special about CUDA.
I don't know specifically about OpenCL, but the big thing about CUDA is that it's supported across operating systems and across the whole spectrum of NVidia GPUs. AMD has ROCm, but they have to be dragged kicking and screaming to support any of the consumer grade GPUs. ROCm is still pretty much Linux only, with limited support on Windows and only the high-end consumer offerings (7900XT and 7900XTX) are officially supported for ROCm. Because they want to push compute users (and AI users) into workstation grade GPUs. In comparison, CUDA is usable and supported even on mid range NVidia GPUs such as RTX 3060. And most of the software around things like AI is done by enthusiasts with gaming PCs, so they write their software for the interface that supports them.
I have an AMD card that I try to use for AI stuff and it's an uphill battle. The most popular workloads such as Stable Diffusion or Llama more or less work. But as soon as you go into other, less mainstream workloads it starts to fall apart. Projects install CUDA versions of torch by default. To make them work you have to uninstall them manually and install the ROCM versions of torch. Then it turns out some Python dependency also uses CUDA, so you also have to find a ROCM fork for that dependency. Then there's some other dependency which only has a CUDA and CPU version and you're stuck.
I think a lot of it is customer service. If you work in ML you probably have some NVIDIA engineer's number in your phone. NVIDIA is really good at reaching out and making sure their products work for you. I am not a Torch maintainer but I am sure they have multiple NVIDIA engineers on speed dial that can debug complicated problems gratis.
Where does SYCL fit into this picture, is it a viable replacement for cross-platform GPU access?
Yep. AMD and Intel had two decades to come up with a CUDA alternative or even a parallel implementation and both failed to do anything. Incompetence.
The alternative to CUDA is called OpenCL. There has been speculations that poor performance of NVIDIA GPUs with OpenCL compared to CUDA is an intentional anticompetitive practice, but I don't feel confident to tell for sure.
OpenCL is an alternative to CUDA just like Legos are an alternative to bricks. The problem with OpenCL isn't even the performance, it's everything. If OpenCL were any good, people could use it to build similarly powerful applications on cheaper AMD GPUs.
OpenCL is just a spec. It's up to companies to implement it in a successful way or not. There is no reason in and of itself that OpenCL can't compete with CUDA on performance. The fact that Apple's Metal, which is pretty good, is actually implemented with a private OpenCL system is proof that the spec is not to blame.
I don't understand. If OpenCL was so good, why did Apple create Metal instead of just using OpenCL?
For similar reasons why Microsoft created DirectX. It allows them to have a system where the software and hardware are more tightly integrated than using a cross-platform spec. It also allows them to situate the API within the context of the operating system and other languages that are used on Apple platforms, making things easier on developers. And at least in that regard, they certainly succeeded. Metal is probably the easiest GPU API offered on any platform. Not necessarily the most powerful, but it’s almost trivial to hand write a compute kernel and spin it off.
Well, performance isn't the issue. Like the parent said, the problem is mostly that CUDA is such a radically mature API and everything else isn't. You might be able to reimplement all the PCIe operations using OpenCL, but how long would that take and who would sponsor the work? Nvidia simply does it, because they know the work is valuable to their customers and will add to their vertical integration.
OpenCL isn't bad, and I'd love to see it get to the point where it competes with CUDA as originally intended. The peaceable sentiment of "let's work together to kill the big bad demon" seems to be dead today, though. Everyone would rather sell their own CUDA-killer than work together to defeat it.
AMD’s first attempt at displacing CUDA-as-a-runtime was called AMD APP (advanced parallel processing) so this layer did indeed exist. It just sucked as badly as the rest of AMD’s slop.
https://john.cs.olemiss.edu/heroes/papers/AMD_OpenCL_Program...
Bonus points: the rest of the software libraries intended to compete with the CUDA ecosystem are still online in the “HSA Toolkit” GitHub repo. Here’s their counterpart to the Thrust library (last updated 10 years ago):
https://github.com/HSA-Libraries/Bolt
Nvidia had multiple updates in the last year the last time I checked. That’s the problem.
Link seems to have gone dead since earlier, lol, but:
https://en.wikipedia.org/wiki/AMD_APP_SDK
And much more importantly for ooencl's success, poor compatibility of OpenCL implementations on Nvidia hardware by Nvidia + Microsoft platform-level software
Eg, opencl on Nvidia GPUs for windows is/was missing (I have a many-year GitHub issue with them + Microsoft), which matters for individuals, and Nvidia does not support OpenCL for its core convenience analytics libraries like the RAPIDS Python ecosystem, which is core to its massive data center market. We initially built for compatibility/distribution, but it didn't matter: That gap closed a lot of doors for us as a small ISV choosing what & how to build, and in turn, ultimately prevents our customers from buying AMD, Intel, etc
I'm not up to snuff on whether that qualifies as using its position anti-competitively anywhere, but it's a real market issue
The horrible/non-support of openCL is why I thoroughly support the government looking into Nvidia's monopoly practices.
That theory doesn't really make sense. Nvidia didn't need AMD's permission to write a high-quality compute interface. Why would AMD need Nvidia's? AMD has their own internal opinions for how well they want compute to work; if OpenCL isn't doing it for them they should have figured that out quickly and then built their own layer as a stopgap. But we've seen what happens when they do that; ROCm. The problem is that they don't really know how and it is a glaring capability gap (didn't know I suppose - I expect they're learning fast).
If anything, the situation with OpenCL suggests AMD and friends were the ones dragging their feet. Nvidia correctly identified that being led by the OpenCL committee would lock them out of billions (trillions, if you believe the stock market) of profit and routed around the blockage rather than compromise their engineering standards.
How do we know the strategy wasn’t to avoid releasing a CUDA alternative until NVIDIA is wounded by an anti-trust lawsuit and then when they get broken up deploy your CUDA alternative and speed past them? Could be playing the long game.
That’s not the case at all. AMD and Intel could be making money hand over fist right now if they had products that worked with the ML ecosystem. If they had any magic bullet they’d be idiots to not have used it years ago, they’re missing out on tens of billions of dollars on purpose for what? Even if NVIDIA got broken up their cards would still have massive demand and profit margins. I don’t think you understand how much of a shortage of compute there currently is, anyone who isn’t blatantly incompetent or ignoring ML can make money. AMD and Intel are just fat corporations who didn’t see the potential and thought business as usual would stay number one. Now they’re standing on the sidelines with their dicks in their hands while Jensen is leading the parade.
Some companies such as standard oil had extraordinarily effective vertically integrated monopolies. The issues came into play when they used their market dominance to secure things like rebate structures with rail roads and made agreements to buy up transit capacity at very favorable rates that left competitors at a serious disadvantage.
A real question might be how Nvidia prioritizes capacity and delivery and pricing to OpenAI and Microsoft for their superior product that has no meaningful competitor. It’s not an issue that Nvidia, like standard oil, had a superior business and product. It’s an issue when their scale and the scale of their adjacent partners lead to agreements that strangle others that need access to the supply chain. For instance if Anthropic is at a disadvantage to open ai for Nvidia capacity due to agreements between them that are mutually beneficial to the exclusion of Anthropic, that’s anti competitive.
Another point with nvidia isn't "better product" but "what API does everyone use" and that's CUDA. Owned by NVIDIA. It would be like if SGI, who invented openGL as an open source version of their proprietary Iris GL, kept it proprietary, and now everyone uses SGI hardware because all games and 3D software use Iris GL. In this scenario SGI may also be very keen to make sure Iris GL has quirks that strongly tie it to SGI-only hardware to keep dominance. That seems a little more "okay you're overleveraging your position" versus "you put out a better product in a fair market"
One could very easily argue this is the status-quo; Microsoft ships DirectX which they withhold from competitors, and Apple does the same with Metal. Both of those monopolize their respective markets (PC gaming and mobile gaming) yet we see zero attempts to reconcile the two.
Because of the perceived hostility between vendors, I would not be surprised to see Nvidia argue that they're already as open as they could possibly be.
Except this isn’t actually even true on the surface.
Apple doesn’t have a monopoly on mobile gaming or even mobile. Windows is more of a monopoly than iOS, but it’s not a monopoly in gaming platforms by a large margin. OpenGL is a totally viable alternative. Etc.
CUDA is definitely a big part of the lock Nvidia has on their market. Other companies could with investment out class then hardware wise. But because of their API lock there’s no point.
Not really, given that Apple platforms have more or less entirely depreciated the featureset with no guarantee or expectation to keep it around.
I don't even believe you. Who else can compete with Mellanox networking and custom ARM v9 cores? Even Apple doesn't really ship anything worth comparing to Nvidia's server offerings, and they're arguably best-positioned to usurp Nvidia's TSMC friendship.
Standard oil wasn't a monopoly.
See https://en.wikipedia.org/wiki/Standard_Oil#Legacy_and_critic...
They even lost market share during the time they were alleged of having and exploiting a monopoly:
See https://fee.org/articles/the-myth-that-standard-oil-was-a-pr... for a more opinionated take.
Bro...
?
If you read the article rather than selectively quoting antitrust movement on standard oil started in 1880s and culminated in 1911. Likewise Netscape vs Microsoft concluded long after Netscape was very dead. The fact litigation and stuff takes a long time to unfold is not a material fact in such a case. A large part of standard oils erosion of market share was an attempt to prop up their case against breakup as well as headwinds against them due to the governments pursuit.
But it’s a bizarre misreading to say they were not a monopoly to say that by the time things hit the fan they had lost a lot of market share from their peak monopoly.
The specific issues that standard oil hit up against was their deals with railroads that solely benefited them by virtue of their scale and their ability to effectively dictate their own pricing for logistics. Likewise they would buy up all capacity on crucial lines for competitors and only partially use it purely to block market access for competitors. There is also a view that being incredibly efficient can reduce competition in markets adjacent to you. Success in business for an individual company isn’t the only goal of a capitalist state, but rather competition in markets is considered a goal of the state in itself - survival of the fittest implies there’s something to survive against, and an ever expanding monopoly growing into new markets and dominating them starts to retard innovation and competition systemically. You don’t have to agree with the thesis, but that’s the way the system works.
Why is it illegal to choose who you sell to? That's not bullying their competitors out of their own market.
If they wanted to, Nvidia could pick winners and losers in AI by withholding GPUs. That would be using their power in one market to influence another market. I haven't seen much evidence of that though.
Except as there’s a backlog if the enter into collusive agreements as a monopoly they are impacting adjacent markets (AI and adjacent companies competing with open ai and Microsoft). Also bulk pricing agreements can be seen as collusive even without inventory scarcity because it structurally advantages a picked winner.
When you hold a monopoly the same rules don’t apply. That’s why it’s often advantageous to allow material competition in your own market because your adjacency influence can be outsized and cause unintended consequences -even if you normally would have been allowed to do such things-. Those unintended consequences don’t require malicious intent they just have to exist and be materially a result of your monopoly. The more your maliciously collude though the worse the remedy will be for you.
Say that Nvidia partners with open AI because they think open AI is the best positioned to succeed (and hence result in increase demand for Nvidia) as opposed to if they got some direct financial kickback, it that legal?
Probably yes if Nvidia has a monopoly. Because competitors of open ai have no alternatives they’ve killed competition. If doing so is to their advantage that doesn’t make it better, that just makes it rational from their perspective. But regulation against anticompetitive behavior targets behavior that structurally kills competition. Most collusion isn’t direct financial transfers but cooperative agreements to dominate adjacent markets for mutual benefit.
I wonder why Jensen wouldn't help his cousin Lisa Su with a good advice :)
In general it isn't surprising that hardware companies like Intel and AMD fall at software - just talk to any programmer at such a hardware shop. What striking is that Jensen is able to have such a prominent and huge software development in a naturally hardware company. NVDA is the only hardware company where my programmer acquaintances don't complain about it being a hardware company.
A lot of what keeps Intel relevant is their prowess in both hardware and software, bringing properly refined products to market when it matters. Intel's failure against Nvidia had far more to do with their business strategy of CPU First missing the broadside of a barn; it's like bringing a knife to a gunfight, of course you're going to get your ass wrecked.
AMD fucking sucks at software though, no disagreement there.
Intel has some great software like MKL and related libraries, but, speaking from experience, due to its internal politics Intel is unable to innovate in a meaningful way or build a reasonable SW strategy for GPUs.
The oneAPI heavily contributed to their GPU failures IMO. Choosing SYCL and promoting non-existing performance portability and pouring resources in the trying to shove CPU and GPU (and at some point FPGA!) into the same programming model while disinvesting from OpenCL was a big mistake, but it was probably the only way to please both CPU and GPU management.
And the problem is not SYCL per se but the fact that they try to make it an open standard rather then trying to provide a way to extract maximum performance from their HW. They needed to build a CUDA alternative, where, after jumping through considerable hoops, you can get to actually programming tensor cores for those who need it. Not surprisingly, Intel is not using SYCL for their GPU kernels (look at oneDNN sources). With Raja gone there may be hope, but I'm not holding my breath.
The only thing that they have that is competitive is Habana's stuff which is outside of the oneAPI.
I would not be surprised if AMD has a similar issue of infighting due selling both CPUs and GPUs.
And NVIDIA does not have this problem.
What are your thoughts on the move from ICC to the LLVM-based ICX? Does that fit in with this product strategy somehow?
You're pretty close, but it's one level of abstraction away from the truth: NVidia spends absolutely earth-shakingly stonking amounts of money working directly with the devs of major studios, fixing issues with their drivers as they come, and in some cases paying studios to lock-in talking to them about the bugs they find.
It's why (and how) they release new driver patches for every AAA release, and part of the reason why it triggers a redownload of the shader cache on a whole slew of Steam games every major version release.
AMD and intel don't have a chance; NVidia is inside the design loop for game development far more thoroughly than other hardware vendors.
Of course AMD and Intel could just do the same thing. But they don't. That's not Nvidia's fault.
In fact I wish Intel and AMD would do the same! It's great customer service to know that games are going to work well out of the box with Nvidia, because Nvidia spent tons of money working directly with devs on optimization and even per-game bugfixes.
As a longtime PC gamer I wish AMD released better GPUs, but they seem incapable of doing so generation after generation. Intel is getting better at the low end, so there's at least some hope there. But Nvidia is just the best in class, because they put in a huge amount of polish and work.
My understanding is this is the same approach Nvidia took with CUDA: investing a massive amount to make it fast, accurate, and broadly available, while AMD continued to push out barely-tested drivers — for example, see hotz's woes with tinygrad and finally giving up and labeling the "red" (aka AMD) tinybox driver quality as "Mediocre" on their product page, with the "green" (aka Nvidia) tinybox driver quality listed as "Great" https://tinygrad.org/ — and to this day restricts their competing library (ROCm) to specific ultra-expensive server-class GPUs.
Intel at least works very closely with Microsoft to iron out the details for Windows.
You're treating AMD and Intel as resource-poor upstarts, which is ridiculous. They have every opportunity to provide the same support as nvidia.
Isn't the CUDA API a fairly significant moat?
I don't think so, no. I had an AMD card; I didn't feel like the lack of CUDA was slowing me down. I managed to implement everything I tried on the CPU just fine. My problem was that every approach I tried to implement my own stuff on the GPU using things like Vulkan or OpenCL led to crashes or frustration. In fairness I didn't know how to do basic operations on a graphics card with OpenGL and would be relying on super basic programming material out of AMD with titles like "how to multiply a matrix on an AMD GPU" [0]. They can be forgiven for not supporting amateurs I suppose ... but now I own a Nvidia card and they have stuff like https://docs.nvidia.com/cuda/cublasdx/introduction1.html.
But I stress that the issue there isn't that CUDA has tutorials. It is that I expect the tutorial code not to lead to a crash or system lockup. There is a well supported path to do those basic operations.
If you look at the CUDA webpage [1] you see things like "cuRAND", "cuFFT" and "cuBLAS". That isn't a moat - those are first year software engineering topics. AMD managed to make it look like a moat by not taking the compute market seriously; and anyone else competes at highly parallel compute.
[0] As far as I can tell; doesn't exist.
[1] https://developer.nvidia.com/cuda-zone
You and a few others have said that OpenCL is buggy on AMD. Is it buggy on Nvidia too?
I don't know. I'd use CUDA rather than experimenting with it; CUDA seems to be where all the investment is and my experience with OpenCL on AMD was so bad I don't feel the cross-platform argument would make any sense.
It is a bit of a chicken-egg problem for me. I couldn't make OpenCL work on an AMD GPU, so I didn't manage to learn that much about OpenCL. At the time I assumed it was just me, but in hindsight I never saw an OpenCL-based approach to a compute problem that worked reliably on my machine so maybe it wasn't.
But I don't think it really matters. The algorithms in the field don't seem to be hard and I never felt like I was struggling when implementing them on the CPU without any special API at all. My issues were conceptually similar to George Hotz's famous rants where he had crashes when running the demo app in a loop. In the experimenting phase I found I couldn't run code on the GPU with any API.
I'm sure the situation improved and part of it was just me; towards the end of my time with AMD I could run stable diffusion inference and it'd work great for 10-40 minutes before the kernel paniced or whatever - so it was definitely technically possible to get a "hello world++" style thing running. But I never felt it was the APIs that were holding me back.
It is if success is defined as deliberate failure.
Personally, a lot of US government actions of the past few decades in hindsight do not look like they had the interests of America or Americans in mind.
Can you name a few examples please?
I wonder if anyone connected to this is short NVDA.
Sure, until you consider the massive difference in accessible capital vs their rivals. Sure, VCs can fund startups, but Nvidia is publicly traded, which is just investment of a different form, and largely a form that you are barely accountable to. And that's on top of all the capital they directly control. That's an incredibly advantageous position to be in, and one that would be too tempting for most to resist exploiting.
AMD actually had more resources than NVIDIA for a good long while (although they did have more places to spread them over, that was their choice to do so), and the balance of power didn't really shift until the 2014-era when AMD really "took their foot off the gas" on R&D. 290X was the last time AMD could be described as "in the lead" and Maxwell opened up a gap that AMD never managed to close.
Raja actually specifically called out 2012 as being the timeframe when that happened, because AMD decided the future of GPUs was iGPUs and not dGPUs, that the latter segment was going to go away entirely as iGPUs got faster and faster. Oops.
https://youtu.be/590h3XIUfHg?t=1956
It's interesting that Jensen seems to be selling this "software development is dead to AI" story when that means his profitability moat (software) is dead as well. All AMD needs to do is "hey Copilot rewrite this CUDA code to run on AMD hardware". Somehow that's not happening.
AMD had 10 years to get competent on software. It's not happening.
Isn't it one of the fundamental applications of a GPU, whether for ML compute or more commonly for 3D graphics, to multiple matrices? In what way did AMD GPUs fail to do this?
He means big matrices.
And when you try to do compute on AMD GPUs they crash your kernel if you look at them wrong.
It's a bit ironic to talk about lawsuits over monopolies in an emerging AI market, that hardly existed 24 months ago, while ignoring the two worst monopolies in different domains, led by Microsoft and Google.
They aren't ignoring Microsoft and Google, that's ongoing too. But I agree that they should not be messing with the AI market.
You're grossly misinformed. CUDA is a huge anti-competitive moat, and that's no accident.
AMD is more than just cards.
Their competitors didn’t win. That’s not the same as failing, insofar as building technology goes. Business, sure
And openly undermined technologies and standards that aimed to do the same in an interoperable way.
This is, in my eyes, enough to hit with a litigation, but we'll see.
All of that can be true AND they're engaging in anti-competitive behavior, which should not be tolerated.
Well I'm sure that the courts will agree with your assertion if it's quick. That's why it's not left up to just one regulatory agency to define an illegal monopoly. I, for one, am glad they are coming under scrutiny. If there's "no there there" then they'll be just fine.