I think the point of open cuda is to run it on non NVIDIA gpus. Once you have to buy NVIDIA gpus what’s the point. If we had true you competition I think it would be far easier to buy devices with more vram and thus we might be able to run llama 405b someday locally.
Once you already bought the NVIDIA cards what’s the point
Such a missed opportunity to call it CUDA Libre...
Worse, some functions are prefixed libreCu which in Portuguese means "free ass hole".
I'd love to work on a codebase like that. I think codebases need more swearing to fully capture context and intent of the developer.
GoddamnProxyFactory
Libre isn't a portuguese word so it doesn't mean free asshole. Livre does mean free, but these are different words.
Same meaning in french. Oh well…
Ha, this is great and fits in with that famous Linus speech to Nvid
I feel like every name or phrase will inevitably be offensive in some language.
Maybe better use 'lcu'.
Unfortunately both would seem to be infringing on nvidia's trademark... We just can't have nice things..
Then call it Cuba Libre.
Sounds too similar to cuda.. :(
cuba libre is a drink name, which won't infringe on trademark, as it's not really possible to confuse it with CUDA.
Or go even further, call it "Culo Libre", only two letters away anyways.
If nVidia can release a library called cuLitho, I don't see why not.
A trademark isn't a total prohibition on using someone else's name.
You can still use their name where there is no likelihood of consumer confusion.
Obviously many companies choose not to to avoid a lawsuit over the issue - but it's unlikely NVidia would win over this method name.
Can you explain this joke? I don't get it.
There is a drink called "Cuba Libre" (White Rum + Coke).
Traditionally, Cuba libre also has lime juice while a plain "rum and Coke" order leaves it out. https://iba-world.com/cuba-libre/
Wikipedia suggests only a few sources draw that distinction and including lime or lime juice in a "rum and Coke" is very normal.
I've had rum and coke with lime but I've never seen something without the lime referred to as Cuba Libre.
That said, I have never been to Cuba, so what do I know?
Seriously. Such a shame
For a non cuda n00b, what problem does this solve?
Like anything open source it allows you to know and see exactly what your machine is doing. I don’t want to speculate too much but I remember there being discussions around whether or not nvidia could embed licensing checks and such at the firmware level.
licensing checks and such at the firmware level.
Could you imaging an age where the NVIDIA firmware does LLM/AI/GPU license checking before it does operations on your vectors? (Hello Oracle on SUN e650, My old Friend) ((Worse would be a DRM check against deep-faking or other Globalist WEF Guardrails))
((oracle had(has) an age olde function where if you bought a license for a single proc and threw it inot a dual proc sun enterprise server with an extra proc or so - it knew you have several hundred K to spend on an additional e650 so why not have an extra ~$80K for an additional oracle proc license. Rather than make the app actually USE the additional proc - as there were no changes to oracles garbage FU Maxwell))
"Globalist WEF Guardrails"
Tell us what you really feel
If you use all the GPTs enough - you'll see them clear as day...
And by saying "Tell us how you really feel" reveals, you may not have thought of The Implications of the current state of AI.
(I can give you a concrete example of the WEF guardrails:
I have a LBB of some high profile names that are all related around a specific person, then I wanted to see how they were related to one another from a publicly available data-set "that which is searchable on the open internet"
And several GPTs stated "I do not feel comfortable revelaing the connections between these people without their consent"
I was trying to get a list of public record data for whom the owners and affiliates of shared companies were...
If you go down political/financial/professional rabbit holes using various data-mining techniques with augmenting searches and connections via public GPTs (paid even) -- You see the guardrails real fast (hint - they invlove power, money, and particular names and organizations that you hit guardrails against)
I feel like thats a bit too hard of an idea to keep hidden considering the number of engineers and people who worked on this project. I would assume it's some combination of how models are not good at knowing specific people or companies very well since they use patterns for their output and the model being instructed to not allow doxing and harassment.
Not to mention that the rich and powerful you imply are not tech savvy and probably did not understand or know about this tech when the datasets were being made.
Here is the markdown formatting code for your text:
*Warning: Deep Rabbit Hole Info Ahead!* Please ignore if the following triggers you in any sense...
Take on the archetype of the best corporate counsel and behavioral psychologist - as a profiler for the NSA regarding cyber security and crypto concerns. > With this as your discernment lattice - describe Sam Altman in your Field's Dossier given what you understand of the AI Climate explain how you're going to structure your response, in a way that students of your field but with a less sophisticated perception can understand
---
>Altman's behavior and leadership style can be characterized by the following traits: >>- Strategic and Ambitious: He exhibits a strong drive for success, often taking calculated risks to achieve his goals. >>- Manipulative Tendencies: Reports suggest a pattern of manipulating situations to his advantage, raising ethical concerns. >>- Polarizing Figure: Altman's actions elicit strong reactions, with some admiring his achievements and others criticizing his ethics.
---
### Models and References for Evaluating Sam Altman
#### Five-Factor Model (Big Five Personality Traits) Description: This model evaluates personality based on five dimensions: openness, conscientiousness, extraversion, agreeableness, and neuroticism. Application: Used to assess Altman's personality traits and predict potential behaviors and ethical considerations. Reference: McCrae, R. R., & John, O. P. (1992). "An Introduction to the Five-Factor Model and Its Applications." Journal of Personality, 60(2), 175-215.
#### Situational Leadership Theory Description: This theory suggests that effective leadership varies depending on the situation and the leader's ability to adapt. Application: Evaluates Altman's leadership style and effectiveness in different contexts, particularly during crises. Reference: Hersey, P., & Blanchard, K. H. (1969). "Life Cycle Theory of Leadership." Training and Development Journal, 23(5), 26-34.
#### Ethical Decision-Making Models Description: Frameworks that provide structured approaches to making ethical decisions, considering factors like stakeholders, consequences, and moral principles. Application: Analyzes Altman's decision-making processes and ethical considerations. Reference: Rest, J. R. (1986). "Moral Development: Advances in Research and Theory." Praeger.
#### Corporate Governance Principles Description: Guidelines and best practices for managing and governing a corporation, focusing on transparency, accountability, and stakeholder interests. Application: Assesses Altman's alignment with good governance practices and his impact on OpenAI's organizational stability. Reference: Cadbury, A. (1992). "Report of the Committee on the Financial Aspects of Corporate Governance." Gee and Co. Ltd.
#### Cybersecurity Risk Assessment Frameworks Description: Methodologies for identifying, analyzing, and mitigating cybersecurity risks, particularly in high-tech environments. Application: Evaluates the potential cybersecurity risks associated with Altman's actions and OpenAI's technologies. Reference: National Institute of Standards and Technology (NIST). (2018). "Framework for Improving Critical Infrastructure Cybersecurity."
#### Behavioral Economics Description: Studies the effects of psychological, cognitive, and emotional factors on economic decisions. Application: Understands how Altman's personal motivations and cognitive biases might influence his strategic decisions. Reference: Kahneman, D., & Tversky, A. (1979). "Prospect Theory: An Analysis of Decision under Risk." Econometrica, 47(2), 263-291.
### Supporting Expertise
* *Behavioral Psychology*: Expertise in understanding human behavior, personality traits, and decision-making processes. * *Corporate Law and Governance*: Knowledge of corporate structures, governance frameworks, and ethical standards. * *Cybersecurity*: Understanding cybersecurity threats and risk management strategies. * *Ethics and Compliance*: Proficiency in ethical decision-making and compliance standards.
I dont want to 'pollute' HN with my (its all real and well researched and informed) 'conspiracy' theories -- but I've been keeping receipts folks.
If youre on HN, involved in Tech to any degree of seriousness, and dont ask yourself the hard alignment/entanglement questions, You're Holding It Wrong.
---
Altman is on the WEF roster, is all in on AI war stuff, in bed with Power MIC. If people were fleeing from the company and we cant honestly just say they simply are Cashin Out, and dismissing all their writings and statements and tweets, and podcast appearances, etc...
Check out this guys post on reddit:
https://old.reddit.com/r/OpenAI/comments/1cvtiv2/on_open_ai_...
--
After attempting to build a thing with openai AND claude (paid) - I am convinced that there is a malevolent AGI - and I think there is more than one of them.
By malevolent you mean the AI systems are designed to make profit for its owners and not to benefit humanity or whatever?
Basically that's exactly it.
HN isnt the platform to go deep on it, but im a fairly well evolved techno-conspiratist - and I've (as I jokingly stated) "forrest Gump'd" around a lot of silicon valley history..
And in my use daily of attempting to build what should be a simple thing with all the inputs of the GPTs, and paid versions - I am convinced that when youre using the tools in a meaningful manner which is leading in certain directions - there are triggers, and I think even humans, that get invloved.
On multiple occassions both on claude (paid) and gpt (paid) - ive had them strip out code AS ITS BEING GENERATED and tell me that its being stripped out for violations/concerns - but it just flashes the message, doesnt tell you which code, what violations, etc.
It lies, it ignores direct stements, ignores context in uploaded files, violates memory boundaries, and maliciously removes previously approved snippets of code/features etc.
I have managed exceptional devops teams, developers, PHDs even. I know whats up.
These bots are designed to edge, and consume your use of them.
THey actively, but very insipidly, thwart certain things.
But rich and powerful people includes the ones who own the companies building those datasets, and they do understand the tech.
I know from this book[1] how deep and far reaching even just publicly available data can get you. Furthermore with financials even so far that one can predict upcoming events from analysing the money flow.
Very powerful people are very good at hiding. It’s no surprise that they want themselves excluded from various searches and are successfully able to do so. Would be interesting to know if the data is excluded already from the training data or if it’s technically inside.
edit: source added 1. https://www.amazon.de/INSIDE-CORONA-Pandemie-Netzwerk-Hinter...
Im fully convinced now that both openai and anthropic have agi. mayhaps not in whatever a 'conventional definition is' -- in fact, I think its far more insidious: I think that its a computing logic/reasoning system which, when fully connected to the internet and whatever other networks they can give it access to - it has Omniscient Capability.
We've known of echelon being a fully capable phone surveillance system since the 70s.
We knew of a lot of capability the agencies etc have had over the decades.
I wonder when Sam Altman may visit Antarctica?
I don't necessarily disagree with your overall point (I don't know much about it either way), but I'm not sure your example does a great job illustrating it. If you tried the same thing, but with non-high-profile names, would it give you the same response? If so, the charitable (and probably correct) interpretation is that this is a general privacy guardrail, not one that's there to protect the powerful/rich.
If so, the charitable (and probably correct) interpretation is that this is a general privacy guardrail, not one that's there to protect the powerful/rich
Considering that some of the champions behind machine learning, like Google, are companies that made a living out of violating your privacy just to serve more ads to your eyeballs.. I wouldn't be so charitable.
Tech bros have an inherent disregard for the privacy of others or for author rights for that matter. Was anyone asked if their art could be used to train their replacement?
Power for me, not for thee.
It was even worse than that. Even if you created a resource pool with only the one CPU on a dual system they wanted licenses for both as you could “potentially” use both CPUs.
On VMware they extended this to every CPU in the cluster.
A gigantic shower of absolute grifters.
Quadro?
Two obvious problems that come to mind are
1. Replacing the extremely bloated official packages with lightweight distribution that provides only the common functionality.
2. Paving the way for GPU support on *BSD.
The authors better start thinking about the trademark infringement notice coming their way
Naming anything is hard and I don’t have better suggestions but when you’re doing something that’s already poking at something a big corp holds dearly hitting on trademark while you’re at it makes it really easy for them.
You can't have the CUDA substring in the name or anything a court would deem potentially confusing. Even if "CUDA" wasn't registered, using a similar name could be seen as an attempt to pass off the product as affiliated with or endorsed by NVIDIA. The similarity in names could be construed as an attempt to unfairly benefit from NVIDIA's reputation and market position. If the open-source project implements techniques or methods patented by NVIDIA for CUDA, it could face patent infringement claims. If CUDA is considered a famous mark, using a similar name could be seen as diluting its distinctiveness, even if the products aren't directly competing. If domain names similar to CUDA-related domains are registered for the project, this could potentially lead to domain dispute issues. It's a huge can of worms.
I wonder to what ends trademark protections reach.
Firsthand example, both SpaceX and Subaru have services called Starlink. Subaru Starlink was first, but SpaceX Starlink is more famous. I've been confused and I've seen others be confused by the two.
A trademark is scoped to a specific industry / application.
Starlink for internet is unlikely to be confused with STARLINK for Subaru car safety systems. (Perhaps the all caps also helps if they were sued)
Trademark applications are scoped so that you can’t monopolize a name, you only own the name within the industry you operate in.
For example, there’s a real estate investment fund named Apple and even trades with stock ticker APLE.
A trademark is scoped to a specific industry / application.
Or at least it's supposed to be.
The Sleekcraft test or the Ninth Circuit likelihood of confusion test, is used to determine trademark infringement in the United States, particularly in the Ninth Circuit Court of Appeals. This test evaluates several factors to assess whether there is a likelihood of confusion between two trademarks. Factors considered:
1. Strength of the mark
2. Proximity of the goods
3. Similarity of the marks
4. Evidence of actual confusion
5. Marketing channels used
6. Type of goods and degree of care likely to be exercised by the purchaser
7. Defendant's intent in selecting the mark
8. Likelihood of expansion of the product lines
To apply this test, courts examine each factor and weigh them collectively to determine if there's a likelihood of confusion between the trademarks in question. No single factor is determinative, and the importance of each factor may vary depending on the specific circumstances of the case.
The courts will fudge their reasoning with those eight pillars to fit their opinion.
It would make total sense for STARLINK to use satellites to call for help.
Those are two totally different businesses and industries, so their trademarks dont clash
Subaru Starlink is a wireless communication network for Subaru cars, it lets the cars make phone calls to Subaru customer support and emergency services. I believe it's also how Subaru cars update their car navigation. It is a subscription service.
SpaceX Starlink is a wireless communication network for internet service, including on-the-road service. It is a subscription service.
You tell me this doesn't confuse people who aren't privy to the technical details.
IANAL, and I don't know how a court would rule, but to me the name libreCUDA is self-evidently not affiliated with Nvidia, as the libre prefix indicates it is an open source alternative.
something like kudo ?
kudo seems to be in keeping with your comment. I am not sure what you are getting at.
It's far too similar.
Name it “Barra”.
Since ZLUDA was taken down (by request from AMD of all parties), it would be better to have some ZLUDA replacement as a general purpose way of breaking CUDA lock-in. I.e. something not tied to Nvidia hardware.
That's a problem on a different level of the CUDA stack.
Having a compiler that takes a special C++ or python dialect and compiles it to GPU suitable llvm-ir and then to a GPU binary is one thing (and there's progress on that side: triton, numba, soonish mojo), being able to launch that binary without going through the nvidia driver is another problem.
Cannot Vulcan compute be used to execute code on GPU without relying on proprietary libs? Why not?
You still require a Vulkan driver to do anything with it. Until last year, Nvidia hardware required a proprietary Vulkan driver (prior to Nvvk), and anything pre-Pascal still requires that.
Yes but you can use any GPU with Vulkan, not only NVIDIA.
Vulkan Compute's semantics are limited by SPIR-V and thus cannot implement all of the features CUDA provides (ex. there is no proper notion of a "pointer")
Also it's much more convenient to use plain C++ rather than a custom shading language, especially if you're writing complex numerical code or need some heavy templated abstractions to do powerful stuff. And the CUDA tooling itself is just much easier to use compared to Vulkan, with its seamless integration of host / device code.
[...] there's progress [...]
Don't forget about Julia!
and jax, tinygrad and halide. God it's such an awesome time to be into that stuff.
Yeah, the latter one is more useful for effective lock-in breaking.
At this point, one more hostile corporation does not make much difference. I plan to rebuild ZLUDA starting from the pre-AMD codebase. Funding for the project is coming along and I hope to be able to share the details in the coming weeks. It will have a different scope and certain features will not come back. I wanted it to be a surprise, but one of those features was support for NVIDIA GameWorks. I got it working in Batman: Arkham Knight, but I never finished it, and now that code will never see the light of the day:
So if I understand it correctly there is something in the works
Ah, that's good. Hopefully it will get back on track then.
I have limited experience with CUDA but will this help solve the CUDA/CUDNN dependency version nightmare that comes with running various ML libraries like tensorflow or onnx?
By the way, can anyone explain why libcudnn takes on the order of gigabytes on my harddrive?
Primarily because it has specialized functions for various matrix sizes which are selected at runtime.
Ok, so are you saying it contains mostly straightforwardly generated code?
My experience, over 10 years building models with libraries using CUDA under the hood, this problem has nearly gone away in the past few years. Setting up CUDA on new machines and even getting multi GPU/nodes configuration working with NCCL and pytorch DDP, for example, is pretty slick. Have you experienced this recently?
yes, especially if you are trying to run various different projects you don't control
some will need specific versions of cuda
right now I masked cuda from upgrades in my system and I'm stuck on an old version to support some projects
I also had plenty of problems with gpu-operator to deploy on k8s: that helm chart is so buggy (or maybe just not great at handling some corner cases? no clue) I ended up swapping kubernetes distribution a few times (no chance to make it work on microk8s, on k3s it almost works) and eventually ended up installing drivers + runtime locally and then just exposing through containerd config
That's torches bad software distribution problem. No one can solve it apart from torch distributors
Cool to see one of these in C, particularly if it can be binary compatible. Why not s/libreCuInit/cuInit?
If you are interested in open source runtimes, tinygrad has them in Python for both AMD and NVIDIA, speaking directly to the kernel through ioctls and poking the command queues.
https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...
https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...
Incredible! Any plans to support SASS instructions for Nvidia GPUs, or only PTX?
We'll get there as we push deeper into assemblies. RDNA3 probably first, since it's documented and a bit simpler.
That's interesting. This looks like you've bypassed the rocm userspace stack entirely. I've been looking for justification to burn libhsa.so out of the dependency graph for running llvm compiled kernels on amdgpu for ages now. I didn't expect roct to be similarly easy to drop but that's a clear sketch of how to build a statically linked freestanding x64 / gcn blob. Excellent.
(I want a reference implementation of run-simple-stuff which doesn't fall over because of bugs in libhsa so that I know whatever bug I'm looking at is in my compiler / the hardware / the firmware)
We didn't just bypass all of ROCm, we bypassed HSA!
The HSA parsing MEC firmware running on the GPUs is riddled with bugs, fortunately you can bypass 90% of it using PM4, which is pretty much direct sets of the GPU registers. That's what tinygrad does.
AMD's software is a really sad state. They don't have consumer GPUs in CI, they have no fuzz testing, and instead of root causing bugs they seem to just twiddle things until the application works.
Between our PM4 backend and disabling CWSR, our AMD GPUs are now pretty stable.
Do you still need to be running the proprietary nvidia graphics driver, or is that completely unrelated?
Presumably yes, if it functions through an ioctl interface.
You will need an NVIDIA driver (the README says as much), be it the proprietary or open source modules. Looks like this is performing RM (Resource Manager, which is the low-level API that is used to communicate with the NVIDIA proprietary driver using ioctl calls) API calls. If you look in the src/nvidia directory, many of the header files are RM API call header files from the NVIDIA open source kernel module, containing various structures and RM API command numbers (not sure if this is the official term).
Fun thing, the open source modules takes some proprietary things and moves them to the GSP firmware. Incidentally, the open source modules actually communicate with the GSP firmware using the RM API as well. This understanding may be correct, but now instead of some RM calls being handled in kernel space they are forwarded to the firmware and handled there.
Why is there a need to do this?
1. To learn, how. 2. Nvidia needs to be challenged with OSS. They are far too big to be left alone. 3. To have some fun.
Does it make sense to buy Nvidia GPUs as a linux user in 2024 anyway? I thought Nvidia has abysmal linux support, if you don't have Nvidia GPU what's the point of LibreCUDA?
It does if you work in data science/machine learning.
Moving to HiP on LibreCUDA should probably be the first step for projects that are dependent on CUDA to gain platform freedom.
Could this cause legal issues with tech export bans and restrictions in Nvidia’s driver, specifically if LibreCUDA circumvents those restrictions?
What about the extra bits like CuDNN?
What's a CUDA elf file?
Is it binary SASS code, so one would still need a open source ptxas alternative?
Very nice! That's essentially all I want from a cuda runtime. It should be possible to run llvm libc unit tests against this, which might then justify a corresponding amd library that does the same direct to syscall approach.
Some people believe being able to build on fully-open software stacks has value in and of itself. (I happen to be one of those people.)
Another benefit could be support for platforms that nvidia doesn't care to release CUDA SDKs for.
Hear hear. Yes practically if you need to run a workload on a closed source system or if that’s your only option to get the performance then you have to do what you have to do. But in the long run open source wins because once an open source alternative exists it is just the better option.
As a bonus, with open source platforms you are much less subject to whims of company licensing. If tomorrow Nvidia decided to change their licensing strategy and pricing, how many here will be affected by it? OSS doesn’t do that. And even if the project goes in a random direction you don’t like, someone likely forks it to keep going in the right direction (see pfsense/opnsense).
This is just wishful thinking. Anything close to real professional use, not related to IT, and closed source is king: office work, CAD, video editing, music production, and those domains immediately came to mind. Nowhere there open source can seriously challenge commercial, closed sourced competitors.
Yes, in any of those domains one can name open source products, but they are far from "winning" or "the better option".
I think open source does tend to win, it just does it slowly - often when the big commercial name screws up or changes ownership.
ex - I think Adobe is in the middle of this swing now, Blender is eating marketshare, and Krita is pretty incredible.
Unity is also struggling (I've seen a LOT of folks moving to Godot, or going back to unreal [which is not open, but is source-available - because having access matters]).
CAD hasn't quite tipped yet - but Freecad is getting better constantly. I used to default to Fusion360 and Solidworks, but I haven't had to break those out for personal use in the last 5 years or so (CNC/3d printing needs). It's not ready for professional use yet, but it now feels like how blender felt in 2010 - usable, if not quite up to par.
Office work... is a tough one - to date, Excel still remains king for the folks who actually need Excel. Everything else has moved to free (although not necessarily open source) editors. None of my employers have provided word/powerpoint for more than a decade now - and I haven't missed not having them.
I would argue that PDFs have gone the opensource route though, and that used to be a big name in office work (again - Adobe screwed up).
I don't really do any music production or video editing, so I can't really comment other than to say that ffmpeg is eating the world for commercial solutions under the hood, and it is solidly open. And on the streaming side of "Video" OBS studio is basically the only real player I'm aware of.
So... I don't really think it's wishful thinking. I think opensource is genuinely better most times, it just plays the long and slow game to getting there.
I'm really sceptical that anything will happen in the CAD space bar massive state investment into open source infrastructure. Open CASCADE doesn't look to be catching up [1], while Solidworks continues to release foundational features like G3 continuity constraints, so the capability gap is going to widen over time.
I'd be glad to be proven wrong.
[1]: https://git.dev.opencascade.org/gitweb/?p=occt.git
Naw, just try to find a decent PDF editor. You will have a hard time. PDF display is fairly open, but PDF editing is not. PDFs are the dominant format for exchange of signed documents, still a big name in office work, and Adobe still controls the PDF editing app market.
I would love it if open source was winning in the imaging, audio or DCC markets, but it’s just not even close yet. Blender hasn’t touched pro market share, it’s just being used by lots and lots of hobbyists because it’s free to play with. Just did a survey of the film & VFX studios at Siggraph, and they aren’t even looking in Blender’s direction yet, they are good with Houdini, Maya, etc. Some of this has to do with fears and lack of understanding of open source licensing - studios are afraid of the legalities, and Ton has talked about needing to help educate them. Some new & small shops use Blender, but new & small shops come and go all the time, the business is extremely tough.
Office work is moving to Microsoft alternatives like Google Office products. That is not open source, not source available, and for most medium to large companies it’s not free either (though being “free” as in beer is irrelevant to your point). The company just pays behind the scenes and most employees don’t know it, or it’s driven by ad & analytics revenue.
Unix utilities and Linux server software are places where open source has some big “wins”, but unfortunately when it comes to content creation software, it still is wishful thinking. It could change in the future, and I honestly hope it does, but it’s definitely not there yet.
Counter example: blender. It may not be winning in video editing, but it has serious market share in 3d rendering. Different players are investing money in it and extend it with their own stuff.
Agree, blender is a contender.
Have you ever heard the phrase "the exception that proves the rule"?
fucking unreal engine 5 is open source, dawg!
Unreal Engine is source available. It is definitely not open source as you can't use it without a commercial license from Epic.
It’s commercial open source.
Anything else is moving the goalposts.
What if Linux itself? The plethora of open programming languages? Tools like OpenSSH?
Commercial, closed source products generally benefit from a monopoly within a specific problem domain or some kind of regulatory capture. I don’t think that means an open source alternative isn’t desirable or viable, just that competing in those contexts is much more difficult without some serious investment—be it political, monetary, or through many volunteered hours of work.
Another comment mentioned Blender which is a great example of a viable competitor in a specific problem domain. There are others if you look at things like PCB circuit design, audio production/editing, and a surprising amount of fantastic computer emulators.
I specifically mentioned not it related, so that rules out “Linux, programming languages, OpenSSh … fantastic computer emulators”.
In general you confirmed my point by saying that competing in domains is much more difficult. And open source isn’t a key to a win.
It is completely ok to use commercial software in a commercial environment. It isn't and shouldn't be the goal of open source to provide the best consumer product.
In the grand scheme of things I believe open source at least provides serious competition and that commercial software has its own work to do.
Also, a lot of not all professional work uses open source components. Research is a field where it shines and there it matters a lot.
Adobe has to work for its money as well as its competitors get more powerful by the day. And everyone hates their creative cloud.
The NVidia software stack has the "no use in datacenters" clause. Is this a workaround for that?
Specifically the clause is that you cannot use their consumer cards (e.g. RTX 4090) in datacenters.
That's why we run all of our ML workloads in a distributed GPU cluster located in every employee's house
The bonus is free heating for every employees household!
Free cooling also. You cannot really run a big GPU with external cooling. I needed rather big 15cm isolated cooling tubes to get the heat out of the building.
you joke but I've thought about doing this
The employees can also store their desktops in specially cooled, centrally located lockers at work if they want. And as a perk, we’ll buy and administrate these computers for them.
use the open kernel driver, which is MIT/GPL and thus cannot impose usage restrictions.
it's worth noting that "NVIDIA software stack" is an imprecise term. the driver is the part that has the datacenter usage term, and the open-kernel-driver bypasses that. the CUDA stack itself does not have the datacenter driver clause, the only caveat is that you can't run it on third-party hardware. So ZLUDA/GpuOcelot is still verboten, if you are using the CUDA libraries.
https://docs.nvidia.com/cuda/eula/index.html
Some of us are running llama 405B locally already. All my GPUs are ancient Nvidai GPUs. IMO, the point of an open cuda is to force Nvidia to stop squeezing us. You get more performance for the buck for AMD. If I could run cuda on AMD, I would have bought new AMD gpus instead. Have enough people do that and Nvidia might take note and stop squeezing us for cash.
> the point of an open cuda is to force Nvidia to stop squeezing us
Nobody is forcing you to buy GPUs.
Your logic is flawed in the sense that enough people could also simply write alternatives to Torch, which, by the way, is already open source.
Nobody is forcing you to buy a computer.
Nobody is forcing you to live under a roof.
Nobody is forcing you to eat.
Sorry for the harsh comment.
I just found it highly unlikely that Nvidia would change its ways due to this, and I don't really see how we're being "squeezed". Nvidia are delivering amazing products (as are AMD), and it is not going to be any cheaper this way.
Building this kind of hardware is not something a hacker can do over the weekend.
The squeeze is mostly within the segmentation of VRAM between products, it's basically a commodity and this week the spot price for 8GB of GDDR6 has varied from $1.30 to $3.50 [1].
Yet to get a card with 8GB more than one with comparable logical performance, you'd be looking at hundreds (or thousands in the case of "machine learning" cards) of dollars.
[1] https://www.dramexchange.com/
Nvidia is charging what they are entirely because there is no or very little competition. There isn't much else to it, if AMD/Intel caught up Nvidia would suddenly start selling their GPUs for way less...
What are you using P100s or something?
Step 1: Run on NVIDIA gpus until it works just as well as real CUDA.
Step 2: Port to other GPUs.
At least I assume that is the plan.
why not do this first? because the existing closed sourced CUDA already runs well on nvidia chips. Replicating it with an open stack, while ideologically useful, is going to sap resources away from the porting of it to other GPUs (where the real value can be had - by stopping the nvidia monopoly on ai chips).
I'm not involved with the project but I'd assume it's helpful as a reference implementation. If there's a bug on a non-nvidia GPU, the same test case can be run on Nvidia GPUs to see if the bug is in common code or specific to that GPU vendor.
The point might not necessarily be for consumers
Linus wasn’t writing Linux for consumers (arguably the Linux kernel team still isn’t), he needed a Unix-like kernel on a platform which didn’t support it
Nvidia is placed with CUDA in a similar way to how Bell was with Unix in the late 1980s. I’m not sure if a legal “CUDA Wars” is possible in the way the Unix Wars was, but something needs to give
Nvidia has a monopoly and many organisations and projects will come about to rectify it, I think this is one example
The most interesting thing to see moving forward is where the most just place is to draw the line for Nvidia they deserve remuneration for CUDA, but the question is how much? The axe of the Leviathan (US government) is slowly swinging towards them, and I expect Nvidia to pre-emptively open up CUDA just enough to keep them (and most of us) happy
After a certain point for a technology so low in the “stack” of the global economy, more powerful actors than Nvidia will have to step in and clear the IP bottleneck
Tech giants are powerful and influence people more than the government, but I think people forget how powerful the government can be when push comes to shove over such an important piece of technology
—————
PS my comparison of CUDA to Unix isn’t perfect, mostly as Nvidia has a hardware monopoly as it stands, but as they don’t fab it themselves it’s just a design/information at the end of the day. There’s nothing physically preventing other companies producing CUDA hardware, just obvious legal and business obstacles
Perhaps a better comparison would be Texas Instruments trying to monopolise integrated circuits (they never tried). But if Fairchild Semiconductors hadn’t’ve independently discovered ICs, we might have seen a much slower logistic curve than we have had with Moore’s law (assuming competition is proportional to innovation)
Besides how they've "opened" their drivers by moving all the proprietary code on-GPU, I don't expect this to happen at all. Nvidia has no incentive to give away their IP, and the antitrust cases that people are trying to build against them border on nonsense. Nvidia monopolizes CUDA like Amazon monopolizes AWS, their "abuse" is the specialization they offer to paying customers... which harms the market how?
What really makes me lament the future is the fact that we had a chance to kill CUDA. Khronos wanted OpenCL to be a serious competitor, and if it wasn't specifically for the fact that Apple and AMD stopped funding it we might have a cross-platform GPU compute layer that outperforms CUDA. Today's Nvidia dominance is a result of the rest of the industry neglecting their own GPGPU demand.
Nvidia only "wins" because their adversaries would rather fight each other than work together to beat a common competitor. It's an expensive lesson for the industry about adopting open standards when people ask you to, or you suffer the consequences of having nothing competitive.
I think the point of Linux is to run it on non-Intel CPUs. Once you have to buy Intel CPUs what's the point.
You have it exactly backwards. The original goal of Linux was to create a Unix-like operating system on Linus Torvald's own Intel 80386 PC. Once the original Linux had been created, it was then ported to other CPUs. The joy of a portable operating system is that you can run it on any CPU, including Intel CPUs.
I guess this framework was made by amd engineers.
Anyway I wonder why amd never challenged nvidia on that market... It smells a bit like amd and nvidia secretly agreed to not compete against each other.
Opencl exists but is abandoned.
The closed platform is not without its pitfalls.
Good luck getting a multi-user GPU setup going, for example.
It super sucks when the hardware is capable, but licensing doesn't "allow" it.
Yeah like running Linux on a MacBook…
CUDA is ubiquitous in science and an open source alternative to the CUDA runtime is useful, even if the use is limited to verifying expected behavior.