return to table of content

Building the DirectX shader compiler better than Microsoft?

mouse_
27 replies
2d6h

Microsoft has no incentive to make good software. Most people will use it no matter what.

lukan
21 replies
2d5h

That is not true. People use it as long as the pain of using it, is lower than the pain of switching. Assuming they have even something to switch for.

pjmlp
18 replies
2d5h

Thankfully Valve is doing the good work to keep game developers targeting Windows and DirectX without caring about alternatives on the PC space.

timlatim
16 replies
2d4h

Apologies if I'm misreading your intention, but are you suggesting that Valve's work on Wine is somehow worse than asking game developers to target Linux/other OSes natively? As a Linux desktop enthusiast, I much prefer the Valve's approach: the library of existing Windows-only games that are unlikely to be ever ported is too vast, and the benefits of targeting a disjointed[1] platform with <2% market share[2] for new games are not at all clear. It's only thanks to Valve that I (and hopefully many other Linux users) do not need to maintain a second Windows system for fun, as the majority of games run perfectly fine on Linux and require nothing more than clicking Install then Play in the Steam client.

[1] Case in point: glibc's compatibility guarantees are weaker than what you get on Windows. (For instance, your system's glibc cannot be older than what a game is built against, which may present problems for devs using Fedora/Arch and players on Debian/LTS Ubuntu, something I've experienced first-hand for my apps.) The X11 to Wayland migration is also still underway. (Though things are getting better, the attitudes of some Wayland maintainers are a bit concerning: "I don't [care] what you think is normal behavior for games. You get certain guarantees with wayland. Deal with it. If clients decide to do exactly what they do on windows or X11 they won't work correctly." [3] I'm not sure game developers would enjoy such reception.)

[2] https://store.steampowered.com/hwsurvey

[3] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18...

pjmlp
11 replies
2d3h

Yes I am, everyone worshiping Valve for that, has skipped classes on OS/2 history lesson.

Porting games from Android/NDK into GNU/Linux is relatively a child's play.

Playstation OS is also POSIX friendly.

Finally every serious middleware engine supports GNU/Linux.

Still the amount of studios that care about GNU/Linux is almost zero.

With Valve, there are no reasons to bother at all as a studio, target Windows/DirectX, let Valve do the work, collect the money with zero additional effort.

Now with Windows based handhelds, Valve will learn what happened to netbooks.

pjc50
3 replies
2d2h

"GNU/Linux" doesn't have a stable binary graphics API. Someone rather controversially pointed out that if you want to make a binary that runs for the longest possible time across the widest set of linux distributions, you should make a Win32/DirectX binary and tell people to install WINE/Proton.

pjmlp
0 replies
1d10h

If they cannot manage themselves, it is not by becoming Windows/Linux that would save the situation.

kbolino
0 replies
2d

Wine/Proton works because it can be recompiled for every version of every Linux distro, whereas the average video game can't.

Honestly, somebody could have done something like this for "native" Linux but the incentives never really fell into place. It could even have been Valve, but I would guess they weighed the options and found this to be the one that gave them the widest compatibility without expecting too much of the game developers, many of whom were reluctant to make their games available on Steam at all.

Ygg2
0 replies
1d22h

Sure Linux has a stable graphics ABI. It's called Proton./j

It's a riff on only stable OS API on Linux is Win32.

badsectoracula
3 replies
1d18h

has skipped classes on OS/2 history lesson.

The comparison with OS/2 only applies at a very surface level - and isn't even the whole reason why OS/2 failed when it had to compete with Windows. Among other things:

1. OS/2 could only run 16bit Windows applications at a time when Windows was switching to 32bit. Even the 16bit Windows applications were not 100% compatible and that is despite using Microsoft's code.

2. OS/2 had much worse hardware support than Windows as everyone was targeting Windows. On the other hand IBM as a whole never put much effort towards OS/2.

3. OS/2 Windows support had applications either run inside an isolated environment or they looked "alien" next to OS/2 applications at a time when GUIs were still trying to look consistent. While this is also the case with Wine/Proton, the focus on games makes this point moot (and people do not seem to care as much about GUI consistency these days).

Now with Windows based handhelds, Valve will learn what happened to netbooks.

So far every single Windows handheld review i've seen that compares it to Steam Deck mentions both how the UX is worse than Steam Deck and games are -ironically- more likely to have issues on the Windows-based one. The only two saving graces for Windows handhelds is that they tend to be faster (but only when running at full throttle which limits their battery lifetime - Steam Deck runs faster at lower watts for better battery lifetime) and that anticheat rootkits work on the Windows handhelds whereas they do not work on Steam.

And you also forget that Valve did try to get game developers target Linux and put a lot of effort in the ecosystem for literally years before making Proton, yet developers largely ignored that.

pjmlp
2 replies
1d10h

Turning GNU/Linux into Windows/Linux won't make it work, regardless.

Microsoft is already half way there with WSL, as soon as they realised folks rather buy Apple gear for UNIX experience, instead of supporting Linux OEMs.

"Runs Windows better than Windows" didn't work last time, and won't now.

And if XBox really goes full speed ahead as cross platform brand, as the console rumor mill has been discussing, lets see how many Microsoft owned studios stuff keep landing on Steam.

badsectoracula
1 replies
11h14m

Turning GNU/Linux into Windows/Linux won't make it work, regardless.

It already works, i play games every day, pretty much all games i play are Windows games and i can count on my hands the number of times i booted on actual Windows the last three years.

pjmlp
0 replies
6h58m

The fact that "..pretty much all games i play are Windows games..." is the proof it won't last.

It suffices for Game Pass and Epic Store to be more appealing to studios targeting Windows, plus all Microsoft owned studios being taken out of Steam.

kbolino
2 replies
2d

GNU/Linux means Linux plus bash, coreutils, glibc, etc.

What do I need to make a video game? Input, sound, graphics. What use is a shell, a userland, or even a C library (if I'm not writing in C) for making a video game?

Android is _not_ GNU/Linux. It doesn't need to provide a shell or userland as part of the platform, and its C library is bionic not glibc. It also provides a lot of things GNU doesn't*, like input, sound, and graphics. It's also not designed to run on a desktop, and the differences between mobile and desktop are non-trivial and slowly growing.

Windows and macOS are full graphical operating systems. Linux is just a kernel. GNU/Linux is just a command-line operating system. Great for servers, terrible for video games.

Moreover, commercial video games are distributed in binary format, not source. Even if input, sound, and graphics were solved on Linux in a consistent way, the developers** can't help but break ABI compatibility every couple of years. Many apps from Windows 3.1 days can still run on Windows 11. What binary from Linux's early days can still run today, never mind with graphics or sound?

* = You could include GLib/GTK+/GNOME but then you're targeting a specific desktop environment and not simply "Linux" or even "GNU/Linux" anymore

** = Except for the kernel developers, upon whom Linus forces "never break userspace" as a hard rule

pjmlp
0 replies
7h0m

I specified Android/NDK for a reason, the NDK has all the same C, C++, SDL, OpenGL, Vulkan stuff that you will find on GNU/Linux.

delta_p_delta_x
0 replies
1d22h

I think you're talking past each other, and actually agree with each other.

I believe pjmlp's point (although it requires a fair bit of reading between the lines) is that Windows already has fantastic backwards compatibility (as you elaborated on), and Valve's work has created a situation such that all developers need to do is target and build for only Windows, release Windows-only binaries; then, Valve/WINE will do the hard work in ensuring they run seamlessly on Linux. This means developers don't need to care about building natively for Linux (à la Factorio and a tiny handful of other games). In other words as another commenter said, the real stable ABI on Linux is Win32 + WINE.

Furthermore, Valve's work also negates the work of open-source engine and game developers who have ported their engines and games to native Linux. This is because developing for Windows is a known quantity, and there is an overwhelming volume of resources, effort, and experience in writing games for Windows.

pjmlp concludes with 'Now with Windows based handhelds, Valve will learn what happened to netbooks', which I gather to mean that the Steam Deck will lose popularity to Asus and MSI's (and soon, other manufacturers too) handheld systems, since running most games directly on Windows is still easier than the occasional faff that someone has to do when running games on WINE/Proton.

blibble
2 replies
2d3h

indeed

Windows is going to end up as the 2030s version of dosbox

edit: as in Windows ends up with the same fate as MS-DOS: replaced with a piece of software analogous to dosbox

pjmlp
1 replies
2d3h

Which runs MS-DOS games, not GNU/Linux.

blibble
0 replies
2d3h

yes, that was indeed my point

Cloudef
0 replies
1d13h

That mesa issue reminded me of wayland issue thats been open for 3 years now where apps crash if theres even a slight stall due to the wayland socket becoming full. Instead of providing quick fix for actual app devs and users who are affected most, they have been designing this perfect solution for apps and compositor to agree on the socket size for 3 years now. Its similar crap with the server size decorations where they finally had to admit people want them and merged the protocol but don't implement it in gnome compositor :) I'd say wayland really was barebones when it was being pushed and major contributions to actually get it usable on desktop was from outsiders.

I'd say win32/flatpak/libretro are the only sane way for games to target linux right now. The fact that linux doesn't have real "runtime" and major components required by games link against libc is what makes linux really unstable for anything that needs to open window, draw stuff using GPU and play audio. It's possible to create static binaries for linux that work for eternity, thanks to kernel being actually stable. But link against something in /usr/lib and it all goes to hell. If the GPU drivers and libs that provided basic window / audio did not depend on libc and were standalone, the situation would be much better. Here's good video about this problem space btw https://www.youtube.com/watch?v=pq1XqP4-qOo

WhyNotHugo
0 replies
2d3h

Honestly, I don’t think this is a bad idea. Games are usually distributed in binary form, and Linux is an awful target for distributing binaries. Especially if you want them to work 10 years later.

Imagine games that dynamically link some library (or glibc). They wouldn’t run on other Linux distributions with mismatching dependencies (or a different libc).

Sure, 100% statically linked binaries might work. But wine also runs on non-Linux.

Windows is a sufficiently homogeneous target, and wine works well enough. Heck, some games run better in Linux+Wine than they do in windows.

npteljes
0 replies
2d5h

Assuming they have even something to switch for.

This one hits the nail on the head, and the reason why not just Microsoft, but a lot of large software players are not incentivized to create better software.

At the end of the day, people like power, to make money, and the people at Microsoft are no exception. And businesses are businesses, enterprises to make money, not altruistic benefactors of humanity, or optimizers of a specific domain, like software. So what business will do are their original thing AND business tactics, and the larger the business, the more tactics they have to employ, otherwise they won't be as large, or even simply won't be, at all. So, on the top, it's all ruthless business tactics. As Microsoft is a large player for a long time, they have quite the rep sheet[0], but they are not unique in doing this. It's the name of the game.

[0] https://en.wikipedia.org/wiki/Criticism_of_Microsoft

herbst
0 replies
1d8h

This. The majority of people I just switched to Linux just stayed there. All it takes is reducing the daily pain of using their own hardware.

'Most' just want their browser to behave like always anyway.

chollida1
3 replies
2d3h

I mean, this is an absurd statement.

Microsoft uses this software to make their own video games. They have a very vested interest in making any software they use better.

yazzku
2 replies
1d19h

It is not absurd at all. Just look and how every PC ships with Windows (and lately doesn't even allow you to boot an alternative OS unless you fiddle with Secure Boot in the BIOS). There is consequently little incentive to make Windows better, and we all know what a complete piece of utter garbage it is. Their next milestone is shoving ads in Outlook and your start menu.

And then the same can be said about a lot of Microsoft products. DirectX is no different; it's the canonical Microsoft piece of shit, and that goes all the way back to the OpenGL days [1].

[1] https://www.gamedeveloper.com/programming/why-you-should-use...

delta_p_delta_x
1 replies
1d19h

we all know what a complete piece of utter garbage it is

Are you saying this as a qualitative statement, or as an emotional and irrational outpouring of hate?

By qualitative, I mean things like performance, compatibility, UI/UX, programmability, software design, etc.

Somehow I doubt it's the former...

yazzku
0 replies
1d1h

Are you really trying to argue that Windows does just fine on all the fronts you mentioned?

immibis
0 replies
1d16h

This is no longer true. Smartphones are the default computing device, and Microsoft has to make an effort to keep people on desktops.

mcraiha
21 replies
2d7h

This is also related to Godot "The reason to make it optional is that Direct3D 12 support currently relies on the proprietary dxil.dll library from the DirectX Shader Compiler being shipped together with Godot, and shipping proprietary software goes against the mission of the Godot project." https://godotengine.org/article/dev-snapshot-godot-4-3-dev-3...

yazzku
11 replies
1d20h

Which part of dxil/dxc is proprietary exactly? Trying to make sense of the license barf at https://github.com/microsoft/DirectXShaderCompiler

zerocrates
6 replies
1d19h

The license (and the code) for dxil.dll/libdxil.so isn't in that repo, they just include the blob in releases. If you look at a release you'll see an additional LICENSE-MS.txt that just covers that dxil signing library.

yazzku
3 replies
1d19h

How is that compatible with the GPL licence from autoconf?

zerocrates
0 replies
1d19h

Looks like the only autoconf bit they use is an old copy of config.guess that was used in the LLVM they forked from (I think they've since taken it out entirely in the LLVM upstream).

There's actually a carveout in config.guess's license that lets you redistribute it under whatever license you want but only if you're actually using an autoconf-based build. The LLVM version this is based off of only uses config.guess and not anything else from autoconf, so that carveout doesn't apply. Instead LLVM just relied on the fact that config.guess is only getting called via the command line, so the GPL's "virality" doesn't apply; LLVM (and now Microsoft) just distributed config.guess separately licensed as GPL alongside their code.

p_l
0 replies
1d16h

GPL doesn't care about inclusion or linking, it cares about derivative work. And somehow I doubt the 3d engine is derivative work of a piece of autoconf.

Dylan16807
0 replies
1d19h

You mean the config.guess script?

That's a completely different program. GPL doesn't jump across programs.

nextaccountic
1 replies
1d16h

On the latest release https://github.com/microsoft/DirectXShaderCompiler/releases/...

There are "source code" files in zip and tar.gz

Aren't those source code for those dlls?

nhubbard
0 replies
15h38m

No, that’s a GitHub thing — it just downloads the Git repo at that release tag, which doesn’t include the source code in this case

charcircuit
3 replies
1d20h

The source code for dxil.dll is not part of that repo.

yazzku
2 replies
1d19h

Looks like this is another case of "Microsoft loves Open Source" then.

charcircuit
0 replies
1d19h

Are any libraries from the Windows SDK open source? Windows application code calling into libraries that are not open source is nothing new.

Dylan16807
0 replies
1d19h

If someone says they love pizza, do you call them a faker every time they eat something else?

moffkalast
6 replies
2d2h

One of those things that the end users won't ever give half a shit about. Same thing when installing some linux distros: "dO yoU wANT To doWNloAd AND iNStAll 3Rd paRty driVers"

Yes. Yes I want my GPU, wifi and bluetooth to work. Get over yourselves and leave that checkbox checked by default.

gkbrk
1 replies
2d1h

GPU, Wifi and Bluetooth commonly work with open source drivers

comex
0 replies
1d22h

More often the issue is firmware rather than drivers.

matheusmoreira
0 replies
1d3h

Huh? I absolutely do not want proprietary drivers tainting my kernel. I barely tolerate firmware blobs. So far only nvidia has required such drivers.

dvngnt_
0 replies
2d1h

some people do

TillE
0 replies
1d22h

It's not an arbitrary ideological decision, it's a legal one. Blame Microsoft for their dumb license on dxil.dll which, as I read it, would require Godot to add some kind of click-through agreement.

AeroNotix
0 replies
2d1h

Speak for yourself.

malkia
1 replies
2d1h

dxil.dll is managed ("dotnet") code, right?

yazzku
0 replies
1d20h

No, it's C++, based on LLVM. https://github.com/microsoft/DirectXShaderCompiler

Edit: apparently dxil.dll is not part of DXC (the classic move to make "open source" software dependent on external proprietary garbage, apparently.) But I'd still doubt it's a managed DLL.

msk-lywenn
13 replies
2d6h

So the signing[1] DXIL.dll does is just a modified MD5?

1: https://github.com/hexops/DirectXShaderCompiler/blob/4190bb0...

Quarrel
5 replies
2d2h

I find it frustrating that the Mach guys have this great write up, but don't just say what the "signing" is.

Anyone paying attention in the article to that point knew it had to be some basic hash or similar, where even in your worst case someone has to reverse engineer some assembly.

After all that effort, just out Microsoft, particularly when we're talking about opensource code that anyone that cares to can just dig through and find (thanks for that, msk!).

LoganDark
3 replies
1d18h

I find it frustrating that the Mach guys have this great write up, but don't just say what the "signing" is.

I think it's for plausible deniability in case M$ ever comes after them for RE reasons. They probably want to be able to say that they didn't use the proprietary blob in order to implement their own code signing.

Cu3PO42
2 replies
1d17h

If that is indeed the reason, I won't blame them. I wouldn't want to upset Microsoft either. If they decide to come after you with an army of lawyers, it doesn't necessarily matter if what you did was legal. And I think it probably was legal in most jurisdictions. Reverse engineering for purposes of interoperability is generally allowed. Of course they could still try to get you on copyright if you didn't do a clean room re-implementation.

LoganDark
1 replies
1d15h

Yeah. It's easy to say you got it from some other open-source project that did their own clean-room reimplementation, unless there's clear evidence on your very own website that you looked at their proprietary code anyway, which would allow them to make the case that your knowledge was tainted and your implementation couldn't have possibly been clean-room.

EMIRELADERO
0 replies
21h11m

But copyright doesn't care about "tainted", it cares about whether there's substantial similarity between the two works. Copying only the system/mechanism but not the exact code is legal, even if you admit you used the actual decompiled code as a direct guide.

meemster
0 replies
1d3h

Just for my own understanding, why was it obvious that it has to be a basic hash? At least it wasn't obvious to me, but I have very little experience in this area.

Culonavirus
2 replies
2d3h

This is the part of HN I really like. People who can look at random C code working with memory and be like "yep this looks like modified X". Pretty amazing (to someone like me who mostly works in high level languages).

msk-lywenn
1 replies
2d3h

With the phrasing of the article, I immediately thought it must have been something simple/dumb. I quickly thought MD5 so I looked at the magic numbers in the C code and looked them up on wikipedia. Then I noticed the four F G H I functions. Still I’m not sure it is (maybe it’s something common in hashes?) and I still don’t know what the change is

EMIRELADERO
0 replies
1d23h

From the commit message[1]:

"dxil.dll is closed-source, so we cannot simply patch it in the same way. To fix this, we outright disable runtime loading of dxil.dll and silence warnings related to it not being present / the final binary not being code signed. Instead, once the compiler would emit a compiled shader blob, we perform our own code signing algorithm (Mach Siegbert Vogt DXCSA) which results in a bitwise identical compiled shader, and thus dxil.dll is no longer needed to perform code signing of shaders."

[1] https://github.com/hexops/DirectXShaderCompiler/commit/7a013...

charcircuit
1 replies
1d22h

Renderdoc, who has had code to do this since 2021 has a good comment that explains it.

https://github.com/baldurk/renderdoc/blob/4a620bb5a16b4de4e2...

amluto
0 replies
1d22h
atq2119
1 replies
2d3h

That "signing" has always been bullshit security theater. You'll note that every other graphics API manages just fine without it. I'm glad somebody posted the "signing" algorithm out in the open.

TillE
0 replies
2d2h

Putting a pseudo-MD5 hash in your file header sounds like someone got confused and wound up caught between over-elaborate integrity check and half-hearted security measure.

It sort of works if your signing tool is part of a private console SDK, but the DirectX SDK was always freely available.

sylware
6 replies
2d5h

I guess the less troublesome would be something like HLSL/GLSL->SPIR-V<->DXIL, or shaders could be authored directly in SPIR-V.

In vkd3d (from wine), I think you have a DXIL->SPIR-V translater (much more robust than any high level shading language converter since it is a much more simple intermediate language).

That said, apart from this abomination of llvm, is there a HLSL->DXIL compiler written in plain and simple C99 (namely which require NOT gcc or clang to compile)?

noahwhygodwhy
3 replies
2d5h

"Shaders could be authored directly in SPIRV" oh god please no lmao

Also to answer your question, no. The hlsl to dxil translation is basically owned by microsoft. There's been little effort to move away from that

NekkoDroid
1 replies
2d4h

Well... they are rewriting & upstreaming DXC to LLVM main (https://github.com/microsoft/DirectXShaderCompiler/wiki/Cont...) and want to kinda deprecate DXC. IIRC currently only compute shaders are supported but I may be very wrong.

sylware
0 replies
1d7h

llvm (apple) is a mistake, but I am not expecting microsoft to do anything else than mistakes anyway.

sylware
0 replies
1d7h

I am really serious about direct authoring of shaders using SPIR-V (with probably a little SSA checker on the side). We can reasonably think it will significantly increase the compatibility and bug avoidance of shaders among various drivers (and that is priceless when you think QA of big video games spaning many different drivers).

I am perfectly aware this will require a bit more upfront work to write shaders, but due to their life cycle, it should be benign and we should get all the benefits (not even mentioning to free the SDK dependency from a massive and complex high level shading language compiler).

I am going to give it a try. I need first a SPIR-V assembler, the one from the khronos spriv tools and the one from llvm are c++ diarrhea then a definitive nono, have to write my own. Let's think long run here: we don't have a _REAL_ standard very high level language yet (python? lua? javascript? perl5? ruby? so_many_others?), I'll go rv64 assembly then (I'll write a mini rv64 interpreter for x86_64).

slimsag
1 replies
2d5h

Yes, SPIRV->DXIL does seem tempting and worth exploring. Very recently I learned Mesa has such a tool, spirv2dxil, though unsure how robust/featureful it is compared to HLSL->DXC->DXIL.

sylware
0 replies
1d7h

I compile mesa every week and I missed that... lol.

That said, vkd3d(wine) is plain and simple C, and I suspect that mesa tool to be part of microsoft "push" in mesa, dirtying it with c++.

flohofwoe
5 replies
2d7h

A great overview of the terrible mess that underlies cross-3D-API shader compilation (and even though it focuses on D3D and Microsoft, it's not much better on other 3D APIs - for instance you simply can't cross-compile Metal shaders from a Linux host - only from macOS and - somewhat recently - Windows).

If the Mach team can pull off this whole "Zig as a cross-3D-API shader compiler" and make it work as smoothly as "Zig as a cross-compilation toolchain", then this would be pretty much the biggest thing in computer graphics since 1995 or so :)

matheusmoreira
1 replies
1d3h

for instance you simply can't cross-compile Metal shaders from a Linux host

Why not? Doesn't that just mean that nobody has implemented it yet?

flohofwoe
0 replies
1d1h

The Metal shader compiler is only distributed as binary blobs for Mac and Windows. Reverse engineering of the undocumented output is possible of course, but that also means keeping track of all the changes Apple does to the output format in new versions.

astrange
1 replies
1d18h

If you can do it on Windows doesn't that mean you can do it on Linux with Wine?

flohofwoe
0 replies
1d7h

It probably works, but Jenga tower hacks like that is exactly the problem ;)

lr1970
0 replies
2d6h

If the Mach team can pull off this whole "Zig as a cross-3D-API shader compiler" and make it work as smoothly as "Zig as a cross-compilation toolchain", then this would be pretty much the biggest thing in computer graphics since 1995 or so :)

And a major boost for Zig -- both as a language and a toolchain.

a1o
4 replies
2d4h

SDL people are working in creating some shader language in the form of SDL_gpu (not the old one), to launch in SDL3, so it could have some cross platform way to work with 3D graphics there for games. I have been looking at it closely for a little while.

jsheard
3 replies
2d4h

Is SDL_gpu meaningfully different from WebGPU? Looking at their goals they seem to be more or less the same, building an accessible lowest-common-denominator abstraction over DX12/Vulkan/Metal, except SDL_gpu is an early work in progress and WebGPU already has two good open implementations.

stephc_int13
2 replies
2d4h

Not sure about WebGPU. Is it really good? Have you checked with some benchmarks?

From my experience, it's been disappointing. Big, complex to setup, not convinced about many tradeoffs.

jsheard
1 replies
2d3h

WebGPU running on the web necessarily has overhead compared to native due to all the extra validation it has to do for security, but there's nothing stopping native implementations from offering unsafe escape hatches as extensions for those who want them.

kvark
0 replies
23h10m

And wgpu has been doing this for years. Things like descriptor indexing are not exposed to the web but used by Rust (mostly) engines on native.

https://wgpu.rs/

delta_p_delta_x
3 replies
2d3h

However, it doesn’t require any additional .dlls to be shipped with the application.

Many video games already do this with all the proprietary middleware they use (Bink, SpeedTree, PhysX, etc). Most launchers (Steam, GOG, Epic, etc) also require their respective .DLLs. Many games also use D3D11On12. Many shipping games in my list have dxil.dll amongst their installed files.

Therefore, an honest question: what's the problem with shipping an additional DLL? The work done here to reverse-engineer and re-implement the code-signing is fantastic—especially the fact that it is bitwise equal to dxil.dll's output. But I am ridiculously lazy and prefer to take the easier way out, and would've just shipped the DLL.

msk-lywenn
0 replies
2d3h

As you pointed out, it’s really not a problem. However, the article points out at the end the real interesting bit: cross building from any OS/arch.

chris37879
0 replies
2d2h

This opensource implementation can be included in existing game engines without requiring the programmer to know or understand what dxil.dll even is. The Mach engine, for instance, can use this to create a compiler that compiles zig code into shaders across all its targeted platforms in a way that doesn't require any additional setup or dependencies for the end user, it's just built into Mach's core.

badsectoracula
0 replies
1d18h

There are more games than AAA games and there are more graphics applications than games. Something that is fine for your average 100GB AAA game may not be for a 30MB game (of which 24MB are DXC) on itch.io or a 3D model viewer/converter/animator/whatever on someone's website.

In addition, even for AAA games (but also for other games and software, this adds extra dependencies that may be broken outside your control. At a company i worked several years ago, we spent more effort upgrading Visual Studio because for some middle we only had DLLs (and their libraries) to link against and had to obtain new versions (part of the effort was that the company that wrote said middleware hadn't upgraded their own so we also became QA for their compatibility with the newer VS, though i don't remember if there were any issues in practice).

Of course having the code doesn't mean updates will be frictionless, but there will certainly be way less friction and you wont need to wait for someone else.

(btw i think Visual Studio has tried to be backwards compatible in recent versions with binary C++ libraries, but i don't think this is something you can rely on in the long term)

Also all that are with the assumption that the requirement is that your code remain on the same platform and targets. But at some point you may want to work with another platform (be it as host or target) and not having source code can make that from incredibly difficult to impossible. This may not seem much of an issue for something as platform-specific as DXIL - but actually notice the article mentioning that the binary blobbiness of DXIL made it impossible to precompile shaders on platforms outside specific Windows and Linux architectures for which the DLL was provided by Microsoft.

unnouinceput
2 replies
2d1h

Quote: "All that was left was that pesky dxil.dll - what sort of magic might Microsoft be employing in that library to “sign shaders”? How can they prevent unsigned shaders from running on Windows machines that aren’t in developer mode? How are they able to distribute that binary on Linux, too? I won’t comment on any of those questions, but will say that you’ll find dxil.dll is NOT a dependency of mach-dxcompiler in any form. You can compile an HLSL shader on a macOS machine using mach-dxcompiler, without the proprietary dxil.dll blob - and end up with a DXIL bytecode file that is byte-for-byte equal to one which runs it on a standard Windows box. Enjoy!"

That above is the real magic. Since he won't comment on the how, I guess he took a swing at poping the hood underneath, and did exactly what Wine developers did 2 decades ago. Any old timers here remembering that scandal? Smart kid to not comment on the how, this way Microsoft won't have any legal leg and since times have changed with all that "Microsoft loves Linux" shit they yell at all corners (not that I believe that for a single second), then it will be swept all under the rug and FOSS wins. For now.

rofrol
0 replies
2d
charcircuit
0 replies
1d22h

Since he won't comment on the how, I guess he took a swing at poping the hood underneath

The way signing is done is already public knowledge that can be found using google as other open source projects have implemented it.

cactusplant7374
1 replies
2d4h

Playing Halo CE on Mac M1 with wine is an abject failure. I am assuming it is because Direct3D support is implemented poorly?

Cu3PO42
0 replies
1d17h

Maybe. If you haven't tried it, you could look into Apple's own Direct3D to Metal translation layer from Game Porting Toolkit. It generally gives better results than DXVK chained with MoltenVK.

trevortheblack
0 replies
1d23h

DXIL is pronounced "dixel", like "pixel" but replace the p with a d

steeve
0 replies
2d4h

I highly advise people to look into the Mach ecosytem, especially mach-sysgpu, which is a complete reimplementation of WebGPU, written (mostly) by Ali Chraghi, who is 17

nraynaud
0 replies
2d5h

Somebody went to take a big one for the team Open Source, thank you.

nightowl_games
0 replies
2d3h

Using zig itself as a shading language is awesome. Zig is the one true language. It's a build system! It's a shading language! Wow!

fifteen1506
0 replies
2d4h

Microsoft <3 Open Source

detay
0 replies
2d4h

Very promising and deserves support.

chrsig
0 replies
1d23h

DXC’s fork of LLVM removed and/or damaged much of the code generation layer and infrastructure [of LLVM]. Given that, supporting DXBC generation in DXC would be a massive task to fix and restore broken LLVM functionality. Due to the large scale of this issue and resource constraints on our team we’re not going to address this issue in [the new] DXC [compiler] ever. > > We may support DXBC generation in Clang in the future (we mentioned that in the original proposal to LLVM). That work is unlikely to begin for a few years as our focus will be on supporting DXIL and SPIR-V generation first.

I appreciate this quote[0] from the microsoft camp. Setting clear expectations that something will not be done is a nice bit of fresh air.

[0] https://github.com/microsoft/DirectXShaderCompiler/issues/57...

Voultapher
0 replies
2d6h

What a joy to read. It's this sort of infrastructure work that unlocks so many doors. Thank you Stephen!

Emigre_
0 replies
2d6h

This is excellent. Bravo, mister! Really interesting stuff.