Show HN: dockerc – Docker image to static executable "compiler"

I know it feels like being pedantic and missing the joke, but "just give me the EXE" is a terrible bug, not a feature request. Distributing unauthenticated, untraced OS-level binaries is just dangerous in the modern world. Safe app distribution requires either elaborate sandboxed runtimes (browsers) or carefully curated and maintained lists of known-safe binaries (app stores, distro package repositories).

We can't be doing this "GIVE ME THE .EXE" anymore. Those days are gone.

Even building from source is questionable, but at least there it requires that the installer be part of (or at least adjacent to) a community of developers who can be expected to have the expertise to notice and recognize bad actors pushing code.

Im surprised no one mentioned that the program the unstable ranter wants is essentually a social media stalker app to find people on all social media sites.

Perhaps some barriers to use are good sometimes.

This isn't an EXE, though.

The next step is to repackage this inside an EXE that runs a Linux virtual machine.

It would make sense that a tool like this would prioritize MacOS/Windows support above all else.

Yep, unfortunately I've not had the time to make it work well on those platforms. I got an initial demo working on MacOS but I'm currently facing the issue that I'm unable to statically compile QEMU on MacOS. I've also started writing a VirtualizationFramework[0] based backend.

Why would QEMU even necessary? Docker runs fine on Windows.

When docker runs on Windows/MacOS it's actually running the containers in a Linux VM. Containers rely on features provided by the Linux kernel.

Maybe it's to avoid requiring the user to install Docker?

The main reason to use dockerc is to avoid the user having to install Docker.

Either way, asking the user to fiddle with Hyper-V settings is bad UX.

Yep I don't think that would be nice. I expect the experience to be transparent to the user.

[0]: https://developer.apple.com/documentation/virtualization

The way container runtimes on Linux work is fundamentally different to MacOS and Windows. You need virtualization (albeit, lighter weight since they can use the host kernel) to run containers on Windows and MacOS.

QEMU is kind of overkill because MacOS provides the VzVirtualMachine API through the Virtualization Framework, which can initialize a VM with the host's kernel. On Windows you can use Hyper-V, which is iirc how docker on Windows gets this done.

If MacOS and Windows had pid/mount/network namespaces, overlayfs, and allowed for unrestricted chroot (MacOS requires disabling SIP for that) then you could do the same thing on all platforms. Today, you need virtualization.

Both the standard Windows and Mac OS versions of docker use VMs (Windows uses WSL2 and IIRC Mac OS uses Virtialization.framework) as the OSes don't provide the a linux kernel for the containerized userspace. If it didn't do this, it'd have to host Windows and Mac OS environments, which would defeat the portable nature of Docker.

So, someone else mentioned this could be sarcasm, but I've got a real application that could use this. It's a complex multi-domain modeling tool incorporating multiple C binaries, python and julia code. We currently deploy with Docker, but there are too many parameters user could set incorrectly in the Docker Desktop interface. We'd love to ship a static binary of this behemoth to cut down on the user/sysadmin deployment error.

You jest, but python is the poster child for, works on my machine.

/s this please, people might get some ideas otherwise...

You should also be able to do that with Cosmopolitan/APE.

You can already do this with these forks of ruby-packer and traveling-ruby:

https://github.com/ericbeland/ruby-packer https://github.com/YOU54F/traveling-ruby

ruby-packer is what I use to distribute a paid CLI, although on macOS only because my product is specifically for customers on macOS.

The advantage of ruby-packer is that it is much simpler, but you need to have access to each OS where you want to distribute your executable. OTOH, with traveling-ruby, you can build executables for all OSes from the same machine.

Yep! It works with any docker image!*

*: https://github.com/NilsIrl/dockerc/issues/6

Every decent application should be a portable executable binary!

I wonder, how does this approach differ from a unikernel?

Thanks!

Will this run ok on a Mac?

I've managed to make it work but unfortunately not in a way that produces portable binaries. I just need to figure out how to selectively statically link some of the QEMU dependencies or write a runtime that uses Apple's VirtualizationFramework.

But Nix is the best way to build images for dockerc.

Learning nix results in needing more therapy, though;)

Omit internet tropes.

Please don't post shallow dismissals, especially of other people's work.

https://news.ycombinator.com/newsguidelines.html

I had to look it up because I couldn't believe it was real but... https://old.reddit.com/r/github/comments/1at9br4/i_am_new_to...

That was Sourceforge really I suppose wasn't it - I enjoy GitHub for the opposite reason really (that I am a 'smelly nerd' and do want code more often than an executable, well actually they said '.exe', and I never want that) but I've definitely come across repos, even maintain one^, where they have that kind of real world leaking in, not really realising GitHub's not for them and then getting frustrated by it or just doing strange things.

(^I'm a collaborator on awesomecv, a latex CV/resume template, and we get a lot of PRs from people trying to edit their own information & experience into the example. I keep meaning to set up a template repo of how to use it properly as a latex class, maybe include rendering in Actions, the way my own use of it is basically, and more strongly point people towards that in the readme, maybe even remove the example. But I've been saying that for years, since before 'template repo' existed in fact.)

Care to elaborate?

Yeah, I don't think this makes that much sense for internal distribution. Especially if you use kubernetes.

The use case is for distributing software to end users.

I hadn't heard of Apptainer/Singularity before but it doesn't seem to provide the ability to create standalone executables.

Unfortunately cosmopolitan wouldn't work for dockerc. Cosmopolitan works as long as you only use it but container runtimes require additional features. Also containers contain arbitrary executables so not sure how that would work either...

As for WASM, this is already possible using container2wasm[0] and wasmer[1]'s ability to generate static binaries.

[0]: https://github.com/ktock/container2wasm

[1]: https://wasmer.io/

The modern ELF. Bash but instead of running executables and piping stdin and stdout your primitives are docker containers.

I think that already exists in the form of buildpack..

Sounds like what my mom would say, tbh

Yep, this is a known issue. It also existed in the past with normal docker. https://github.com/NilsIrl/dockerc/issues/6

Probably because it has a super nice build toolchain, especially when you consider cross compilation.

There's quite a few folks using zig _just_ for the toolchain, while still writing C.

The main reason Zig was chosen is because the project was started at a hackathon[0] at which there was a prize for "best use of Zig". Beyond that there were also other reasons: 1. I have been wanting to try out Zig 2. It fit the requirement of being a so called "systems language".

However having written it in Zig I have a few retroactive reasons for why Zig was a good choice (if not the best choice):

* The build system allows to make the runtime a dependency of the "compiler". I don't think any other language has that.

* The interoperability with C/systems calls is amazing (in comparison to anything but C/C++)

* The ability to embed files

* Makes it incredibly easy to make static binaries

[0]: https://treehacks.com/

[1]: https://github.com/NilsIrl/dockerc/blob/68b0e6dc40e76c77ad0c...

Yep pretty much.

The executables bundle crun (a container runtime)[0], and a fuse implementation of squashfs and overlayfs. Appended to that is a squashfs of the image.

At runtime the squashfs and overlayfs are mounted and the container is started.

[0]: https://github.com/containers/crun

Sounds like universal appimage

don't give me ideas ;) (yes)

That's for the developers to build the binary, not the end users. But yeah.

Thanks for the bug report! Just pushed a change that fixes it (and made a new release with the issue fixed: https://github.com/NilsIrl/dockerc/releases/)

dockerc can, the produced executables cannot, at least not without some tweaking.

Even if you path through `/dev/fuse` and `/usr/bin/fusermount3` (using -v) it fails the mount the fuse filesystems with the error message "Operation not permitted".

It might be possible to make it work if it falls back to extracting the container when failing the mount the fuse filesystems.

That's really cool. I didn't know about that. Thanks for sharing.

There's a few things that seem to make it unsuitable for the intended use case of dockerc:

* The container extracts itself which means there is quite a bit of overhead for large images. dockerc-built executables have the same startup time whether the image is 2.2GB or 25MB.

* The executables produced by enroot don't seem suitable for running standalone. At least the example doesn't seem to suggest so.

I don't expect this to interact with the docker daemon in any way. With networking executables generated by dockerc behave in the same way as a native application running outside a container.

What about using Zon for deps, instead of a git submodule?

I couldn't find documentation quickly enough (dockerc was initially during a hackathon and was my first time using Zig). I plan to fix this eventually.

They have the same goal but achieve it differently.

dockerc allows you to re-use your existing docker images without having to spend time packaging something up.

The applications running from dockerc-generated executables run inside a container so they are guaranteed to be hermetic.

It will depend heavily on the docker image you're trying to ship. For example with macos-cross-compiler[0] the resulting binary is over 2GB. With python:alpine[1] it's only 25MB.

Because the image isn't copied whether the image is 2GB or 25MB the startup time will be nearly instantaneous for both.

The runtime adds 6-7MB of overhead although I expect that this can be reduced to less than 3MB with some work.

[0]: https://github.com/shepherdjerred/macos-cross-compiler [1]: https://hub.docker.com/_/python

Only if you don’t use multistage builds

I thought it was an acceptable trade-off given that AppImage has the same limitation.

An alternative is to extract the image to disk but that has quite a bit of overhead.

How do you deal with resource sharing? This starts with just filesystem mounts, but also concerns ports, possibly devices, and probably many other things I'm forgetting. Is this somehow configurable?

I'm not too sure what resources you're talking about in general. Mounts are in a temporary location so they shouldn't conflict. Each container uses 2 when it is running. In terms of ports, you won't be able to have multiple applications using the same port (whether they are built with dockerc or not). As for devices I don't think there's any issues there.

How does this compare to AppImage? IIRC that also puts everything into a squashfs.

It's very similar to AppImage in spirit. I haven't looked at the AppImage implementation but I suspect a lot of things are similar.

The difference with AppImage is that this makes it trivial to convert existing docker images into something that can run as an executable. It also offers stronger hermeticity guarantees as the application runs inside of a container.

If a user without CAP_SYS_USER_NS executes one of the binaries built by dockerc, do you handle that gracefully in any way?

It's not something I've paid much attention to. This falls back to the container runtime which currently outputs "clone: Operation not permitted" when ran with `sudo sysctl -w kernel.unprivileged_userns_clone=0`.