For the past few months we have been working hard to provide a fast, reliable and secure KVM backend for VirtualBox. VirtualBox is a multi-platform Virtual Machine Monitor (VMM) with a great feature set, support for a wide variety of guest operating systems, and a consistent user interface across different host operating systems.
Cyberus Technology’s KVM backend allows VirtualBox to run virtual machines utilizing the Linux KVM hypervisor instead of the custom kernel module used by standard VirtualBox. Today we are announcing the open-source release of our KVM backend for Virtualbox.
I have ever only heard KVM in the context of a Keyboard Video Monitor-type device but somehow I can't fully fit that into the concept of a virtual machine. Does it mean something different here?
https://linux-kvm.org/page/Main_Page
Is there a connection to User Mode Linux (UML) from around the same time? Or are these completely unrelated projects? I get that running a kernel in the user space provided by another kernel is not really the same as a proper hypervisor, but have never really dug deep into why and what the various tradeoffs are.
There is no real connection to UML here. Hardware virtualization (Intel VT, AMD-V) are much faster in practice and also don't require the guest operating system to be heavily modified. So besides as curiosity or test vehicle, approaches like UML are pretty dead.
Thanks.
So, slow as it may be, the win for UML (which seems to still have a heartbeat) is that it can run on uP without any specific virtualisation capabilities, right? If I could run Linux on a Z80/6502 then in theory I could run a virtualised Linux on a Z80/6502.
Yes - plus the original win of UML was also being able to run virtual instances on a kernel without proper virtualization capabilities.
In the early 2000s people used to use UMLs as a hosting platform - they didn't have the same security isolation as a proper VM (or even, necessarily, of a container) though.
How do containers have better security isolation than UML?
When I tinkered with UML I think it was prior to cgroups (2007) [0] so my guess is that escaping the UML instance was easier.
[0] https://en.wikipedia.org/wiki/Cgroups
The “original” UML is/was, I believe, NetBSD running as a “rump kernel” and something that virtualization of the actual kernel does not, directly and on its own, fill the shoes of.
UML is older than NetBSD rump kernels.
I stand corrected.
I want to know what Linux a386 was. Couldn't ever really understand what it did.
Yes, quite different. It's basically a project that allows you to use Linux as a hypervisor. A very popular project.
https://en.m.wikipedia.org/wiki/Kernel-based_Virtual_Machine
Thanks!
While an over simplification, here is the context
There are two large hypervisors in the Linux world.
Xen, which extends the kernel to support virtual CPUs with time slices.
KVM, which assigns each virtual core a process that uses the Linux scheduler.
When a hardware vm vcpu core is preempted there is vmexit call that has to reset registers etc... and it is expensive.
Xen is what legacy AWS instances ran on and has advantages for being fair to guests is an easier task.
KVM has the advantage of gaining the benefits of the Linux scheduler which is red black tree based and well optimized.
When a new CPU comes out for example, KVM gains support from the upstream while Xen has to support it themselves.
Once technology like cgroups improved the benefits of letting your thread complete and not be preempted due to the time slice expiring avoided the cost of vmexit.
In theory, leveraging the inherently optimized core Linux features is what will also benefit virtualbox.
Most people who use KVM are using an abstraction layer like libvirt that hides how it is implemented.
In fact if you look at the processes you will see qemu even if KVM is how it is implemented.
Happy 10000 Day!
https://xkcd.com/1053/
It means Kernel-Based Virtual Machine, a VM engine dating back to 2007: https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine
Any chance this gets upstreamed?
I’d actually much, much rather see it set up as a proper fork (rebranded etc.), and then the features that Oracle extorts people with in the “free” but not actually free extension pack (like USB pass through) re-implemented and included directly with no ‘extension’ required.
Much of the reason I refuse to use Virtualbox for anything is how scummy Oracle is.
At a previous employer Oracle sent a nastygram because they saw downloads of the VirtualBox extensions pack (which is free to download, but requires a license) coming from our IP block. This despite the fact that we were a big Oracle customer (tons of Oracle DBs; granted, we hadn't purchased VirtualBox licenses). I'd rather not deal with a vendor that's that antagonistic.
FTFY.
Having spent time at another large Oracle customer that was later acquired by Oracle I've seen how they do this internally too. And you can't just send it off to Legal to write back that "we didn't actually use it"...
This employer has a massive network with a bunch of guest networks (it's a hospital chain), so I thought they could just say there was no way to know whether it was employees or not. Of course, IIRC they just paid the ransom and added new restrictions on how employees could manage their work machines.
Haha. I understand the sentiment. That's a pretty large effort though and needs some funding as well.
I mean it's Oracle we're talking about here.
"What you think of Oracle, is even truer than you think it is. There has been no entity in human history with less complexity or nuance to it than Oracle."
Bryan Cantrill
https://www.youtube.com/watch?v=-zRN7XLCRhc&t=1980s
Thanks. This is pure comedy gold. In particular, that part about the acquisition by Oracle (from minute 33)
Oracle already had an unfinished and broken KVM backend in the code that was not exposed. Whether they incorporate this polished KVM backend is anyone's guess at this point.
How is it broken ? I used it for a day recently (from ubuntu repo, windows guest) and it worked ok.
The KVM backend in vanilla VBox cannot be activated without changing the code. If you tried it, you tried the vanilla VBox hypervisor (vboxdrv) instead of KVM.
I'm quite inexperienced with Virtualization. Are there benefits to kernel based virtual machines beyond (what I assume is the primary benefit) performance?
I wouldn't call it kernel based. It's not like this is an in-kernel emulation. I would stick with hardware virtualization.
KVM is a userspace API, and kvm-intel/kvm-amd are the drivers for the hardware.
You will be using hardware features. That's also why it is in the kernel: nothing but the kernel should have full unlimited access to the CPU to set this up.
So you could say it must be in kernel to keep the kernel secure. And the performance benefit is "just" exposed hardware features.
The kernel does not provide additional things. As far as I understand: you set up a dedicated memory space and handle traps that halt the execution e.g. when the VM talks to the PCI bus. (It's been a while since I looked this up)
But you need the pieces, especially virtual PCI devices. That's where qemu or VirtualBox enter the scene (or minimalist systems like firecracker). They provide a repository of virtual hardware and all the auxiliary methods to boot a virtual machines. You also need to emulate something like a BIOS or UEFI.
You can think of it as your CPU removing the need to emulate the very same CPU (and a memory controller). You still need to emulate the rest though! But running on the same CPU removes most performance penalties. You run at native speed.
Newer generations can even nest this. Having virtual machines in virtual machines. That's mostly useful for cloud environments so that the cloud provider can run kvm based VMs and you are still able to run VMs inside that VM.
if in the past they already were using a kernel module then your reply doesn't explain anything.
So they went from using hardware virtualization (provided by intel/amd) with their kernel module to the KVM one. I don't know which benefits it brings
Without being at all up to date on the current state of things, the Virtual Box third party kernel module was historically of famously poor quality[1], even putting aside the general pains of third-party kernel modules.
[1] https://www.phoronix.com/news/OTk5Mw
there's no "keeping the kernel secure" and "allowing access to the hardware".
in security research, you either run your samples in qemu without even kvm or you don't.
That clarifies some things and gives me some tails to chace after! Thanks for the detailed response!
You can’t use virtualbox’s kernel module and kvm at the same time. This basically means that you can’t use virtualbox and qemu at the same time.
If you use virtualbox with this new backend, you can use it concurrently with qemu (and a few other virtualisation tools).
KVM is also part of Linux itself, so there’s a lot less haste with setting it up.
virtualbox drivers are/were a constant source of kernel panics on MacOS and Linux too, so that should not be underestimated.
IIRC they also disabled ASLR kernel wide.
Additionally, and perhaps less important: USB3 is a commercial feature of virtualbox, there are stories of companies getting C&D letters (or Audits/Invoices) from Oracle because a developer had installed the virtualbox extensions..
Starting with VirtualBox 7, USB3 no longer requires the commercial extension pack but is part of the OSE release. See https://github.com/cyberus-technology/virtualbox-kvm/blob/de...
The name KVM is a bit confusing. It doesn't do anything fundamental different than VirtualBox. The difference is that KVM comes by default with any Linux. VirtualBox ships it's own Linux kernel module for that. That has drawbacks. You typically can't use the newest Linux or the newest features.
KVM also always has the hottest new (performance-relevant) features, because Intel and AMD will always build their hot stuff into KVM first.
Honest questions:
How does this work in licensing terms? If VB foss enough?
Do you expect Oracle to merge this?
If oracle doesn’t merge this, will you keep on maintaining it, potentially forking VirtualBox?
It's as FOSS as the VirtualBox open source edition.
That would be nice, but I wouldn't hold my breath. Oracle gonna Oracle.
We don't intend to fork VirtualBox. VBox has a somewhat modular architecture where you can plug-in different hypervisor backends. That's what we did. It's not as modular, but our changes to core VirtualBox code is very small.
As far as our plans go, we are pretty open at this point. We are very interested to get to know people that find this useful!
This would be useful for anyone who needs to run bleeding edge linux kernels. Most other hypervisors have poor support when you're pinned to testing distros (For security reasons). KVM and virt-manager are uh not exactly user friendly, so being able to use the KVM backend and always be compatible with the new kernels while having the UX and UI of vbox is actually a very huge deal. This is one of the main reasons I really really hope you all manage to upstream this.
I built and tried this, it worked great, so excellent work there. I found the processor CPU core counts being grayed out unpleasant, it's not clear to me how I am supposed to adjust my core and ram values now, maybe document this?
That's odd about the core count. I only get that behavior if my host system only has 1 CPU to begin with (tested in qemu/KVM with nesting). Could you comment on your host system parameters a bit?
This is definitively not expected. You could also try setting the CPU/Mem configuration via VBoxManage. Maybe you get a good error message then.
`VBoxManage modifyvm <vm_name> --cpus <number of cpus>`
`VBoxManage modifyvm <vm_name> --memory <amout of memory in MB>`
Why? How does this generate revenue for CyberusTech? What is the long term goal?
Those are good questions. I don’t understand why you were downvoted.
To answer: We are offering service contracts and contract engineering services around virtualization, KVM and a couple of other topics.
The long term goal for this specific project is to support the security goals our customers have and to enable a couple of KVM features in Virtualbox as well. We plan to keep this updated with upstream Virtualbox and KVM.
Please upstream this code. That ensures this work will live forever with better maintenance without you all having to chase upstream vbox AND will be easier to justify use if it's an included vbox component. I would find it difficult to justify using this to my bosses "lets just go use this custom fork of vbox" isn't going to fly for most.
so, the plan is to take on docker hub?
edit: nevermind. i assumed you were a team inside oracle.
We're offering commercial support. We can also help with graphics virtualization and other topics (e.g. performance tuning and automated testing in real world scenarios).
As a note, the first character of your COPYING file in the repo seems to be typo-d. ;)
Ooops. Will fix. :) Thanks!
That's already b0rken in the Oracle sources.
Oops. I probably should have checked before mentioning it anyway. ;)
So can this run/import existing vbox VMs?
Yes, we are switching between vanilla VBox and KVM VBox during development quite often and the VMs are fine with it.