Cloud Native Virtualization Technologies

Posted on

I’ve been investigating cloud native virtualization technologies in the past week. More specifically, I tried to:

  1. create/poweron/ssh into/poweroff/delete a virtualmachine by using qemu/kvm;
  2. create/poweron/ssh into/poweroff/delete a virtualmachine through libvirt+esxi;
  3. create/poweron/ssh into/poweroff/delete a virtualmachine through libvirt+qemu/kvm;
  4. create/ssh into/suspend/delete a container through libvirt+lxc;
  5. try out kata-container by following its quickstart guide:
    1. build the kernel;
    2. build the rootfs;
    3. add kata-container as an extra runtime to docker;
    4. run docker to start a container through kata-container so the container is actaully running as a vm;
    5. only qemu/kvm is used;
  6. try out firecracker by following its quickstart guide which basically boots up a vm by using provided kernel and rootfs;
  7. try out ignite by following its quickstart guide to import a kernel from a docker image, and deploy a container vm, ssh into it and delete it;
  8. also read some documentations on:
    1. kubevirt
    2. virtlet

After which, I get a rough idea of:

  1. what is libvirt;
  2. how is kubevirt, virtlet architectured and using libvirt;
  3. what is kata-container, firecracker, cloud-hypervisor, qemu/kvm, lxc/lxd;

To summary:

  1. kubevirt and virtlet all leverage libvirt to provision vms. They are mainly using the qemu/kvm mode for libvirt;
    1. kubevirt includes an operator, a handler per K8s node and a launcher per
      vm;
    2. virtlet instead implements a CRI proxy which calls virtlet process to provision the
      vm besides the inband call to docker;
  2. firecracker now provides a binary that allows you to boot a microvm with provided kernel and rootfs fast. It opens up a socket and listens of REST requests;
  3. ignite is a thin wrapper over firecracker as far as I call tell, including the abilities to easily manage kernel images, rootfs in an organized, docker-image way. But it’s not OCI-compliant, so it can only be used standalone now. Besides, it seems it’s integrated with weave net cni by default;
  4. kata-container is pretty well interfaced compared with ignite. It’s a OCI-compliant container runtime so it can be easily swapped in to replace runc in docker, also through cri-o, it can also be used in K8s as the runtime. On the backend, it leverages virtcontainer library to abstract Hypervisor calls into an interface. qemu/kvm, cloud-hypervisor, firecracker, acrn are supported now according to the doc;
  5. cloud-hypervisor and firecracker are built based on rust-vmm library, which includes many useful components to build vmm quickly, effectively and securely. Among them, it includes a kvm ioctl wrapper: kvm-bindings.

All my findings are recorded into my personal wiki and I’ll see if I can host it somewhere.

The following picture demonstrates their relationships:
Knowledge graph