Lightweight Virtualization: the Container Ecosystem and Firecracker MicroVMs for Serverless

Virtualization is a core component of cloud computing and the key technology to optimize usage of hardware's resources. While it has been around for decades, new innovations continue to improve its efficiency and performance for modern workloads.

In this blog post, we provide a brief overview of virtualization and its history as well as explain how Firecracker and lightweight virtualization are fueling modern deployments. Firecracker is an open-source technology used at Koyeb to power serverless workloads.

This is our first post one about virtualization and the container ecosystem. It's also part of our series about Firecracker, our first Firecracker post explains how the technology works and the second covers why we think it is so great.

Specialized Virtualization Technologies for Different Workloads

Virtualization technology creates multiple isolated environments, also known as virtual machines (VMs), from a single physical resource. The software responsible for this process is known as the hypervisor, or Virtual Machine Monitor (VMM). A hypervisor sits on top of a piece of hardware and distributes its resources to the VMs that the hypervisor generates and manages.

The history of virtualization is rooted in the early days of computing. The first success is commonly situated to be during the mainframe days of the 60s. Computer scientists at IBM were seeking to reduce the expensive costs of running computers and overall maximize the efficiency of those computer resources. Their innovations led them to be able to run multiple isolated systems at the same time with the CP-40 system.

The two main categories of Hypervisors

There are two types of hypervisors:

• Type 1 Hypervisors - Bare-metal hypervisors that run on the physical hardware and do not have an underlying OS. Examples: VMware ESXi and Microsoft Hyper-V.
• Type 2 Hypervisors - Hosted hypervisors that run on top of a standard Host OS, which is running on a piece of physical hardware. Examples: VMware Workstation and VirtualBox.

Type 1 Hypervisors are the ones used in the server world as they are purposely designed for performance. On the other side, Type 2 Hypervisors are mostly used on PCs where individuals mostly use standard operating systems.

In the cloud computing industry, Type 1 Hypervisors are the norm. They are used to increase efficiency. At Koyeb, we currently use Fedora CoreOS with Linux KVM and Firecracker as a Type 1 Hypervisor.

Virtualization in the Cloud Computing Industry

As the cloud computing industry heavily relies on virtualization, different kind of virtualization have emerged inside of Type 1 Hypervisors. There are three main approaches to server virtualization that have been around since the 2000s.

	Kernel-level	Paravirtualization	Hardware-assisted
Projects	Linux cgroups, OpenVZ	Xen	KVM and also Xen
Guest Kernel	Same as Host	Modified	Standard
Security	Low by default	High	High
Performance Overhead	Low	Medium	High
Boot-time	Milliseconds	10s of seconds	10s of seconds

1. Kernel-level virtualization

   Kernel-level virtualization like Linux cgroups is lightweight and has fast startup times, but low security and poor isolation by design. If you want to run multi-tenant workloads, you will need to combine kernel-level isolation with other technology to provide multi-tenant workloads to achieve satisfactory levels of security and isolation. If you want to learn more about kernel-level security, check out Jessie Frazelle's blog post Containers, Security, and Echo Chambers.

2. Paravirtualization

   Paravirtualization uses a modified kernel approach to virtualization. It provides a high level of security with more overhead and longer boot-times than a kernel-level approach. Paravirtualization like Xen, an open-source hypervisor built-in the Linux kernel, still powers a large part of AWS today.

3. Hardware-assisted virtualization

   Hardware-assisted virtualization provides the best isolation for workloads, but it requires the support of the processor, and startup times are longer. KVM is a type of hardware-assisted virtualization. The cloud computing industry heavily relies on this kind of virtualization. Since the 2010s, most IaaS providers have been using this kind of technology thanks to the emergence of specialized instructions on x86 architectures.

VMs using hardware-assisted virtualization are great for enabling multi-tenant workloads by creating truly secure and isolated environments however modern apps require even faster startup times and more efficient distribution of computing resources.

The Container Ecosystem

The incentives advancing virtualization technology today remain similar to those decades ago: the quest for even more performant and efficient ways to use physical resources continues.

Today, containers are a popular deployment choice for modern apps because they provide consistent deployments across devices with a standardized unit of software and low resource overhead.

Since the emergence of the container in the early 2010s, the container ecosystem has significantly grown, now consisting of many components divided into several categories such as container images, runtimes, and orchestrators. Simply put:

• Container images focus on packaging. Examples: Docker and the Open Container Initiative.
• Container runtimes focus on container execution. Examples: containerd and gVisor.
• Container orchestrators handle multi-node container management. Examples: Kubernetes and Apache Mesos.

Even though the ecosystem has flourished, security, isolation, and performance remain a core concern, especially in multi-tenant environments similar to the Koyeb Serverless Platform.

Container Runtimes and Virtualization

This is where container runtimes come into the pictures since container runtimes execute containers. The main three approaches to executing containers in an isolated manner use the different types of server virtualization:

• Hardened cgroups, which let you filter system calls your apps can do with tools like SELinux, Seccomp, and Apparmor. Depending on the policies you define, applications can directly perform syscalls to the host kernel. This is the most lightweight runtime approach.
• Paravirtualization approach, like gVisor and IBM Nabla, which are application kernels. These solutions use custom-designed kernels to isolate the application and intercepts all of its system calls. These custom kernels then call to the host kernel.
• Hardware-assisted virtualization like Firecracker and Cloud-hypervisor. These solutions are like a virtual machine, but only use the computing resources essential for your workload. They let you run a Linux Kernel inside of a VM using KVM. Both these runtimes are based on rust-vmm, a fork of Chrome OS VMM (crosVM).

Firecracker and Lightweight Virtualization is a Solution for Serverless Workloads

Firecracker was designed to run serverless functions and containers securely and efficiently. It is a lightweight virtualization technology that uses Linux KVM to provision and manage microVMs.

KVM transforms Linux into a Type 1 hypervisor that enables you to run multiple isolated environments. Since KVM is built into Linux, it includes all the operating system-level components that a hypervisor needs.

Firecracker microVMs are more lightweight compared to other VMs using QEMU thanks to Firecracker's minimalist design. Firecracker was written in Rust, a compiled language that is memory-efficient, stable, and fast. During the design, unnecessary devices and guest functionality were dropped in order to achieve a smaller memory footprint and reduce security risks.

MicroVM technology enables workloads to be as secure and isolated as hardware-assisted VM solutions while providing the density and speed offered by container technologies. If you want to learn more, you can read our Firecracker MicroVMs: Lightweight Virtualization for Containers and Serverless Workloads or 10 Reasons Why We Love Firecracker MicroVMs blog post.

Firecracker: A Key Component of the Koyeb Stack

At Koyeb, we've selected Firecracker to securely run our users' serverless workloads. As a cloud service provider, Firecracker enables us to provide multi-tenancy security, fast startup times for scale-to-zero and autoscaling features, and increased density compared to earlier virtualization technologies. Securing and powering serverless deployments, Firecracker is a vital component of the Koyeb stack.

Koyeb is the next-generation and developer-friendly serverless platform where you can deploy web apps and services, APIs, event-driven functions, background workers, and more.

See the benefits of going serverless, get started with a free account today!

Here are some useful resources to get you started:

• Koyeb Documentation: Learn everything you need to know about using Koyeb.
• Koyeb Tutorials: Discover guides and tutorials on common Koyeb use cases and get inspired to create your own!
• Koyeb Community: Join the community chat to stay in the loop about our latest feature announcements, exchange ideas with other developers, and ask our engineering teams whatever questions you may have about going serverless.

If you want to learn more about Firecracker and microVMs, check out our Firecracker MicroVMs: Lightweight Virtualization for Containers and Serverless Workloads and 10 Reasons Why We Love Firecracker MicroVMs blog posts.