Container Engine vs Container Runtime

Linux Kernel supports two interesting features that laid the foundation for the Container
Technology.

1. Namespace and
2. Control Group a.k.a CGroups

Latest version of Linux kernels supports about 8 types of Namespaces.

1. PID namespace
2. Network namespace
3. Mount namespace
4. IPC namespace
5. UTS namespace
6. User namespace
7. CGroup namespace
8. Time namespace

Namespace helps in isolating the resources used by a process from the other processes
running in different namespaces.

Control Groups helps in applying container specific resource usage restrictions.

Different vendors developed their own Container engines leveraging the Namespace and
Control Groups Linux Kernel Features.

Docker is one of the most popular Container Engine. However, there are many other
Container Engines. To name a few

 LXD
 Rkt
 CRI-O
 Podman

Docker is a Container Engine developed by Docker Inc organisation. Docker is developed in

Go programming language. Docker comes in 2 flavours

1
 1. Community Edition and
2. Enterprise Edition

Docker depends on containerd Container Runtime. Container Runtime manages

 container life-cycle
 image transfer and storage
 container execution and supervision
 storage and network attachments

Containerd Container Runtime depends on runc and hcsshim. Runc is a command line tool
that helps creating and running containers in Linux. While hcsshim, helps in creating and
managing Windows Containers. Containerd is not meant to be used by end-users directly.
Typically, the Container Engines interact with the Container Runtime to create and manage
containers.

runcis used by containerd, Podman, CRI-O also by Docker indirectly as Docker depends on
containerd.

What does the Container Engine do then?

 handle user-inputs from end-users and REST clients

 downloading container image from Container Registry
 extracting the container image on disk once downloaded
 massaging data received from user or REST clients in a way that can be passed to
Container Engines
 preparing container mount points
 interacting with Container Engines

In short, Container Engines are the ones, the end-users use to manage images and containers.
While the Container Engines depends on Container Runtime to do the low-level container and
image management stuffs.

2
Container Runtime Interface (CRI): Past,
Present, and Future
Learn about the need for the Container Runtime Interface and the history of container
runtimes, how CRI is used today, and how Docker’s lack of support for CRI will impact your
project.

What is Container Runtime Interface?

Container runtime interface (CRI) is a plugin interface that lets the kubelet—an agent that
runs on every node in a Kubernetes cluster—use more than one type of container runtime.
Container runtimes are a foundational component of a modern containerized architecture.

CRI was first introduced in Kubernetes v1.5. Prior to the introduction of the CRI, rkt and
Docker were directly integrated into the source code of the kubelet. This made it difficult to
integrate new container runtimes with Kubernetes. CRI enables Kubernetes users to easily
make use of multiple container runtimes, and enables developers of container runtimes to
easily integrate them with the Kubernetes ecosystem.

In this article:

 A Brief History of Container Runtimes

 Why Does Kubernetes Need CRI?
 What are runc and the Open Container Initiative (OCI)?
 Which Container Runtime Engines Support CRI?
 Does Docker Support CRI?
 Container Runtime Security with Aqua

A Brief History of Container Runtimes

Docker and Kubernetes have gained huge popularity over the past few years, but the concept
of containerization is not new. Here is a brief history of the evolution of container runtime
technology, leading up to the introduction of the CRI standard:

2008: CGroups added to Linux

cgroups were introduced into the Linux operating system, and a project called Linux
Containers (LXC) used groups and namespaces to create an isolated environment for
running Linux applications. At roughly the same time, Google began a parallel
containerization project called LMCTFY.

2013: Release of Docker

Docker Container was released, built on top of LXC. Its main innovation was the ability to
easily define container images, which allowed users to package containers and move them
consistently between physical machines. 

2015: Kubernetes released and CNCF takes off

3
  Kubernetes version 1.0 was released, and the Cloud Native Computing Foundation (CNCF)
was founded to promote container and serverless technology. Google donated the
Kubernetes project to the CNCF. 
 Just as Kubernetes was taking off, the Open Container Initiative (OCI) was founded, with the
goal of creating a governance structure for the burgeoning container ecosystem. The OCI
created a standard specification for containers, known as the OCI Runtime Specification. 
 A new tool called runc was built in line with OCI specifications. It became a standard
component that interprets the OCI Runtime Specification and makes it possible to run
containers. runc is a low-level component used throughout the container ecosystem, for
example by popular container runtimes Docker, CRI-O and Kata Containers.

Why Does Kubernetes Need CRI?

To understand the need for CRI in Kubernetes, let’s start with a few basic concepts:

 kubelet—the kubelet is a daemon that runs on every Kubernetes node. It implements the
pod and node APIs that drive most of the activity within Kubernetes. 
 Pods—a pod is the smallest unit of reference within Kubernetes. Each pod runs one or more
containers, which together form a single functional unit.
 Pod specs—the kubelet read pod specs, usually defined in YAML configuration files. The pod
specs say which container images the pod should run. It provides no details as to how
containers should run—for this, Kubernetes needs a container runtime.
 Container runtime—a Kubernetes node must have a container runtime installed. When the
kubelet wants to process pod specs, it needs a container runtime to create the actual
containers. The runtime is then responsible for managing the container lifecycle and
communicating with the operating system kernel.

In the early days of Kubernetes, the only container runtime was Docker. A bit later,
Kubernetes introduced rkt as an additional option. However, Kubernetes developers quickly
realized that this was problematic:

 Tightly coupling Kubernetes to specific container engines could break Kubernetes, as

container runtimes and Kubernetes itself evolved.
 It would be difficult to integrate new container engines with Kubernetes, because this
requires a deep understanding of Kubernetes internals. This would create an effective
monopoly on container runtimes within Kubernetes.

The solution was clear: creating a standard interface that would allow Kubernetes—via the
kubelet—to interact with any container runtime. This would allow users to switch out
container runtimes easily, combine multiple container runtimes, and encourage the
development of new container engines. 

In 2016, Kubernetes introduced the Container Runtime Interface (CRI), and from that point
onwards, the kubelet does not talk directly to any specific container runtime. Rather, it
communicates with a “shim”, similar to a software driver, which implements the specific
details of the container engine.

4
What are runc and the Open Container Initiative (OCI)?

The Open Container Initiative (OCI) provides a set of industry practices that standardize the
use of container image formats and container runtimes. CRI only supports container runtimes
that are compliant with the Open Container Initiative. 

The OCI provides specifications that must be implemented by container runtime engines. Two
important specifications are:

 runc—a seed container runtime engine. The majority of modern container runtime
environments use runc and develop additional functionality around this seed engine.
 OCI image specification—OCI adopted the original Docker image format as the basis for the
OCI image specification. The majority of open source build tools support this format,
including BuildKit, Podman, and Buildah. Container runtimes that implement the OCI runtime
specification can unbundle OCI images and run its content as a container. 

Which Container Runtime Engines Support CRI?

The following table shows the most common container runtime environments that support
CRI, and thus can be used within Kubernetes, their support in managed Kubernetes platforms,
and their pros and cons.

Support in
Container
Kubernetes Pros Cons
Runtime
Platforms

Google Tested at huge scale, used in all

Kubernetes Docker containers. Uses less
No Docker API socket.Lacks
Containerd Engine, IBM memory and CPU than
Docker’s convenient CLI tools.
Kubernetes Docker.Supports Linux and
Service, Alibaba Windows

Mainly usage within Red Hat

Red Hat Lightweight, all the features needed
platformsNot easy to install on
OpenShift, SUSE by Kubernetes and no more.UNIX-
CRI-O non Red Hat operating
Container as a like separation of concerns (client,
systemsOnly supported in
Service registry, build)
Windows Server 2019 and later

Provides full virtualization based on

QEMUImproved securityIntegrates Higher resource utilizationNot
Kata
OpenStack with Docker, CRI-O, containerd, and suitable for lightweight container
Containers
FirecrackerSupports ARM, x86_64, use cases
AMD64

New project, less mature than

Accessible via direct API or
AWS other runtimesRequires more
All AWS services containerdTight kernel access using
Firecracker manual steps, developer
seccomp jailer
experience still in flux

5
Does Docker Support CRI?

The short answer is no. In the past, Kubernetes included a bridge called dockershim, which
enabled Docker to work with CRI. From v1.20 onwards, dockershim will not be maintained,
meaning that Docker is now deprecated in Kubernetes. Kubernetes currently plans to remove
support for Docker entirely in a future version, probably v1.22. 

However, Docker images will continue to work in Kubernetes, because they are based on the
OCI image specification.

Here is what the deprecation of Docker in Kubernetes means for you, depending on your use
case:

 Kubernetes end-users do not need to change their environment, and can continue using
Docker in their development processes. However, developers should realize that the images
they create will run within Kubernetes using other container runtimes, not Docker.
 Users of managed Kubernetes services like Google Kubernetes Engine (GKE) or Elastic
Kubernetes Service (EKS) need to ensure worker nodes are running a supported container
runtime (i.e. not Docker). Customized nodes may need to be updated.
 Administrators managing clusters on their own infrastructure must reinstall container
runtimes on their nodes (if they are currently running Docker) to avoid clusters from
breaking, when Docker support is removed in the future. Kubernetes nodes should run
another, CRI-based container runtime, like condainerd or CRI-O.

Container Runtime Security with Aqua

When it comes to workload protection at runtime – prevention and detection isn’t enough.
True runtime security means stopping attacks in progress. That means enforcement that
happens after the workload has started. This does not mean policy controls that are applied
before a workload starts. Why does this matter? Because if you think you are stopping attacks
in a production environment, but all you are doing is applying a policy like OPA, for example,
you are not achieving the intended control and outcome of protecting against real attacker
behavior in cloud native environments. Shift-left is only prevention, which we all know is
important, but just one layer of a true defense-in-depth approach.  With Aqua, importantly,
whether the method is mitigating an exploit or stopping command and control behavior, the
workload security policies are granular and can be used without downtime or binary actions to
only allow or kill an image.

6
3 Types of Container Runtime and the
Kubernetes Connection
Discover the three types of container runtime, and understand how container runtimes are the
silent engine under the hood of container orchestrators like Kubernetes

What Is a Container Runtime?

A container runtime, also known as container engine, is a software component that can run
containers on a host operating system. In a containerized architecture, container runtimes are
responsible for loading container images from a repository, monitoring local system
resources, isolating system resources for use of a container, and managing container
lifecycle. 

Common container runtimes commonly work together with container orchestrators. The
orchestrator is responsible for managing clusters of containers, taking care of concerns like
container scalability, networking, and security. The container engine takes responsibility for
managing the individual containers running on every compute node in the cluster.

Common examples of container runtimes are runC, containerd, Docker, and Windows
Containers. There are three main types of container runtimes—low-level runtimes, high-level
runtimes, and sandboxed or virtualized runtimes.

In this article:

 3 Types of Container Runtimes

 How Kubernetes Works with Container Engines
 The Container Runtime Interface (CRI)
 Container Runtime Security with Aqua

3 Types of Container Runtimes

1. Low-Level Container Runtimes

The Open Container Interface (OCI) is a Linux Foundation project started by Docker, which
aims to provide open standards for Linux containers. The main open source project developed
by OCI is runC, released in 2015. runC is a low-level container runtime that implements the
OCI specification. It forms the basis for many other container runtime engines.

The OCI provides runtime specifications. Runtimes implemented according to OCI specs are
called low-level runtimes, because the primary focus is on container lifecycle management. 

Native low-level runtimes are responsible for creating and running containers. Once the
containerized process runs, the container runtime is not required to perform other tasks. This
is because low-level runtimes abstract the Linux primitives and are not designed to perform
additional tasks. 

The most popular low-level runtimes include:

7
  runC—created by Docker and the OCI. It is now the de-facto standard low-level container
runtime. runC is written in Go. It is maintained under moby—Docker’s open source project.
 crun—an OCI implementation led by Redhat. crun is written in C. It is designed to be
lightweight and performant, and was among the first runtimes to support cgroups v2.
 containerd—an open-source daemon supported by Linux and Windows, which facilitates the
management of container life cycles through API requests. The containerd API adds a layer of
abstraction and enhances container portability.

2. High-Level Container Runtimes

Examples of popular high-level runtimes include:

 Docker (Containerd)—the leading container system, offering a full suite of features, with free
or paid options. It is the default Kubernetes container runtime, providing image
specifications, a command-line interface (CLI) and a container image-building service. 
 CRI-O—an open-source implementation of Kubernetes’ container runtime interface (CRI),
offering a lightweight alternative to rkt and Docker. It allows you to run pods using OCI-
compatible runtimes, providing support primarily for runC and Kata (though you can plug-in
any OCI-compatible runtime). 
 Windows Containers and Hyper-V Containers—two lightweight alternatives to Windows
Virtual Machines (VMs), available on Windows Server. Windows Containers offer abstraction
(similar to Docker) while Hyper-V provides virtualization. Hyper-V containers are easily
portable, as they each have their own kernel, so you can run incompatible applications in
your host system. 

3. Sandboxed and Virtualized Container Runtimes

The OCI includes specifications for sandboxed and virtualized implementations:

 Sandboxed runtimes—provide increased isolation between the containerized process and

the host, as they don’t share a kernel. The process runs on a unikernel or kernel proxy layer,
which interacts with the host kernel, thus reducing the attack surface. Examples include
gVisor and nabla-containers. 
 Virtualized runtimes—provide increased host isolation by running the containerized process
in a virtual machine (through a VM interface) rather than a host kernel. This can make the
process slower compared to a native runtime. Examples include kata-containers and the now
deprecated clearcontainers and runV.

Related content: read our guide to leading container engines (coming soon) ›

How Kubernetes Works with Container Engines

Container orchestrators like Kubernetes are responsible for managing and scaling
containerized workloads. In Kubernetes, the kubelet is an agent that runs on every computing
node. It receives commands specifying what containers should be running, and relays them to
a container runtime on the node. It also collects information from the container runtime about
currently running containers, and passes it back to the Kubernetes control plane.

The kubelet communicates with the container engine through the standard Container Runtime
Interface (CRI), described in the next section.

8
When Kubernetes collaborates with container engines, the central responsibility of the
container engine is to give the orchestrator a way to monitor and control the containers that
are currently running. It deals with: 

 Verifying and loading container images

 Monitoring system resources
 Isolating and allocating resources
 Container lifecycle management

To carry out these activities, the engine draws on the resources required to run a container. It
makes use of standardized interfaces to coordinate the resources, including:

 Container Storage Interface (CSI)—regulates how containers access storage equipment

 Container Networking Interface (CNI)—specifies how containers communicate over a
network

The Container Runtime Interface (CRI)

To deal with the increasing difficulty of incorporating multiple runtimes into Kubernetes, the
community specified an interface—particular functions that a container runtime would need
to put in place on behalf of Kubernetes—named the Container Runtime Interface (CRI). 

This corrected the issue of having extensive integration between container runtimes and the
Kubernetes codebase, which became difficult to maintain, and made it more difficult to
develop new container runtimes that would support Kubernetes. 

The CRI interface also makes it clear to developers of container runtimes, which functions
need to support to work with Kubernetes. The primary functions are:

 The runtime needs to be capable of starting/stopping pods

 The runtime must deal with all container operations within pods—start, pause, stop, delete,
kill
 The runtime should handle images and be able to retrieve them from a container registry
 The runtime should provide helper and utility functions around metrics collection and logs

Container Runtime Security with Aqua

Aqua’s container runtime security controls protect workloads from attack using a combination
of system integrity protection, application control, behavioral monitoring, host-based
intrusion prevention and optional anti-malware protection. 

9
10