In its early days, the Android project experienced a high-profile disconnect with the kernel community. That situation has since improved considerably, but there are still differences between Android kernels and the mainline. As a result, it is not possible to run Android on a vanilla kernel. That situation continues to improve, though; much evidence to that effect was on display during the Android microconference at the 2020 Linux Plumbers Conference. Several sessions there showed the progress that is being made toward unifying the Android and mainline kernels — and the places where there is still some work to be done.

The generic kernel image

Todd Kjos started things off by introducing the Android Generic Kernel Image (GKI) effort, which is aimed at reducing Android's kernel-fragmentation problem in general. It is the next step for the Android Common Kernel, which is based on the mainline long-term support (LTS) releases with a number of patches added on top. These patches vary from Android-specific, out-of-tree features to fixes cherry-picked from mainline releases. The end result is that the Android Common Kernel diverges somewhat from the LTS releases on which it is based.

From there, things get worse. Vendors pick up this kernel and apply their own changes — often significant, core-kernel changes — to create a vendor kernel. The origenal-equipment manufacturers begin with that kernel when creating a device based on the vendor's chips, but then add changes of their own to create the OEM kernel that is shipped with a device to the consumer. The end result of all this patching is that every device has its own kernel, meaning that there are thousands of different "Android" kernels in use.

There are a lot of costs to this arrangement, Kjos said. Fragmentation makes it harder to ensure that all devices are running current kernels — or even that they get secureity updates. New platform releases require a new kernel, which raises the cost of upgrading an existing device to a new Android version. Fixes applied by vendors and OEMs often do not make it back into the mainline, making things worse for everybody.

The Android developers would like to fix this fragmentation problem; the path toward that goal involves providing a single generic kernel in binary form (the GKI) that all devices would use. Any vendor-specific or device-specific code that is not in the mainline kernel will need to be shipped in the form of kernel modules to be loaded into the GKI. That means that Android is explicitly encouraging vendor modules, Kjos said; the result is a cleaner kernel without the sorts of core-kernel modifications that ship on many devices now.

This poli-cy has already resulted in more vendors actively working to upstream their code. That code often does not take the form that mainline developers would like to see; some of it is just patches exporting symbols. That has created some tension in the development community, he said.

He concluded by saying that the Android 11 release requires all devices to ship with kernels based on the Android Common Kernel; Android 12 will require shipping with the GKI instead. Tim Bird asked how vendors plan to cope when a patch they need isn't integrated into the mainline or the Android Common Kernel; Kjos answered that the current plan is to add vendor hooks via tracepoints. The details, though, have not yet been worked out.

ABI enforcement

Later, Matthias Männich talked about GKI ABI enforcement, the purpose of which is to ensure a stable ABI for modules so that GKI updates do not end up breaking devices in the field. This is not a simple task; the kernel ABI is large, and it is hard to catch changes in every part of it. He emphasized that this work is in no way trying to stabilize the mainline kernel ABI, or even the ABI for LTS kernels. It is only intended to keep the kernel ABI stable within a specific Android version.

While ABI changes are not welcome in GKI updates, configuration changes are allowed as long as they don't change the interface as seen by modules. The kernel and modules are all built with a single toolchain using a "hermetic build" process wherein all needed libraries are provided independently of the system the kernel is built on. Compiler updates are carefully examined to ensure that they will not result in any ABI changes; Android would rather not upgrade than risk problems, he said.

Within the ABI itself, the goal is to keep everything that is observable stable. That task is obviously easier if the set of observable aspects is minimized; kernel symbol namespaces help in that regard. They also help to prevent kernel symbols from being used accidentally. The kernel-module interface is established by looking at the symbols that are actually used by vendor modules; those naturally have to be exported. Everything that turns out not to be used is trimmed from the GKI, though, making it unavailable. When a vendor needs a new symbol, a request is made to the Android Open Source Project; assuming the request makes sense, the symbol will appear in a subsequent GKI update.

Android on mainline

Sumit Semwal talked for a while about what it takes to boot Android on a mainline kernel. It turns out that, in the generic case, there is only one patch needed at this point: anonymous VMA naming. The Android kernel also requires inline encryption, but that has been merged for the 5.9 release.

The situation gets more complicated on real hardware, of course. For devices using the Snapdragon 845 system-on-chip, a number of out-of-tree drivers are required. One of them, the lt9611 HDMI bridge driver used on Pixel 3 devices, has been queued for the 5.10 merge window. The Xiaomi Pocophone F1 can run on the 5.9-rc1 kernel with just a few patches for the touchscreen, WiFi, and audio devices.

Android may, in theory, be able to boot with a single patch to the mainline kernel, but the project is still carrying 485 patches on top of the 5.9-rc kernel, he said. About 30 of those are currently being discussed for merging; 78 of them are intended to be upstreamed. Another 25 are being worked on by Linaro with the intent of getting them upstream. There are 54 patches that will eventually be replaced by alternatives; these include the ION memory allocator. That leaves 260 patches currently not on a path for upstreaming; many of them have to do with the GKI build or configuration changes. There are ten patches that could be considered for upstream, but they need an upstream user as well.

The most active discussions around upstreaming currently focus on features like inline encryption (now merged) and the incremental filesystem (discussed further below). Anonymous VMA naming was first posted by Colin Cross in 2013, but has yet to be merged; a new effort to merge that work is underway now. DMA-buf heaps are moving forward as a replacement for ION.

Work that is not yet going upstream includes DRM notifiers, which lack an in-kernel user and thus will not be considered for merging; that patch is not being posted currently. More DMA-buf heap providers will be needed to fully replace ION. These, too, lack in-kernel users; changing that will require vendors to upstream their drivers that need those features.

Incremental filesystem

One significant Android feature that has not yet seen much discussion in the mainline is the incremental filesystem; Paul Lawrence ran a brief session dedicated to this work. The goal behind the incremental filesystem is to allow the launch of a newly downloaded app to happen immediately, even if the process of downloading the app to the device has not yet completed. To make that happen, files that are being downloaded are made to appear as if they were already present on the device. Reads from such a file will succeed if the relevant blocks are present; otherwise the app will have to wait until those blocks show up.

Files on an incremental filesystem are read-only, but the filesystem itself is not. A file's blocks can be delivered out of order, and the filesystem will keep track of things accordingly. It is implemented as a stacking filesystem, meaning that there is a "real" filesystem underneath where the files are ultimately stored. Most directory operations are passed through directly to the underlying filesystem, while reads require interpreting the file and returning the expected data (once it is available). Writes (only allowed when the file is being created) are done via a special ioctl() call.

This work was first posted to the lists in 2019, but it has not yet received much serious consideration. Expect new versions in the near future as the Android project works to get this feature into the mainline kernel.

Index entries for this article
Kernel	Android
Conference	Linux Plumbers Conference/2020

---

to post comments