Developer Guides on Flatcar Container Linux

Building custom kernel modules

Mon, 01 Jan 0001 00:00:00 +0000

Create a writable overlay

The kernel modules directory /usr/lib/modules is read-only on Flatcar Container Linux. A writable overlay can be mounted over it to allow installing new modules.

NOTE: On older releases - 3115.0.0 and before, published before April 2022 - /usr/lib64/modules was used instead. To build for these releases, please use /usr/lib64/modules instead of /usr/lib/modules (but ideally, update to a more recent version).

Local test environment

To test the steps below in a local QEmu instance, you’ll need to add extra storage to that instance. The default 6GB rootfs won’t suffice as the devcontainer itself is ~6GB (uncompressed; compressed it’s ~480MB). The simplest way to do this is to forward the local host directory to qemu via 9p.

Building production images

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

This guide discusses automating the OS build process and is aimed at audiences comfortable with producing, testing, and distributing their very own Flatcar releases. For this purpose we’ll have a closer look at the CI automation stubs provided in the scripts repository .

It is assumed that readers are familiar with the SDK and the general build process outlined in the CI automation .

Stabilisation process and versioning

The Flatcar OS version number follows the pattern MMMM.m.p[ppp] - “M” being the major number, “m” the minor, and “p” the patch level. Specifically:

Flatcar Container Linux disk layout

Mon, 01 Jan 0001 00:00:00 +0000

Flatcar Container Linux is designed to be reliably updated via a continuous stream of updates. The operating system has 9 different disk partitions, utilizing a subset of those to make each update safe and enable a roll-back to a previous version if anything goes wrong.

Partition table

Number	Label	Description	Partition Type
1	EFI-SYSTEM	Contains the bootloader	FAT32
2	BIOS-BOOT	Contains the second stages of GRUB for use when booting from BIOS	grub core.img
3	USR-A	One of two active/passive partitions holding Flatcar Container Linux	EXT2
4	USR-B	One of two active/passive partitions holding Flatcar Container Linux	(empty on first boot)
5	ROOT-C	This partition is reserved for future use	(none)
6	OEM	Stores configuration data specific to an OEM platform	BTRFS
7	OEM-CONFIG	Optional storage for an OEM	(defined by OEM)
8	(unused)	This partition is reserved for future use	(none)
9	ROOT	Stateful partition for storing persistent data	EXT4, BTRFS, or XFS

For more information, read more about the disk layout used by Chromium and ChromeOS, which inspired the layout used by Flatcar Container Linux.

Guide to building custom Flatcar images from source

Mon, 01 Jan 0001 00:00:00 +0000

The guides in this document aim to enable engineers to update, and to extend, packages in both the Flatcar OS image as well as the SDK, to suit their own needs. Overarching goal of this collection of how-tos is to help you to scratch your own itch, to set you up to play with Flatcar. We’ll cover everything you need to make the changes you want, and to produce an image for the runtime environment(s) you want to use Flatcar in (e.g. AWS, qemu, Packet, etc). By the end of the guide you will build a developer image that you can run under qemu and have tools for making changes to the OS image like adding or removing packages, or shipping custom kernels. Note that we chose this guide’s “qemu” image target solely to enable local testing; the same process can be used to produce images for any and all targets (cloud providers etc.) supported by Flatcar.

SDK bootstrap process

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

This document aims to provide a high-level overview of the SDK build (“bootstrap”) process.

SDK bootstrapping is implemented in bootstrap_sdk and happens in 4 stages. Gentoo’s catalyst is used to run each of the stages in an isolated chroot. Each stage requires a “seed” tarball which contains the root filesystem used by that stage. The first stage will use an existing SDK as its seed (i.e. root FS); stages 2 to 4 will use the output of the previous stage as its seed (root FS).

Tips and tricks

Mon, 01 Jan 0001 00:00:00 +0000

Finding all open pull requests and issues

Searching all repo code

Using repo grep you can search across all of the Git repos at once:

repo grep CONFIG_EXTRA_FIRMWARE

Note: this could take some time.

Base system dependency graph

Get a view into what the base system will contain and why it will contain those things with the emerge tree view:

equery-amd64-usr depgraph --depth 1 coreos-base/coreos-dev

Get a tree view of the SDK dependencies:

Release guide

Mon, 01 Jan 0001 00:00:00 +0000

This guide covers the complete Flatcar Container Linux release process from preparation through publication. It is intended for Flatcar maintainers and contributors who need to prepare and execute official releases.

Quick Reference

Key Dependencies:

WireGuard access to Jenkins
HSM key for update payload signing
GitHub repository access (flatcar-linux, flatcar-nebraska)
GPG key in Jenkins keyring

Critical Path:

  flowchart LR
    A[Tag Preparation] --> B[Jenkins Build]
    B --> C[Testing]
    C --> D[Go/No-go Meeting]
    D --> E[Release Job]
    E --> F[Artifact Distribution]
    E --> G[Update Payloads]
    F --> H[Nebraska Promotion]
    G --> H
    H --> I[Public Announcements]

Prerequisites

Ensure the following requirements are satisfied before starting the release process: