Porting Baserock to BuildStream

Since Baserock builds a complete operating system from scratch it is an interesting first milestone, also this is first on the list because its almost completely covered already by the basic functionality already implemented in BuildStream.

Here are some tasks which fall into the scope of completing the migration.

Note:: Its possible we move ahead some of tasks in Flatpak Support such as artifact splitting and filtering here, in order to get more flexible and desirable deployments out of a build.

Migrate Base Runtime

The base runtime built in Baserock required a manual migration due to the "no host tool" policy of BuildStream, however this was not a large job, and Baserock's base runtime does not change very much.

The manually migrated build-essential can be found here for now.

Status: Done

Conversion scripts for Baserock projects

The migration script already exists and lives here.

Remaining issues with this script:

• Generated YAML is in random dictionary order, need to implement ordering in the output for a nicer result.

• Need to port over Baserock's system-integration commands. These should be added as integration-commands in the public bst domain of the generated elements, as specified in the documentation

Currently Baserock's definitions for building a GNOME system are known to convert well and build, however after implementing system integration bits, it is not expected to work out of the box. This is because Baserock expects to build as root and does some root requiring things in the build process, such as assigning files arbitrary uids and gids.

Status: Done

Notes

Conversion scripts do not handle the build base, something needs to be manually created to build on top of since we dont support any usage of host tooling. Further, the deployments are not converted automatically, this needs a little bit of documenting but we have a bootable image deployment technique we can use.

Document migration process

The migration process needs to be documented, preferably as a README file added to the conversion script repository.

This should explain what inputs and conditions the script expects, and what might happen in the process which the user may have to take care of manually (such as modifying scripts to not have a hard requirement on being root).

Status: Done

Compose Element

Not sure what this element should be called, but it will be interesting to have an element which takes a collection of dependencies and simply composes a full single image based on those, post processing elements can optionally be run on the output of a composed image, before proceeding to a deployment.

Status: Done

Notes

This exists as the compose element and now also supports the new split-rules concept. The compose element is able to create a composition of the elements build dependencies and filter the content of those dependencies depending on the split domains which were selected. It also has the option of including any orphaned files (not caught by any of the split rules) and to include files generated/modified by the integration commands.

Bootable Image Deployment

We need to start adding deployment elements to BuildStream. The first and most interesting deployment will be one which creates a bootable system image.

This will be done by identifying elements in the deployment element's dependencies and selectively staging those dependencies in different places.

• One set of dependencies will be staged into the sandbox root, this is what will be used to run tools like syslinux and mkfs, we dont want to rely on host tooling for this kind of deployment.

• The other set of dependencies will represent the payload which the deployment intends to include, these will be staged into /buildstream.

The image deployment should have some sane defaults to describe things like desired filesys type for the image and the overall image size, which can be configured in the deployment element's YAML in a given project.

Status: Done

Notes: This is currently not implemented as a general image element which will be desirable but instead, using the newly introduced script element.

I have tested that using the script element I am able to generate a fully bootable image using:

• vfat boot partition (this is the only way that is really supported by syslinux to install without root privileges)
• mtools (to copy in the initrd/kernel/syslinux.cfg files into the boot partition)
• parted to partition a file
• mkfs.ext4 -d option, newer versions of e2fsprogs allow populating a newly created filesystem with a directory content (we use this to create a filesystem image)
• dd to splice the created filesystem images directly into the full disk image