Building system components

If you want to contribute to GNOME applications, please visit the Newcomers page, and follow the instructions you find there.

While GNOME applications can be easily built through Flatpak, system components require deeper integration with your environment. This means that you need to have an updated version of the operating system as a requirement.

In order to avoid damaging the OS installation you're using, GNOME uses BuildStream. BuildStream builds GNOME components and their dependencies, and safely installs them in a separate environment, without interfering with your OS installation.

Getting Started

Requirements

In order to build GNOME using BuildStream, you will need at least a recent stable Linux based system, or you can use Docker in some cases as described in the BuildStream documentation

Depending on what you want to build (building all of the release modules will obviously take more space than only building one application that you care about), you will want to have a lot of free disk space. The base runtime providing the system dependencies and toolchain alone will cost a couple of GB. For a full build of GNOME from scratch, we recommend having at least 20GB of free space at your disposal.

Install BuildStream

See the instructions here to install BuildStream

Note that you dont need to have any C compiler or anything beyond the requirements of BuildStream itself, as host tooling will never be used.

Configuring BuildStream

Follow the instructions in the documentation for user specific configurations, before going further, make sure that you have free space in the locations where BuildStream is going to do its work.

The default is to do the work in ~/.cache/buildstream

Further, the default behavior is to be as deterministic as possible, which means that any time a module is built, every module depending on that module must be rebuilt. For developer purposes, we recommend using non strict build planning mode, which only requires a rebuild of any module which has changed since the last build.

You can achieve this by adding the following to your ~/.config/buildstream.conf:

projects:
  gnome:
    strict: False

You can also decide this on each invocation by passing the --strict or --no-strict arguments, configuring the above will only change the default.

Further, by default BuildStream will try to build up to 4 modules in parallel, if this is too heavy for your system, you can reduce this value, for example:

scheduler:
  builders: 2

If you have 50 some odd cores on your developer laptop, we are all jealous, and you can feel free to increase the number of parallel builds possible. However note that BuildStream will only parallelize builds where possible, according to the dependencies.

Building GNOME

BuildStream allows you to build and run GNOME platform components and their required modules in a separate environment, isolating the installation from your operating system; this allows you to develop, test, and modify GNOME components without compromising the ability to use your computer.

Download the GNOME project

Download the GNOME project using git:

$ git clone https://gnome7.codethink.co.uk/gnome-modulesets.git
$ cd gnome-modulesets

For the remainder of this document, we will assume that your current working directory is the gnome-modulesets checkout.

The above URL is a temporary read-only location for the automatically converted modulesets. This URL will be changed to a GNOME git repository.

Obtain base runtime and source code

Since we do not use any host tooling, your first build will take a long time because it will download a base runtime containing all of the tools and dependencies you need to build GNOME.

It is recommended that you first make sure you have a runtime cached locally before building anything:

$ bst build base.bst

This will run for a while, however once you have the base runtime, subsequent updates when the runtime changes will not be as costly.

Another operation which might take some time is obtaining source code and references to the latest versions of sources.

Taking GTK+ as an example, you can fetch the source code as such:

$ bst track --deps all --except base.bst core-deps/gtk+-3.bst

Note that tracking an element will accomplish two things:

  • Perform the initial downloads of the required git repositories
  • Modify your local checkout of gnome-modulesets.git by adding the latest commit references of each module that was tracked.

Every time you build, you will build exactly these versions of sources, until you use bst track again to obtain newer versions.

Your first build

Finally, you can build the project you're interested in. For example, if you want to build GTK+:

$ bst build core-deps/gtk+-3.bst

Depending on what you are building and the machine you are building on, this can take a while.

Running software built with BuildStream

To run an application that you have built, use the shell command.

$ bst shell core-deps/gtk+-3.bst

[ ... BuildStream now prepares a sandbox and launches a shell ... ]

$ gtk4-demo

{i} In case you are trying to develop integrated components such as GNOME Shell, GDM, gnome-session or gnome-keyring, these are more difficult and not yet supported. In the future BuildStream will generate a system image for you to test in a VM.

Developing

So far we have showed only how to build and run GNOME modules where the source code is opaque and controlled entirely by BuildStream.

However of course you will want to develop your own software within GNOME and test changes you make to any GNOME module, for this we use the workspaces feature.

For example, to open a workspace for GTK+ in your home directory, simply run:

$ bst workspace open core-deps/gtk+-3.bst ~/gtk

This will clone the GTK+ repository from the local cache with a checkout at the exact commit which would be used in the present project state, you are now free to pull new changes from the upstream repository at will, switch branches and modify source code directly, the next build will use the content of the workspace directly.

By default, the workspace will use the public upstream git URI, if you have GNOME commit access you may want to change the origin URL in your workspace like so:

$ cd ~/gtk
$ git remote set-url origin ssh://username@git.gnome.org/git/gtk+

Updating a single module

Assuming you have configured BuildStream to use a non strict build plan, it is possible to upgrade only a single dependency at a time and then test that your application works when running against an updated module, without rebuilding everything in between.

For example, to test gedit against a refreshed GTK+, you can run the following commands:

$ bst track core-deps/gtk+-3.bst

[ BuildStream fetches the latest GTK+ ]

$ bst build core/gedit.bst

[ With non-strict build planning mode enabled, this only needs to rebuild GTK+ ]

$ bst shell core/gedit.bst

[ Now you get a shell with your prebuilt gedit, running against an updated GTK+ library ]

When things go wrong

Sometimes things will not build the first time around especially when building the current development branches of everything. When a build fails BuildStream will offer you an interactive prompt with some options to view the log file, or get an interactive shell to debug what was wrong with the build.

Barring any bug which might occur with the tooling, the only causes of failures can be:

  • Network connectivity issues, failing to download some sources
  • A legitimate bug in the modules we build

If a legitimate bug occurs in the build, the only recourses are to:

  • find a previous version of the said module which does not cause the build to break
  • fix the bug yourself

In any case, the first thing to do might be to file a bug against the offending module, whether it is the fault of that module or one of its dependencies will be sorted out by the maintainers.

If you want to fix the bug yourself:

  • Follow the instructions mentioned above to open a workspace on the offending module
    • modify the source code in the workspace
    • try your build again
    • repeat until this works
  • Create a patch, using git format-patch in the regular way, and submit that for review in the appropriate bug tracking

  • When the upstream maintainers of the given module have accepted your patch:
    • close your local workspace using bst workspace close

    • try to build with the new version of the offending module to verify that the fix was successful upstream

Next steps

Now that you know how to make changes and test it in the desired module you want, next step is to learn how to approach solving bugs in GNOME.

So just go to Newcomers/SolveProject and let's start understanding the code.

Newcomers/BuildSystemComponentBst (last edited 2017-11-11 09:23:01 by TristanVanBerkom)