Building system components

If you want to contribute to GNOME applications, please visit the Newcomers page, and follow the instructions you find there.

While GNOME applications can be easily built through Flatpak, system components require deeper integration with your environment. This means that you need to have an updated version of the operating system as a requirement.

In order to avoid damaging the OS installation you're using, GNOME uses BuildStream. Buildstream builds GNOME components and their dependencies, and safely installs them in a separate environment, without interfering with your OS installation.

Getting Started

In order to get started the only requirement is to have 20 GB free disk space and using a Linux distribution.

We will use the folder "$HOME/Projects" for our work development. Execute in a terminal:

mkdir --parents $HOME/Projects
cd $HOME/Projects

Install BuildStream

Install Buildstream by following the instructions here.

After that, install bst-external for GNOME Buildstream integration. To do so, execute in a terminal:

git clone https://gitlab.com/BuildStream/bst-external.git
cd bst-external
pip3 install --user -e .
cd ..

Set up Buildstream for GNOME

Create a file at ~/.config/buildstream.conf with the following content:

# This configuration reduces the amount of necessary rebuilds
projects:
  gnome:
    strict: False

# This configuration lets you control the number of allowed parallel builds
scheduler:
  builders: 2

Documentation about the user configuration options can be read in the manual.

Now we can move to download the actual GNOME projects with Buildstream, for that download GNOME Build Metadata that describes all the GNOME modules and dependencies:

git clone https://gitlab.gnome.org/GNOME/gnome-build-meta.git
cd gnome-build-meta

Your first build

Now we are ready to do our first build. It will take longer since it has to build all the dependencies. Further builds should take much less time. We will build gtk in our examples, you can change it to any other project in the elements folder. Execute in a terminal:

bst build --track-all sdk/gtk+-3.bst

Running software built with BuildStream

To run an application that you have built, use the shell command.

bst shell sdk/gtk+-3.bst

[ ... BuildStream now prepares a sandbox and launches a shell ... ]

gtk3-demo

{i} In case you are trying to develop integrated components such as GNOME Shell, GDM, gnome-session or gnome-keyring, these are more difficult and not yet supported. In the future BuildStream will generate a system image for you to test in a VM.

Developing

So far we have showed only how to build and run GNOME modules where the source code is opaque and controlled entirely by BuildStream.

However of course you will want to develop your own software within GNOME and test changes you make to any GNOME module, for this we use the Buildstream workspaces feature, which will simply tell Buildstream that it needs to use local code for a specific project, instead of the one from the repositories. Performing this step is needed only once.

To open a workspace execute in a terminal:

bst workspace open sdk/gtk+-3.bst ~/Projects/gtk

This will clone the Gtk repository from the local cache with a checkout at the exact commit which would be used in the present project state, you are now free to pull new changes from the upstream repository at will, switch branches and modify source code directly, the next build will use the content of the workspace.

If you have GNOME commit access you may want to change the origin URL:

cd ~/Projects/gtk
git remote set-url origin git@gitlab.gnome.org:GNOME/gtk.git

From here, it's possible to develop iteratively by simply make changes in the code and then executing:

cd $HOME/Projects/gnome-build-meta
bst build sdk/gtk+-3.bst

And then to run the changes:

bst shell sdk/gtk+-3.bst
gtk3-demo

If you want to do tests such as "meson test" or "make check" you need to enter a build shell. For that, execute:

bst shell --build sdk/gtk+-3.bst
ninja test

Updating to latest build metadata

As you can see, once we did the first build we don't update anymore the dependencies. Althought this is fast, this could be problematic if something changes in your application that needs a new dependency version. For that, it's recommended to periodically pull the latest changes from the GNOME Build Metadata repository. To do that, simply:

cd $HOME/Projects/gnome-build-meta
git pull --rebase

Then continue with your development as usual.

Updating a single module

It is possible to upgrade only a single dependency at a time and then test that your application works when running against an updated module, without rebuilding everything in between.

For example, to test gedit against a refreshed Gtk, you can run the following commands:

bst track sdk/gtk+-3.bst

[ BuildStream fetches the latest GTK ]

bst build core/gedit.bst

[ This only needs to rebuild GTK ]

bst shell core/gedit.bst

[ Now you get a shell with your prebuilt gedit, running against an updated GTK+ library ]

When things go wrong

Sometimes things will not build the first time around especially when building the current development branches of everything. When a build fails BuildStream will offer you an interactive prompt with some options to view the log file, or get an interactive shell to debug what was wrong with the build.

Barring any bug which might occur with the tooling, the only causes of failures can be:

  • Network connectivity issues, failing to download some sources
  • A legitimate bug in the modules we build

If a legitimate bug occurs in the build, the only recourses are to:

  • find a previous version of the said module which does not cause the build to break
  • fix the bug yourself

In any case, the first thing to do might be to file a bug against the offending module, whether it is the fault of that module or one of its dependencies will be sorted out by the maintainers.

If you want to fix the bug yourself:

  • Follow the instructions mentioned above to open a workspace on the offending module
    • modify the source code in the workspace
    • try your build again
    • repeat until this works
  • Create a patch, using git format-patch in the regular way, and submit that for review in the appropriate bug tracking

  • When the upstream maintainers of the given module have accepted your patch:
    • close your local workspace using bst workspace close

    • try to build with the new version of the offending module to verify that the fix was successful upstream

Next steps

Now that you know how to make changes and test it in the desired module you want, next step is to learn how to approach solving bugs in GNOME.

So just go to Newcomers/SolveProject and let's start understanding the code.

Newcomers/BuildSystemComponent (last edited 2019-03-19 16:11:47 by CarlosSoriano)