BuildStream performance

Introduction

This page should be used to track ongoing work to monitor and improve the performance of BuildStream.

Work should be categorised so that all bugs relating to one aspect of BuildStream's functionality are discussed together.

Generally useful links:

1. Startup time

Time spent starting the Python interpreter and importing the 'buildstream' module. A good testcase is bst --help.

Open issues:

2. Pipeline initialization

Time to load the .bst files, calculate cache keys, contact remote caches and check availability in the local cache. A good testcase is `bst show`.

Open issues:

3. Staging area construction

The 'Staging dependencies' task that occurs as part of bst shell|checkout|build. The easiest way to test is bst shell on an element that has no integration commands.

Profiling with Pyflame shows Python path/string processing operations high on the profile.

Caching manifests (#82) could save time here.

Open issues:

There has been discussion of using sandboxfs to construct the staging area more quickly.

4. Integration and build commands

Mostly the speed of the FUSE sandbox that the integration commands run on top of.

5. Automated tests / CI

Speed of BuildStream's test suite and integration test suite, plus CI speed for projects using BuildStream.

Fixes:

6. Other

Speed of `bst checkout`

OSTree optionally calls fsync() while doing a checkout, disabling this seems to speed things up in some cases.

Projects/BuildStream/Roadmaps/Performance2018 (last edited 2018-01-03 11:38:31 by SamThursfield)