This page should be used to track ongoing work to monitor and improve the performance of BuildStream.
Work should be categorised so that all bugs relating to one aspect of BuildStream's functionality are discussed together.
Generally useful links:
Pyflame produces pretty flamegraphs.
All issues carrying label Optimization
1. Startup time
Time spent starting the Python interpreter and importing the 'buildstream' module. A good testcase is bst --help.
2. Pipeline initialization
Time to load the .bst files, calculate cache keys, contact remote caches and check availability in the local cache. A good testcase is `bst show`.
3. Staging area construction
The 'Staging dependencies' task that occurs as part of bst shell|checkout|build. The easiest way to test is bst shell on an element that has no integration commands.
Profiling with Pyflame shows Python path/string processing operations high on the profile.
Caching manifests (#82) could save time here.
There has been discussion of using sandboxfs to construct the staging area more quickly.
4. Integration and build commands
Mostly the speed of the FUSE sandbox that the integration commands run on top of.
5. Automated tests / CI
Speed of `bst checkout`
OSTree optionally calls fsync() while doing a checkout, disabling this seems to speed things up in some cases.