Cross Building
Contrary to projects like buildroot or yocto, BuildStream takes a stance against true cross compilation of anything beyond the base toolchain, which will always require cross compilation at least for bootstrapping purposes.
Conversely, there is only so far you can go with trying to optimize virtualized build environments before you end up with something like scratchbox2, which requires extensive knowledge for configuration, not to mention with that level of complexity; the repeatability of exactly the same builds 10 years down the road is questionable.
BuildStream will address the problem of cross building by walking the middle ground, virtualize the build environment for cross compilation of components beyond the toolchain, but not so much that the result becomes unmaintainable.
A supporting part of this will be implemented in BuildStream proper, while a reference bootstrap will be maintained by the Baserock reference builds.
Note: This milestone is in fact entirely orthogonal to other milestones in the roadmap and work can potentially commence on this immediately.
Understanding of host / target architecture
Currently BuildStream supports an --arch argument for denoting the target architecture, and only supports building for host compatible architectures (like building a 32bit runtime on a 64bit platform). This architecture defaults to the detected architecture indicated by uname.
BuildStream will need an additional argument for determining both host and target architectures. This may sound trivial and unneeded, but it's strictly only needed for the initial bootstrap stages so that host compatible base dependency may be selected for the host runnable toolchain we use at first (remember: No Host Tools in BuildStream).
Status: Done
Cleanup and Support cross building in Baserock
Baserock's initial runtime builds build-essential (already ported to BuildStream here) are already very close to being adequate for creating a cross bootstrap, although Baserock has traditionally used a different approach by using a cross toolchain built on top of build-essential.
This needs to be polished and fixed, so that the base build-essential itself may produce cross tools and cross compile a native compiler for the target architecture.
Stage 1 tooling here is to compile a libc and cross compiler, using the tools provided in a host compatible base (instead of using raw host tools). Here mostly only a host -> target version of binutils, gcc and accompanied glibc need to be built
Stage 2 is to use the cross tools we built in stage 1, to output a native target -> target toolchain with binutils, glibc, gcc, busybox and some others such as make and gawk.
Baserocks build essential already runs very closely to this, and Rob Landley's Aboriginal project already provides a very comprehensive guideline of how this can be done cleanly and reliably, so that the same build metadata can be reused for any target arch we add support for bootstrapping.
In the case we are building for a host compatible architecture, these same tools will be used in the same Sandbox in the regular way moving forward in the build, as cross compilation is no longer needed.
In the case we are building for a host incompatible architecture, these tools we've produced cannot be run on the host. If we want to build beyond the bootstrapping of the toolchain we will need a virtualization enabled Sandbox.
Semantically, this new build-essential will depend on an import of tools which can be run on the %{host-arch}, but it will produce output that is executable on %{target-arch}. Projects which depend on build-essential at this point will depend on a build-essential of %{target-arch} and need no longer care about what is the %{host-arch}.
Note: This step is not particularly a requirement for moving on, so long as a base toolchain already compiled for the desired target exists in some shape or form, we can move on to virtualized cross compiling in the next step. This step is only important to demonstrate a capacity for bootstrapping a new environment on a target, when no binary runtime environment exists yet for that target.
Status: In Progress
Cross Sandbox
BuildStream has an abstraction layer allowing multiple sandboxes to be implemented, the default sandbox being the bubblewrap based sandbox.
A new sandbox implementation needs to be created which:
- Requires host qemu be installed with the appropriate binfmt for the %{target-arch}
- Runs host qemu with the appropriate parameters for user mode virtualization of %{target-arch}
- Preferably still uses bubblewrap to launch qemu inside a nicely sandboxed environment
BuildStream will have to detect that a project is depending on a base dependency that is of a host incompatible architecture, and in that case only it must use a different Sandbox implementation for this.
Note: This cross sandbox does not require that a cross toolchain be built with buildstream in baserock, any runtime with the required base tooling for the target architecture will do (we can equally do this using an import of already compiled debian packages for instance).
Status: TODO
Optimize
The first step to optimization will be to use a cross compiler running outside of the Cross Sandbox (probably running inside another bubblewrap based sandbox), so that we can have the sandbox abstract away the complexity of distributing compilation tasks to a faster environment over distcc.
For this, the sandbox itself will need to require a cross compiler, which for Baserock can be obtained as the same stage1 cross environment generated in an eventual build-essential. Hopefully the Cross Sandbox can simply enable distcc in the case that a cross compiler is present.
Status: TODO