3.4. Testing and Quality Assurance

Part of the Yocto Project development and release process is quality assurance through the execution of test strategies. Test strategies provide the Yocto Project team a way to ensure a release is validated. Additionally, because the test strategies are visible to you as a developer, you can validate your projects. This section overviews the available test infrastructure used in the Yocto Project. For information on how to run available tests on your projects, see the "Performing Automated Runtime Testing" section in the Yocto Project Development Tasks Manual.

The QA/testing infrastructure is woven into the project to the point where core developers take some of it for granted. The infrastructure consists of the following pieces:

Originally, much of this testing was done manually. However, significant effort has been made to automate the tests so that more people can use them and the Yocto Project development team can run them faster and more efficiently.

The Yocto Project's main Autobuilder (autobuilder.yoctoproject.org) publicly tests each Yocto Project release's code in the OE-Core, Poky, and BitBake repositories. The testing occurs for both the current state of the "master" branch and also for submitted patches. Testing for submitted patches usually occurs in the "ross/mut" branch in the poky-contrib repository (i.e. the master-under-test branch) or in the "master-next" branch in the poky repository.

Note

You can find all these branches in the Yocto Project Source Repositories.

Testing within these public branches ensures in a publicly visible way that all of the main supposed architectures and recipes in OE-Core successfully build and behave properly.

Various features such as multilib, sub architectures (e.g. x32, poky-tiny, musl, no-x11 and and so forth), bitbake-selftest, and oe-selftest are tested as part of the QA process of a release. Complete testing and validation for a release takes the Autobuilder workers several hours.

Note

The Autobuilder workers are non-homogeneous, which means regular testing across a variety of Linux distributions occurs. The Autobuilder is limited to only testing QEMU-based setups and not real hardware.

Finally, in addition to the Autobuilder's tests, the Yocto Project QA team also performs testing on a variety of platforms, which includes actual hardware, to ensure expected results.