Notes from VadimRutkovsky:

• Gnome Continuous rebuilds apps on each commit and runs integration tests - seems to be the best option for current task

Ping walters or owen for details on this

• Installed tests format is similar to .desktop file format, checks exit code of the tool defined in 'Exec' parameter

• build.gnome.org doesn't store the output produced from passed tests: our test should take care of storing results itself

• QE team has chosen behave tool to speed up test development. It associates the human-readable steps with actual automation steps

Ping shivani about the details

Current proof of concept:

1. the only application covered is EOG

2. Test is running a series of dogtail actions following the same scenario for defined locales (by setting LANG env var)

3. string are defined in english, dogtail looks for appropriate translation in this application and locale.
4. Test stores screenshots in separate folder, html report is removed if the test has been successful

Sample results

Current problems and limitations:

1. build.gnome.org builds master/latests branch only
2. we don't have a proper "hung test" detection on Continuous - any test lasting more than 5 minutes will be terminated, no result stored. We should split big test for now.

3. some modules are tagged - ftbfs, incompatible with other apps etc. Check manifest for details ("tag" and "tag-reason" fields)

4. some fonts might be missing on Continuous
5. tests produce html report, but as we embed the screenshot in the report the report is huge and unusable
6. for each application built on Continuous we need a short scenario to follow while test is making a screenshot
7. in some locales words translated from English might resolve into several translations: we should then write a separate test with defined strings. Translators then should also make sure that tests is updated after according translation modification.