Because it is often difficult to remember how to run tests for Gaia, the front-end for Firefox OS, I’ve noted down some of my frequently used commands here. Consider this document a brain dump, made primarily for my own forgetful self.
The build system targets do not automatically deploy changes to source code. This appears to be the de-facto workflow with Node.js, however odd it seems. The correct workflow is to re-install the changed package each time you make a change.
You can use a tool like Gareth Aye’s npmr
that will detect which local packages have changed
and install only those.
You call it with the
every time you make a change.
As far as I understand, the long term plan is to upstream this behaviour to npm itself and make it a default.
Most of the tests I run are Marionette related. The Marionette Node.js client comes with unit- and integration tests.
Running the unit tests is fairly straight forward
because they do not rely on a B2G Desktop browser binary.
Yet, because they depend on the
node_modules Makefile target,
they still need to download the latest B2G build off
although the tests themselves does not use it.
The integration test target encompasses
all the integration tests for Gaia,
including integration tests for the Marionette client.
To limit the tests only to those related to the harness,
one can use the
HARNESS_ONLY output variable.
This is essentially a predefined test list that populates
TEST_FILES variable lets you override
the tests that are picked up from the various manifests
by specifying the (relative or absolute) paths to individual tests,
separated by spaces.
In this case it will run the test files test, cases, separated, by, and spaces.
An important peculiarity with the Gaia test targets
is that they run tests marked as ignored in manifests by default.
This means you need to set
to the path of the ignore manifest used by Treeherder (TBPL) explicitly.
does not replicate how tests are run by the test slaves.
To properly replicate the exact environment under which the tests are run,
you should consider running them in Linux containers.
Since Gaia tests are run on Taskcluster, you can access the artifcat link from the job information pane on Treeherder, where Taskcluster gives you the exact instructions for running a test job locally on your system.
For integration tests
or tests that require the use of a browser binary,
you can specify
the absolute path of the binary
to have the build system pick up on it.
Unfortunately it will still download the latest B2G binary even if this output variable is set.
Note that since a few months ago, you must now specify the b2g-bin binary, and not the b2g shell script that wraps it. If you do the former, and you run your tests non-headless, you will get a popup saying it cannot load your profile. If you run the tests headless, they will simply time out after a very long time.
To get access to the Gecko log
like you would with
--gecko-log - with mach,
you can set the VERBOSE output variable.
In Gaia, make wraps Node.js which in turn wraps a Python virtualenv, wrapping another Node.js process managing the B2G binary. In other words there are many opportunities to mess up process ownership.
It can sometimes be wise to ensure that you have no defunct/zombie processes lying around, because it happens that the Marionette client connects to old processes which will cause you much pain.