February 2015 Status

This month I'll be running an experiment with turning my daily status updates into having a more narrative diary or notes format.

It's important for me to highlight that whoever reads these notes should be aware that they're not edited in any way, and so follow the style of a brain dump more than something meant for publishing.

If this is a successful experiment I might consider renaming the status filename to notes or diary. I should also highlight that it's a development diary, and contains nothing confidential or personal of any sort.

2015-02-01

Travelled back to London, via Istanbul, with Turkish Airlines (TK 811 and TK 1985). Second flight was with a modern 777-300ER that was fairly comfortable.

Added Bellboy Bar at the Hotel Berdichevsky to the cocktail standards.

2015-02-02

Back in London. Missed Denelle Dixon-Thayer's presentation because it wasn't in my calendar, and I'd been travelling so decided to catch up on sleep. Was told my name was noted. Noted.

Worked on bug 1107706 most of the day, specifically on cleaning up the existing patches and rewriting the history. I continue to be impressed by hg histedit.

After fixing up the issues since the last try run I triggered new try runs on mobile (pushlog) and desktop (pushlog).

Filed bug 1128563, commented on 1128440, and replied to a discussion about return values from expected conditions in explicit waits on selenium-users@.

2015-02-03

Wrote a failsafe function for getting the number of colours available in the current terminal (which is essentially equivalent to tput colors and much better than checking if TERM is dumb).

Continued work on bug 1107706, specifically looking into the tests that were failing from the previous try runs. Spent time chasing down what I believed to be promises problems, that is, functions returning too early, in executeScript. This turned out to be a dead end, but findElement still deserves some more attention.

Filed bug 1128997, and spec bugs 27949 and 27950.

2015-02-04

Had my one-on-one with dburns where we discussed many things, including building standards in different countries, some Mozilla official matters, and progress on refactor tracked in bug 1107706.

Had a closer look at the try results from Monday. Looks like most of the issues come down to two issues. The first is the “malformed packet, expected key error” that is likely caused by some function returning too soon or missing a promise. The second is a script timeout error on B2G emulators which at this point is more mysterious.

I keep spending absurd amounts of time configuring, building, and getting tests running. This is a big contributor to my current inefficiency as I need to reproduce failures from the try runs against various configurations.

The setup of gaiatest was particularly challenging, primarily because I had year old version of manifestparser which was apparently called ManifestDestiny earlier. This means pip uninstall manifestparser would uninstall the correct version that python setup.py develop had installed for me. The old verison would get picked up first on Python's import path, causing issues when it was imported to gaiatest, because some of the test manifests in Gaia rely on a few features that wasn't present in the version I had installed.

In the end I found the correct command for running the Gaia integration tests (Gip) locally:

% gaiatest --restart --type b2g --binary /home/ato/Code/gecko/build/b2g/dist/bin/b2g --profile /home/ato/Code/gaia/profile --testvars /home/ato/Code/gaia/testvars.json gaiatest/tests/manifest.ini

Quite a mouthful.

Finally I was able to locate and fix the problem with malformed packets. It was indeed a problem with the packet being malformed, but not in the expected way. For executeScript in content mode we were returning undefined which wasn't getting marshaled to JSON's null (None in Python). The solution was to pass along a function reference of Dispatcher.sendOk to CommandProcessor.execute, in addition to Dispatcher.send passed along already.

Previously the assumption was that whenever Response.value was undefined, that would mean the packet should be changed to mean “ok”. As it turns out it's perfectly legal to return undefined's from JavaScript.

The rather unsettling part to the story is that this wasn't caught by Marionette's own unit test suite. That means the unit test suite isn't doing any sanity testing on return values of scrips, or of protocol correctness. It was caught by test_browser_navigation.test_browser_back_button which relied on UpperCaseStateManager, which when uninitialised returns undefined for isUpperCase. Scary.

2015-02-05

Two new issues discovered in bug 1107706: A call to error.isError from executeAsyncScript in chrome context never returns, and the script timeout parameters for this command aren't being respected.

The first problem was apparently caused by the "result" in obj check, which will throw TypeError if obj is a boolean. The solution is to return early if obj is null or not an object.

Examining error.isError I found that it does not throw errors that are propagated to the caller for some reason. The issue with checking a boolean also applies to any primitive which would also throw TypeErrors. Resolved this issue by having the function check that obj is an object before performing the other failure-prone checks.

Later I discovered the first real race condition in passing messages between the chrome- and content layer. I had always known that this had been a problem in Marionette previously, and that this was the reason unique command IDs were introduced. There appears to be a checkLoad function attached to the content browser that calls sendOk, which causes a race condition if you have the message listeners open whilst performing a ListenerProxy call.

Now most commands in marionette-listener.js actually use command IDs to pass messages back and forth between content- and chrome. Whilst doing some changes to marionette-elements.js' EM_find routine I had earlier decided to drop the use of command IDs because I (mistakenly) believed callbacks were removed when not used. This appears not to be the case for checkLoad, and so because there might be other unknown bugs related to this, I decided to put the command ID passing back in. This appears to have solved the problem.

Local testruns in the evening showed that running Gip with xvfb gave different results than without. This could be related t to the repaint issues I'm seeing X forwarding to Mac.

Fixing these three issues moved us from 27/144/90 (pass/fail/skip) to 155/16/90. Pretty decent improvement, and a good day's work.

2015-02-06

For bug 1107706 I fixed up the changes from yesterday and conducted some cleanup in errors.js and dispatcher.js, and triggered new try runs for desktop and mobile (pushlog).

To my great surprise the Gaia integration tests (Gip) are now passing, despite 16 failures locally. On querying dhunt it appears a number of the tests require special test variables to run as they interact with external services such as IMAP servers, GMail, and Facebook.

Still the same failures on B2G emulator tests. Had a look at GD_get (the navigation routine) and I suspect it won't work as it's currently written with e10s. It surprises me that the “error loading page” error throws at all. I would've expected the function to return “ok” because there's no promise guard for checkLoad.

Rewrote the function to use a promise, and triggered a new try run, with no guarantees that it will help.

Furthermore it's problematic that this test can only be reproduced natively on Linux, that is, not through X forwarding.

2015-02-09

Triggered a new try run (pushlog) because there was a syntax error in the previous one, still for bug 1107706.

In the afternoon I went to Heathrow to catch my flight to Oslo. I can warmly recommend the Piccadilly line during rush hour. Attended A-team meeting from lounge.

2015-02-10

Followed up on some email discussions, did some administrative work, and did some research on bidirectional socket programming.

2015-02-11

Not feeling great, so took a sick day. Was nevertheless able to do a little bit of work by submitting a PR for the WebDriver spec to add bug-assist.js. I also followed up on a discussion on making it explicit that a response ended the session. We need to file a bug about this.

In the afternoon I managed to get out another pull request for guarding against the reuse of sessions to the WebDriver specification.

In the evening I reviewed the final pieces of bug 1119211 which was long overdue. Apparently the mozreview people have introduced a linter bot for Python which is acting very annoying by opening issues that reviewers can't close. Apparently only the owner of a review can close issues in mozreview. Annoying, because it increases the load put on the author. I also keep getting confused by child and root reviews; I still think they cause confusion.

2015-02-12

Did some considerable work on the WebDriver specification together with jgraham. Filed pull requests #9, #10, #11, #12, #13, and #14. The work centred mainly on cleaning up the chapter on sessions.

I also filed bugs 28002, 28003, 28004, and 28005 against the spec.

2015-02-13

Gave jmaher some feedback on his writeup about the search for extraneous test automation. I also provided a little bit of feedback on jgraham's suggested wpt WebDriver client replacment. We're going to have to replace the current one because it doesn't give us enough low-level control to test what we need to test of the protocol.

2015-02-17

Provided some feedback on jgraham's pull request to provide a main loop for the WebDriver specification.

2015-02-18

Worked hard on finding out why the test results from try were different to running tests locally on the B2G ICS Android emulator for bug 1107706. It appears the emulator freezes up on running executeScript("window.open('chrome://content/marionette/test.xul', 'xul')") from test_execute_async_timeout in chrome context, a test that passes on try.

2015-02-19

Fixed up a missing heading in PR #12 for the WebDriver specification.

Continued debugging failing local tests for bug 1107706. After a lot of work on this, I discovered from discussions with jgriffin and dburns that the failing tests, all to do with operating on XUL components in chrome context, are in fact not meant to be run on B2G. That they pass on try is still a mystery, despite the fact that I spent time trying to figure out why this is. I'm left to conclude that the test ordering cannot be guaranteed, and that there probably is some previous test which state bleeds over into these, allowing them to pass.

It's frustrating having spent two full days of work on something which was as easy (and as demotivating) as adding the @skip_if_b2g decorator on those tests.

After disabling the tests related to XUL components in chrome context on B2G however, I have the same number of failing tests as on try. The remaining failed tests are all related to emulator callbacks, something I feared a long would be a problem.

2015-02-20

Mike Smith volunteered me to be a mentor for the wpt/testharness.js/WebDriver integration. sole offered to give me advice on mentoring GSoC, if needed.

Started work on updating the emulator callback code in bug 1107706 to work with the new dispatcher system. I believe it will be possible to reuse the same data structures for both the chrome- and content contexts, which is good.

2015-02-22

Submitted bug 1135502 about the accessibility.typeaheadfind pref not working with e10s and event listeners for keys. Also attached a (sadly manual) test case.

Continued the work on refactoring the emulator callback code for bug 1107706. It appears since runEmulatorCmd in the listener was changed to use sendResponse that the return value gets intercepted in ListenerProxy, then yielded back to executeWithCallback in the driver. The question is if we want emulator callbacks to go through the command implementation or we want them to be implicit through the old Marionette:runEmulatorCmd global message listeners.

What is certain is that we need to cancel the implicit response sent from executeWithCallback because it's wrapped by CommandExecutor.execute.

In the evening I provided a positive review of bug 1118298.

2015-02-23

Continued work on emulator callbacks for bug 1107706. Reached all tests passing, but felt the solution was inelegant.

Also attended A-team fortnightly meeting.

Pushed new try runs for desktop and mobile.

2015-02-24

Concluded that my work on bug 1107706 has actually unbroken the hidden jobs for B2G Android Emulator opt, compared to their previous state. I think this is good enough for landing the patches.

Reviewed r/4085.

2015-02-25

Worked mainly on cleaning up the patches for bug 1107706 to make it presentable.

Provided feedback on Selenium 3 planning document.

2015-02-26

Started rebasing patches for bug 1107706, and most of them were fairly straight forward. I'm uncomfortable with using hg rebase but this may be because I'm new to it. I applied the patch to driver.js manually, and I'm having issues with some of the removals of CPOW code to get e10s working that chmanchester did recently.

Provided feedback on bug 1100155.

2015-02-27

Continued work on bug 1107706 whilst travelling by plane to Norway. Also started work on removing Presto-OperaDriver support from Selenium.

2015-02-28

With no small sense of nostalgia and remorse, I landed the patches to remove Presto-OperaDriver support from Selenium. Bye-bye, Opera.

I also raised the question of switching the canonical source repository to Github on the selenium-developers@ mailing list. It's ironic that I would be the one to bring it up, seeing as I was the staunchest defender of GC when we discussed it last.