April 2015

2015-04-01

Fixed bug 1150041, and pushed some fixups to bug 941085 because of try failures on Windows (due to shorter Content-Length) and e10s (because of new bug 1149998.

Had my 1:1 with dburns. Discussed new deliverables for Q2, of which I think I'm going to have fewer than last quarter.

Submitted PR 19 to WebDriver which merges all source docs into a single, canonical webdriver-spec.html.

Reviewed bug 1150247.

2015-04-02

Set up Travis builds for PR's to the Selenium project, and enabled building of tests by default as part of the build target.

Filed bug 1150512 about moving Marionette's test runner from mozhttpd, which is considered unsupported, to wptserve. This might help us avoid issues such as bug 14574 in Python which only triggered on Windows XP opt as part of a test for bug 941085. I suspect that mozhttpd's handling of Content-Length's for multipart/form-data in POST's is to blame, and that this triggers an exception path in Python's standard library.

Disabled the offending test on Windows for bug 941085, and pushed to inbound.

Filed a few bugs in preparation for the work on bug 945729:

Apparently I fixed the extraction of the wrong variable in bug 1150041 yesterday. I've requested some more information from gecko.log to debug this properly.

Submitted PR's #131 and #132 to marionette-js-client. Both minor fixes to get tests running on my system.

2015-04-07

Made another attempt at landing bug 941085 with a fresh try run which should hopefully have Gip tests enabled.

Fixed up some test issues for bug 1150522 and pushed to inbound.

Reviewed r/6683 which tries to reduce memory consumption in Marionette which recently spiked after landing bug 1107706.

Worked on bug 1150523 to add lookup of errors by string to the Marionette Node.js client.

2015-04-08

Did my 1:1 with dburns where we discussed deliverables for Q2.

Started work on bug 945729 that changes the Marionette server to use strings for errors as I thought it would be good to verify the client patches. This turned out to be a good idea, as I found a few bugs from the patch for bug 1150522 that I landed earlier.

Filed issue 432 on Selenium about defining the error strings in a new status field in the atoms, as Marionette currently has to do a conversion. Also filed bug 1152409 to remove the error codes from Marionette, which sadly depends on us upgrading the atoms (which isn't trivial).

Also fixed bug 1152425.

2015-04-09

Filed bug 1152682 to address two review issues from r/6727. Fixed up remaining issues.

Aligned the list of error strings in Selenium's atoms with the WebDriver protocol's.

Submitted PR 26 to the WebDriver specification that corrects some error strings.

Reviewed r/4610.

Finished investigating B2G Desktop Linux opt failures for bug 941085. Turns out that GaiaTestRunner relies on BaseMarionetteTestRunner setting self.httpd as part of the start_httpd routine. My patch renamed start_httpd to create_httpd and made it return the httpd object instead of setting it. Reverting this change and deprecating start_httpd solved the problem. It's a substantial problem that the Marionette test runner is so tightly-coupled as it discourages change. Pushed the patch to inbound and keeping fingers crossed.

Filed bug 1152865 about incorrect trychooser syntax for gaia-ui-test jobs.

2015-04-10

Spent the morning replying to some emails and PRs to do with WebDriver.

The patch for bug 941085 was backed out yet again after some instability on Mn-e10s this time. Either e10s isn't being entirely truthful to us about its load events, or WebElement.click is doing something very wrong. I added another explicit wait for the URL to change after the submit element is clicked, and pushed it back up to inbound.

It turns out bug 1150041 inadvertently fixed the wrong issue, so filed bug 1153240 to address the real issue. njpark reviewed and I pushed to inbound.

2015-04-11

Resurrected old patch for bug 984921 and pushed that to review.

Fixed up PR 135 to the Marionette Node.js client with more tests. Turns out I had missed some pretty glaring scenarios.

2015-04-13

Reviewed r/4651, and filed bug 1153822, bug 1153911, bug 1153828, and bug 1153832.

Sent an email to the WebDriver WG about including a stacktrace field for error responses.

Begun work on bug 1153832 to introduce the new dispatching technique to the listener.

2015-04-14

Continued work on bug 1153832. Got everything mostly working. One remaining failing test.

Attended internal MoCo meeting.

2015-04-15

Continued work on bug 1153832. Pushed try run. Also pushed it to review to get some early feedback. Later in the evening the review was accepted and I pushed it to inbound.

Filed bug 1154691 to align the errors in Marionette with those mandated by WebDriver. Pushed to review and try.

Also submitted PR 29524 for gaia but had some troubles with the PR title and autolander not cooperating.

Filed bug 1154681 about using a static lookup table for errors.py. Will do this after bug 1154691.

Reviewed r/4669. Provided some feedback on bug 1141418, but it's in all honesty a silly bug.

Had 1:1 with dburns. Need to make my deliverables slightly more concrete to properly visualise the amount of work involved.

Made bug 1100545 the tracking bug for correcting errors used in Marionette to match those mandated by the WebDriver specification. This currently depends on bug 1154757 and bug 1152682.

2015-04-16

Today is a WebDriver specification work day with dburns and jgraham. This is an initiative to try to get together for one day a month to make progress on the specification.

Sent an updated version of my Q2 deliverables with more concrete actions to dburns.

Found two bugs in bug 1153832 which caused it to get back out yesteday. Pushed a new try run to verify.

Pushed bug 1154691 to inbound. It was later backed out because the change to Gaia hadn't landed. Got davehunt to review that and landed it on Gaia's Github repo. Gaia on Github then needs to sync to Mozilla's internal git repo, which has to sync to hg.

Submitted PR 102 to wptrunner that removes InvalidResponseException because wptrunner uses the in-tree Marionette client. So this is a requirement to land bug 1154691.

2015-04-17

Reviewed PR 28 to WebDriver, and it's both unnecessary and wrong. Also reviewed PR 29.

Bug 1153832 was backed out due to failures in Gip(a). The investigation showed that because XPCOM is a special snowflake in this world, it throws an error on Object.getPrototypeOf(err) if err is an XPCOM/NS exception. Instead are forced to check if "result" in err because hasOwnProperty doesn't follow the prototypal chain and generally causes more problems.

Now, the simple test assertion framework that somebody thought was a really good idea to build into Marionette returns {"result": true} when an injected script calls marionette.generate_results. This causes the error code path to be triggered, but since it's missing some properties required to serialise the error, the code that serialises it throws and we return back up the stack frame to wrapping it as a normal value.

This causes the error to be printed to the browser console, and because of a bug that error.report always expects an Error prototype prints “Marionette threw an error: [object Object]”. But since we're still treating it as a regular value, this is serialised by the regular executeScript return value serialiser which introspects the properties of the object and writes a {value: {error: "err is undefined"}} value, which is returned over the socket as a normal packet.

To summarise, we need a way for error.isError to positively tell if an error is an XPCOM exception or not, and we can't look at the result property alone to do this. The solution I came up with deserves to be posted here for posterities sake:

// Because XPCOM is a cesspool of undocumented odd behaviour,
// Object.getPrototypeOf(err) causes another exception if err is an XPCOM
// exception, and cannot be used to determine if err is a prototypal Error.
//
// Consequently we need to check for properties in its prototypal chain
// (using in, instead of err.hasOwnProperty because that causes other
// issues).
//
// Since the input is arbitrary it might _not_ be an Error, and can as
// such be an object with a "result" property without it being considered to
// be an exception.  The solution is to build a lookup table of XPCOM
// exceptions from Components.results and check if the value of err#results
// is in that table.
const XPCOM_EXCEPTIONS = [];
{
  for (let prop in Cr) {
    XPCOM_EXCEPTIONS.push(Cr[prop]);
  }
}

Bug 1153832 is now ready for integration, but inbound is closed again.

Reviewed r/7133.

Filed bug 1155658 about the IllegalArgumentError in the Marionette server carrying the wrong string code; it should be “invalid argument”.

Also filed bug 1155660 about investigating the possibility for using individual message listeners per browser.

Submitted PR 30 to WebDriver which reorganises some content near the start of the specification and generally improves the flow of the document.

Filed bug 1155703 about getting rid of FrameSendFailureError and FrameSendNotInitializedError. Proposed a migration plan.

Filed bug 1155708 about moving Marionette.Element#location and Marionette.Element#size in the Marionette Node.js client to using getElementRect internally. This will be in preparation for landing bug 1153911.

Filed bug 1155716 about removing the submitElement command from Marionette and managing the risk of doing that throughout our codebase.

Patched bug 1154525 and pushed it to review.

Patched bug 1155658 and pushed it to review.

2015-04-20

Bug 941085 caused the regression bug 1155260. It looks like setting the baseurl property on Marionette is missing. Pushed r/7289 to review. Also triggered a full try run.

Landed bug 1154691 on inbound again as jgraham pushed an update to wptrunner.

Rebased and fixed up bug 945729, and triggered a new try run.

Patched bug 1152682, created r/7293, and pushed a try run.

Patched bug 1154757, created r/7301, and pushed try run.

My patches for bug 1153832 and bug 1154691 were backed out by the sheriffs because suspicion of breaking the dt job on Linux x86 asan, which is silly.

2015-04-21

Relanded patches for bug 1153832 and bug 1154691 with agreement from tomcat.

Pushed bug 945729 to inbound, which removes number errors from the Marionette server. Fingers crossed.

Patched bug 1154681 about static error lookup tables in the Marionette Python client. Created review r/7351 and pushed a try run.

Bug 945729 was backed out of inbound for bustage of wpt tests. Turns out that the isOldStyleError check was still in place, and the message structure was exactly the same used by wptrunner. Removed this test and the tests are now passing. Triggered a full full try run to verify.

2015-04-22

Triggered another try run of bug 945729 which failed spectacularly. Since it was based off inbound, I grafted the patch onto central and triggered yet another one.

chmanchester noticed an interesting issue in my patch for bug 1154525, which is about making HTMLElement.location and HTMLElement.size use HTMLElement.rect internally. It turns out that the implementation of getElementLocation in Marionette has been wrong all long, as it returns the position of the element relative to the viewport instead of the position relative to the document. We probably want to fix this, but we need to be absolutely sure we don't break any existing test code that relies on this behaviour for further calculations. I triggered a full try run to determine this.

Found that it's extracting the values from the dict incorrectly. Triggered new try job to verify that it was fixed.

Filed bug 1157192 about turning the listener into an object. Started working on this.

Filed bug 1157253 about moving ListenerProxy to use the ES6 Proxy concept.

Filed bug 1157255 about looking at a symbol's type instead of using try…catch when binding properties and functions to the content sandbox.

Filed bug 1157257 about looking at the type of the error raised in Marionette, and if it's not a recognised WebDriver error we should concatenate the Error's name and message property in the message as we marshal to send it back to the client.

Filed bug 1157258 about fixing null comparisons in the Marionette server.

Filed bug 1157259 about fixing the reference to the error ScriptTimeout in the listener.

Filed bug 1157261 about correcting the cuFrame reference in listener.js:441.

Filed bug 1157266 about fixing the reference to the error UnableToSetCookie in the listener.

Filed bug 1157271 about fixing variable assignments using var in the listener.

Filed bug 1157273 about making the deleteSession function in the listener synchronous.

Reviewed r/7439.

Spent most of the day working on bug 1157192 about turning the Marionette listener into an object.

1:1 with dburns was cancelled.

2015-04-23

Filed and fixed bug 1157665 based on some feedback from hallvord. Pushed to review and try.

Patched bug 1157257 that includes an error's name in the marshaled WebDriver error object.

Continued work on bug 1157192. The patch is mostly ready so I pushed a try run. The try run revealed some more issues, so triggered a new one after a few fixups.

Patched bug 1157253 to use a Proxy object instead of __noSuchMethod__, which is deprecated. Pushed r/7507 and a try run.

Reviewed r/7519.

2015-04-24

Pushed bug 1157665 and bug 1155658 to inbound.

Filed and fixed bug 1158113 with r/7599 and try.

Fixed up some issues with bug 1157253, about getting rid of the use of __noSuchMethod__ in ListenerProxy. chmanchester pointed out some architectual problems with how the constructor for ListenerProxy now returns a Proxy object instead of itself.

Worked on bug 1157192 and did a try run.

Did bug 1157258 which fixes null comparisons in driver.js.

Patched bug 1155708 as PR 138.

Filed bug bug 1158219 about the Marionette Python client sending the field id when switching to the default browser frame. It shouldn't be set at all. Discovered this as I was working on bug 1157258.

2015-04-27

Rebased and fixed up a nitpick on bug 1157258, and pushed it to inbound.

Rebased and pushed bug 1158219 to inbound.

Fixed up remaining issues for bug 1157253 which is about porting the ListenerProxy to use the ES6 Proxy object. An important issue raised by chmanchester was that it was reflecting upon itself, and returning a proxy instance from the constructor. I addressed this by distinguishing between the two proxies we have here: The first is the reflection proxy on the JS object, the second the interface to provide a nicer API to content space. This means we now have ListenerProxy returning a Proxy instance, reflecting on CurrentContentSender, which used to be called ListenerProxy.

Pushed a try run and to r/7507.

Work on bug 1157192 to make the listener an object.

2015-04-28

Filed bug 1159219 as a follow up to bug 1157253 about splitting ListenerProxy and ContentSender to a separate JS module, and generalising ContentSender to not always work on the current browsing context's content frame.

Submitted bug 1159219 to review and made a try run.

Reviewed PR 502 in Selenium.

Filed bug 1159232 to avoid use of ES6 template strings in GeckoDriver#sendAsync.

Filed bug 1159238 about generating a UUID internally to ContentSender.

Submitted bug 1159232 to review and made try run.

Submitted PR 31 to the WebDriver specification that adds a stacktrace field to the error responses.

Submitted patch for bug 1159238 that generates a UUID internally to ContentSender when making calls to content space, instead of using the current command ID.

Filed bug 1159358 after investigation I did on bug 1159238 and a conversation with jgriffin about how the ContentSender could potentially be aware of remoteness changes internally.

Pushed bug 1157257 to inbound.

Fixed bug 1157266 that corrects the UnableToSetCookieError error in the listener. Submitted to r/7767 with try run.

Fixed bug 1157259 that corrects the ScriptTimeoutError error in the listener. Submitted to r/7771 with try run.

Fixed bug 1157255 that uses typeof to determine how to export properties and functions from MarionetteObj on the sandbox. Review is r/7787 and there is also a try run.

2015-04-29

Pushed bug 1159219, bug 1159238, bug 1159232, bug 1157266, and bug 1157259 to inbound.

Fixed up review issues for bug 1157255.

Landed PR 502 to Selenium.

Resolved bug 984595 as a duplicate of bug 885982.

Submitted PR 33 to the WebDriver specification which adds "firefoxos" as a supported operating system for the platformName capability, and simplifies defining vendor extension OSes.

Fixed bug 1155716 that removes the submitElement command from Marionette. To my great surprise, it wasn't implemented in the Node.js client and not used in the Gaia tests. The review is r/7831 and pushed a try run.

Submitted PR 29794 for bug 1159674 in preparation of bug 1155703 to remove FrameSendFailureError and FrameSendNotInitializedError symbols from Marionette.

Fixed bug 1155703 to remove these symbols. Review can be found in r/7837 and there is also a try run.

2015-04-30

Pushed bug 1157255 and bug 1155716 to inbound.

Rebased patch for bug 1159674 which re-triggered tests to pick up correct marionette-client version, which is what caused the test failures.

Rebased bug 1155703 and retriggered try run in hope that Gaia has been bumped on central.

Reviewed PR 34 to the WebDriver specification that attempt to define New Session.

Fixed up another issue in PR 138 for marionette-js-client.

Reviewed PR 32 to the WebDriver specification that attempts to define processing capabilities.

Spent the evening hacking on rustfmt.