Persistent connections
in WebDriver

A bug was recently reported that some of WebDriver’s error codes used the 408 Request Timeout HTTP status code and that this is in conflict with HTTP clients trying to use persistent connections:

It might be logically correct to use this error code, but we have a collision with HTTP semantics implemented in HTTP clients. To support Keep-Alive we allow retries in HTTP clients, and if a client sees code 408 it thinks that the server has gone and attempts to retry the request. As a result, we see duplicate calls to Execute Script, Navigate To, or Refresh subsequent to a timeout.

The Keep-Alive header enables persistent, long-lived connections over HTTP. These help speed up command requests to the driver by not creating throwaway TCP sockets for each transaction. Reusing the same socket is great for request-intensive protocols such as WebDriver.

The eminent Mark Nottingham has written a guide to thinking about HTTP status codes that it is possible to draw advice from:

The first thing to know about HTTP status codes is that they are structured; the first digit indicates what kind of response it is, and the other two indicate specific handling for the response. The kinds of responses defined are:

So, for example, 201 Created is a 2xx status code, indicating a successful response, and the 01 specialisation indicates that the request created a resource, located at the URL indicated by the Location header field.

Likewise, a 409 Conflict is a 4xx status code, so it means that there was a problem with the request, rather than the server’s handling of it. 09 on a 4xx tells the client that its request conflicted with the server’s state, and that conflict needs to be resolved before the request is sent again.

We ended up choosing 500 Internal Server Error as a replacement. It is unfortunate to make backwards incompatible changes to WebDriver, but the performance penalty of not being able to support Keep-Alive in a compatible fashion was deemed too big a sacrifice.

My educated guess is that the change won’t require any client changes, as these mostly don’t assert or validate the status code of an error response. The upside is that you can now enjoy faster WebDriver sessions in conforming driver implementations.