Web Driver Tutorial^{since 1.1.0}

Introduction

Web Driver Sampler automates the execution and collection of Performance metrics on the Browser (client-side). A large part of performance testing, up to this point, has been on the server side of things. However, with the advancement of technology, HTML5, JS and CSS improvements, more and more logic and behaviour have been pushed down to the client. This adds to the overall perceived performance of website/webapp, but this metric is not available in JMeter. Things that add to the overall browser execution time may include:

1. Client-side Javascript execution - eg. AJAX, JS templates
2. CSS transforms - eg. 3D matrix transforms, animations
3. 3rd party plugins - eg. Facebook like, Double click ads, site analytics, etc

All these things add to the overall browser execution time, and this project aims to measure the time it takes to complete rendering all this content.

2 Minute Guide

To get started, first follow the installation instructions and then create a basic Test Plan with a Thread Group containing 1 thread. The next steps will cover adding the following elements to your test plan:

1. Firefox Driver Config
2. Web Driver Sampler
3. View Results Table

Your basic Test Plan should look like this after adding the Thread Group:

Firefox Driver Config

Add a Firefox Driver Config to your Thread Group.

Web Driver Sampler

Add a Web Driver Sampler to your Thread Group.

Add the following Javascript code to the Script section to navigate to Google.

WDS.sampleResult.sampleStart()
WDS.browser.get('http://google.com')
WDS.sampleResult.sampleEnd()

View Results Table

Add a View Results in Table to your Thread Group and then run your test. You should see something like the screenshot below that captures the result of the sample.

10 Minute Guide

The 2 minute guide is useful to get something up and running, but at the cost of some understanding. This section hopefully fills in some of the gaps/questions that can come from going through the previous guide.

Thread Group

JMeter allows the creation of multiple threads, and each thread is responsible for creating load on the server. However, for the Web Driver use case, the reader should be prudent in the number of threads they will create as each thread will have a single browser instance associated with it. Each browser consumes a significant amount of resources, and a limit should be placed on how many browsers the reader should create. However, the load testing cloud providers may help to scale WebDriver test up to thousands of real browsers, look at BlazeMeter for example.

From experience, the number of browser (threads) that the reader creates should be limited by the following formula:

C = N + 1

where C = Number of Cores of the host running the test
and N = Number of Browser (threads).

eg, if the current reader's host has 4 cores, the formula would yield:

4 = 3 + 1

meaning that the script should have a MAXIMUM of 3 threads.

This might should a little limiting. However, it is important to note that this is mainly used to complement JMeter's HTTP Sampler. The Web Driver Sampler is meant to be run in conjunction with a HTTP Sampler so that from the server's perspective, the load is production like. At the same time, the web sampler will simulate the user experience of interacting with the website/webapp whilst the server is under load, hence measuring the real user's experience at the same time.

Firefox Driver Config

For the `Web Driver Sampler` to work, it will need to know what type of browser to run the tests with. The simplest browser to use is Firefox, as there are no external dependencies required (unlike the Chrome Driver Config). As such it has been chosen for the quick tutorial. For a more thorough understanding, it is useful for the reader to try out different Driver Configs to better evaluate which browser best suits their needs.

At the time of writing, only Chrome and Firefox browsers are supported. However the reason why Web Driver was chosen is because of the breadth of browsers it can control and script. The roadmap has more details on when more browsers will be added, however the initial browsers are just a starting point, and more browser support will be added in the future.

Web Driver Sampler

There is more in depth information on the Web Driver Sampler page. However, it is important to understand that the content of the script editor is to allow the user total control over when the interactions with the browser as well as when the sampling should be conducted. This last part is important as it will allow the reader to not only measure the amount of time it takes to load a URL (as the above example), but also to measure AJAX requests as well (more on this in the Web Driver Sampler page).

In the below example, the `Browser.get()` method call will make the browser load the specified URL. The `SampleResult.sampleStart()` will start timing and then calling `SampleResult.sampleEnd()` afterwards will stop the timing and allow JMeter to use the `SampleResult` when measuring the results in the `View Results in Table` component.

The `Browser` object exposed in the `Script` section is an instance of the `WebDriver` object documented in the Selenium documentation. It is recommended that the reader have a look at the documentation to see what methods are available on the WebDriver API, to better understand what can be scriptable on the Browser instance.

It is also up to the reader to ensure that they verify that the correct page has been loaded and then setting the corresponding `SampleResult.setSuccessful(boolean)` method when invoking this script. Since the full WebDriver instance is available to the reader, they should be able to verify that the page has loaded successfully and then set the correct status on the SampleResult. The example below will verify that the page contains the correct title and uses this to indicate success/failure on the SampleResult.

WDS.sampleResult.sampleStart()
WDS.browser.get('http://google.com')
WDS.sampleResult.sampleEnd()

if(WDS.browser.getTitle() != 'Google') {
    WDS.sampleResult.setSuccessful(false)
    WDS.sampleResult.setResponseMessage('Page title is not Google!')
}

For a more thorough understanding, the reader should familiarise themselves with the SampleResult documentation as well, to better understand what is scriptable on the SampleResult object.