Test Driven Development

I've read a lot for and a lot against Test Driven Development (TDD) but I don't every remember something TDD has really helped me with. If you write a test then write the code which makes the test pass the end result is a piece of code which demonstrates how the application code works. I have used a number of open source applications and like every library/framework/application I have used the documentation ends up being way out of date. But if you are writing tests for all the features you add (before or after you write the code) you are essentially creating examples of how to use the library/framework/application. So even with the documentation falling out of date, so long as the code has a good base of unit tests, you can figure out how it works. I find this helpful in multiple ways. When I'm automating I tend to use tools like Selenium. Whenever I need to use a new feature of Selenium I can look at the documentation but ultimate, the unit tests for the framework tell me how it actually behaves. Additionally, the applications I am testing are written by people who understand TDD and have a good set of unit tests for each application. So I find it easy to under how new features are implemented by looking at the unit test for the application I am testing. For example, Selenium has a TakesScreenshot class. The source for it would be found at:

https://code.google.com/p/selenium/source/browse/java/client/src/org/openqa/selenium/TakesScreenshot.java

So if I want to see the tests for it, change 'src' to 'test' and change 'TakesScreenshot.java' to 'TakesScreenshotTest.java' for a full URL of:

https://code.google.com/p/selenium/source/browse/java/client/test/org/openqa/selenium/TakesScreenshotTest.java

When I look at this latter file I see many examples of how to use the TakesScreenshot class.

WebDriverWait versus FluentWait

The WebDriver code waits for the page to load but today we have dynamic websites using things like angularjs and Ajax. WebDriver does not wait for the javascript to execute. So you might have something like:

<a href="http://{{env}}.company.com/foo/bar.html">{{env}}</a>

When the page loads, javascript runs and converts {{env}} to some defined value. So on the test environment {{env}} might convert to test, on stage it converts to stage and on production it converts to www. However, WebDriver will not wait for the javascript to make the substitution.  The end result is clicking the element will cause WebDriver to go to "{{env}}.company.com" when we really wanted it to wait and go to "www.company.com".

So how do we make WebDriver wait for the variable to be updated by javascript?

The answer is WebDriverWait or FluentWait. Below are examples of how you can use WebDriverWait and FluentWait to wait for the javascript to finish:

import com.google.common.base.Function;
import com.google.common.base.Predicate;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.*;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.support.ui.*;
import java.util.Arrays;
import static java.util.concurrent.TimeUnit.*;
import static org.junit.Assert.assertFalse;

public class TestingFluentWaitAndWebDriverWait {
  WebDriver driver;
  WebElement button;

  @Before
  public void setUp() {
    System.setProperty("webdriver.chrome.driver", "/Users/ThoughtWorks/IdeaProjects/SeleniumTesting/chromedriver2");
    DesiredCapabilities dc = DesiredCapabilities.chrome();
    dc.setCapability("driver.switches", Arrays.asList("--start-maximized"));
    driver = new ChromeDriver(dc);
    driver.manage().window().maximize();
    driver.get("http://www.google.com");
    driver.findElement(By.cssSelector("#gbqfq")).sendKeys("\"Darrell Grainger\"");
    button = driver.findElement(By.cssSelector("#gbqfb"));
  }

  @Test
  public void testWebDriverWait() {
    new WebDriverWait(driver, 3).until(ExpectedConditions.visibilityOf(button)).click();
    assertMyBlogLinkExists();
  }

  @Test
  public void testFluentWait() {
    new FluentWait<WebElement>(button).withTimeout(3, SECONDS)
        .pollingEvery(100, MILLISECONDS)
        .until(new Function<WebElement, Boolean>() {
          public Boolean apply(WebElement w) {
            return w.isDisplayed();
          }
        });
    button.click();
    assertMyBlogLinkExists();
  }

  @Test
  public void testFluentWaitPredicate() {
    new FluentWait<WebElement>(button).withTimeout(3, SECONDS)
        .pollingEvery(100, MILLISECONDS)
        .until(new Predicate<WebElement>() {
          public boolean apply(WebElement w) {
            return w.isDisplayed();
          }
        });
    button.click();
    assertMyBlogLinkExists();
  }

  private void assertMyBlogLinkExists() {
    final String linkText = "QA & Testing";
    try {
    new FluentWait<WebDriver>(driver).withTimeout(3, SECONDS)
        .pollingEvery(100, MILLISECONDS)
        .ignoring(NoSuchElementException.class)
        .until(new Function<WebDriver, Boolean>() {
          public Boolean apply(WebDriver d) {
            WebElement link = d.findElement(By.linkText(linkText));
            return link.isDisplayed();
          }
        });
    } catch(TimeoutException te) {
      assertFalse(String.format("Timeout waiting for link: '%s'", linkText), true);
    }
  }

  @After
  public void tearDown() {
    if(driver != null)
    driver.quit();
  }
}


All three tests will result in the same outcome.

Creating good locators

I am constantly seeing people look for tools which will create locators. I think a lot of us starting by using Firebug to examine the DOM and select locators. With Chrome I can now right-click on an element and select Inspect Element.

Now of this really helps figure out what is a good locator. So people are still looking for a tool which will tell them the best locator for an element.

If I go to http://www.google.com and inspect the Google Search button I will see:

<button class="gbqfba" aria-label="Google Search" id="gbqfba" name="btnK">
<span id="gbqfsa">Google Search</span>
</button>

We might think that the Selenium locator By.cssSelector("#gbqfba") might be a good locator. One definition of a good locator is that it needs to be consistent and unique. For years the Google Search button has had the id attribute of gbqfba. So it is consistent. It is an id and according to the HTML standard there can be only one element for each id value. So it will be unique.

But is it a good locator? I will argue that it is not a good locator. If I had the following Selenium code:

driver.findElement(By.cssSelector("#gbqfq")).sendKeys("Darrell Grainger");
driver.findElement(By.cssSelector("#gbqfba")).click();
assertThat(driver.findElement(By.xpath("//a[contains(text(),'Testing')]")).getText(), startsWith("QA"));

it will run consistently over years. But what happens if something changes and I need to refactor this code? The first line is sending my name to something with id='gbqfq', it is clicking something with id='gbqfba'. What if we changed the HTML to have:

<button class="gbqfba" aria-label="Google Search" id="gbqfba" data-qa='search button'>
<span id="gbqfsa" data-qa='text for search button'>Google Search</span>
</button>

then the Selenium code could read:

driver.findElement(By.cssSelector("[data-qa='search input box']")).sendKeys("Darrell Grainger");
driver.findElement(By.cssSelector("[data-qa='search button']")).click();
assertThat(driver.findElement(By.cssSelector("[data-qa='first search result']")).getText(), startsWith("QA"));

Now if I'm looking at the code and have no knowledge of the website being automated, it should be easy for me to figure out that the first element is the search input box, the second element is the search button and the last element is the first search result.

Essentially, by adding the data-qa attribute the elements we are trying to locate:

• easier to locate the element
• easier to read the code when running/refactoring it
• make the code less brittle

A possible complaint would be that adding these elements could interfere with other elements on the page. The whole 'data-' syntax is a standard for HTML5. If there is a change someone is going to use data-qa then you can use data-foo where foo is the name of your application or your company name or your name.

Another possibility is that it is not unique. It only has to be unique to the given page. If the value is incredibly descriptive then it should be unique. You make the value of data-qa reflect the information necessary for someone testing the application manually then it should be unique. If it is not unique then how would a manual tester find THE element if its description was not unique?

I also considered adding more data to the page would increase what the CSS and XPath engines had to index more information. So I tried loading up page with these attributes added and could not measure any noticeable impact on the performance.

The only other concern I can image is if I add too much data or too verbose descriptions on the data-qa values then I could increase the amount of data transmitted. More bytes transmitted might impact performance of the web page. Wasn't able to see any significant impact in this area but your mileage might vary. So this is something to watch out for.

Forcing a remote computer to restart

Occasionally I find I need to reboot a remote computer. In my current situation the remote computer is Windows XP and accessed via VNC. However the VNC server will occasionally stop responding.

If the machine's IP address was 1.2.3.4 and I try:

shutdown /m 1.2.3.4 /r /t 1

it does not reboot the computer. Assuming there is nothing to save on the computer and I just want it to reboot and restart the services, I am happy to just kill processes until I hit something critical and it reboots.

However, if you hit the RPC process, you will be unable to send the command to kill more processes. So you have to be careful about which process you are killing. What I have found is killing the lsass.exe process will cause the computer to reboot. So issuing:

taskkill /im "lsass.exe" /s 1.2.3.4 /f

Will kill the lsass.exe proccess on the remote machine and force a reboot of that computer. The reboot will not be immediate. A 30 second countdown will happen as the system warns you a critical process was killed. However it will reboot the computer after the countdown is complete.

Browser statistics

A lot of discussion goes into browser statistics. Whenever a company is creating a new web application they need to decide which browsers will be supported and their order of importance.

Years ago it was a given that Internet Explorer was the #1 browser. Just the shear volume of Windows users and lack of serious competition in that area made Internet Explorer the dominant browser.

Then along came Firefox and later Chrome. Additionally, Mac Desktop is becoming a large enough market that Safari is worth considering.

So how do we decide which browsers we should support and what order we should rank them in?

There are many sites which give statistics on browser usage:

W3 SchoolsStatCounterW3 CounterClickyNetMarketShareWikiMedia

and probably a whole lot more to check. If we look at the different sites we can see some radically different numbers. Some claim Chrome is close to 60% of the market. Others have Chrome around 30% of the market.

Why the big difference? Different sites will gather statistics on different people. The W3 Schools site is very popular among programmers and software testers. I tend to like Chrome because it has the built-in Inspect feature and other helpful development tools. Firefox is okay when you install Firebug or other tools but often we want to test with a clean browser and don't have these tools available.

Starting with Internet Explorer 8 it came with development tools. The tools in Internet Explorer 9 are better but nothing as good as the tools in Firefox or Chrome. So Chrome and then Firefox are going to be the browser of choice for programmers and software testers.

Not surprisingly, W3 Schools is the site which claims Chrome is almost 60% of the market. But it will be biased towards power users, programmers and testers. My mother doesn't use a computer. If she did it would be a Windows 7 with Internet Explorer 11 or a Mac with Safari 6.1.

So which statistics site you select depends on your target audience. You don't want to use the statistics from W3 Schools if your target audience is my mom.

What if you have a website and you are going to be updating it or adding another website with the same target audience? Then you have been hopefully gathering statistics on your existing website. Statistics from your existing website will tell you exactly who your users are. If you statistics say that 97% of your users are using Internet Explorer 6 then you need to support Internet Explorer 6. You could make it painless to upgrade your existing user base to a new version of the same browser but odds are it is safer to just support what your users are using.

The only time I would say you might not want to completely follow the statistics on an existing website would be if you are attempting to grow your market and know the new customer base is going to be a different type of user. Let's say that my user base is traditionally Internet Explorer users but I want to start targeting Mac users. I want to make sure that I don't sacrifice existing users for the new user base. So I will continue to value the Internet Explorer users but I want to start looking at statistics for when the operating system is Mac OS X. If I drill down to just Mac OS X users, which is the dominate browser?

In addition to which browser there is also the resolution for the display. This tends to be a little easier to decide if we are talking desktop. You pick the small display which is widely supported. Originally, I would design a website so it looked good on 640x480. Then the average display was 800x600. Later we would start seeing wide displays like 1440x900. I found it easiest to set a minimum size that would be supported. So you don't want so many things on a menu that the menu is longer than say 600 pixels high if you supported 800x600. We can use tools to resize the browser to the different sizes and make sure things still look good. This is easy to do on pretty much any desktop today.

Back in the days of EGA and VGA the number of colours you would support was an issue but today it isn't really something people worry about any more.

This is how I generally recommend clients decide on which desktop browsers to support. It does not really address the idea of mobile browsers. This is an area which is still in flux. I see a lot of different mobile browsers. My own personal experience has been updating my browser can make it more unstable. So if I have a browser which runs on my device I'll stick with it. However, if I get a new device I tend to find the newer browsers come with it and run better than old browsers. So you should be seeing a wider range of browsers on mobile devices than on desktops. Additionally, mobile devices include smart phones and tablets. There are full size tablets, mini tablets, large smart phones, medium smart phones and small smart phones. The different resolutions can vary much more than desktop devices.

Additionally, I cannot take a large device and resize the browser to small sizes. The idea of a window on a desktop doesn't exist on many mobile devices. So testing on the different resolutions is a major concern. Fortunately, with emulators you can run a software emulated version of a device and check that the website displays properly on it. Or you can use features of our existing browser to pretend to be a specific mobile device. This will change the size and User-Agent information to make your web application display as if it was getting rendered on a mobile device. Again, Chrome Inspect has some built-in features in this area which are quite helpful. They should not replace emulators or even better actual devices when doing system testing.

Ideally, in a test lab I will have all the machine with the largest display we want to support. I will then use a virtual computer player like Virtual PC, Parallels, VMware, etc. to emulate all the other operating systems. A Mac works really well because you can use it to run Mac browsers, a virtual PC like Parallels or Fusion to run Windows, Linux and Solaris x86, XCode to emulate iOS devices and Eclipse or IDEA to emulate Android devices.

If the client runs mostly Microsoft products and Mac isn't a concern then you can use Virtual PC on a Windows 7 device to run Windows XP and Windows 7 virtual machines for free.

For virtualizing all the Windows machines right now you can go to http://www.modern.ie/en-us/virtualization-tools#downloads to download virtual machines for Windows XP, Windows 7 and Windows 8. Everthing from Internet Explorer 6 to Internet Explorer 11.

Opening two windows for one WebDriver

I'm currently testing a project with two applications. The first is an administrator tool for updating/modifying the customer site and the second is the actual customer site.

There are three ways to test a project like this. The first is creating one instance of WebDriver and access each application one at a time, e.g.

WebDriver driver = new ChromeDriver();
driver.get(adminToolURL);
// do some admin stuff
driver.get(customerSiteURL);
// do some site stuff

The second would be to create two instances of WebDriver. One to access the admin tool and the other to access the customer site, e.g.

WebDriver adminDriver = new ChromeDriver();
adminDriver.get(adminToolURL);
WebDriver driver = new InternetExplorerDriver();
driver.get(customerSiteURL);
// do some admin stuff with adminDriver
// do some site stuff with driver

The nice thing about this scenario is that you might have different browser requirements. The admin tool only has to work with one browser and the customer site has to work with a variety of browsers. So I can hard code the adminDriver to one browser and configure the test framework to change the browser. Run 1, InternetExplorerDriver; run 2, ChromeDriver; run 3, SafariDriver; etc.

The third way is to create one instance of WebDriver and open two windows. If you use the driver.get() method, it will open the site in the current window. To open a second window you'll need to use something in Selenium to force a second window. The trick is to use JavascriptExecutor to run javascript which opens a second window, e.g.

WebDriver driver = new ChromeDriver();
driver.get(adminToolURL);
Set<String> windows = driver.getWindowHandles();
String adminToolHandle = driver.getWindowHandle();
((JavascriptExecutor)driver).executeScript("window.open();");
Set<String> customerWindow = driver.getWindowHandles();
customerWindow.removeAll(windows);
String customerSiteHandle = ((String)customerWindow.toArray()[0]);
driver.switchTo().window(customerSiteHandle);
driver.get(customerSiteURL);
driver.switchTo().window(adminToolHandle);

The first two lines are straight forward and open a window with the admin tool. The next few lines does the following: get a list of the currently open windows, save the window handle for the admin tool window, open a new window using executeScript, get a second list of the currently open window (this will be the same as the first list PLUS the new window). Next remove all the original windows handles from the second list. This should leave the second list (customerWindow) with only one window handle, e.g. the new window. The last three lines show you how to switch between the customer site window and the admin tool window.

WebDriverWait

In an older article I wrote about a method I created called waitForElement. After looking through some of the WebDriver code I found WebDriverWait. You should read the original article before you read this one.

The WebDriverWait class extends FluentWait<WebDriver> which is a special version of FluentWait which has WebDriver instances.

In the article for waitForElement I was talking about how Selenium cannot detect that Javascript has finished altering the DOM. So if you try to interact with an element with Selenium and Javascript is still altering it, you will have undefined results. The waitForElement method would wait for the element to be full rendered, i.e. for the Javascript to be finished. The WebDriverWait serves the same purpose.

To define a WebDriverWait would be the following:

WebDriverWait wdw = new WebDriverWait(driver, timeoutInSeconds, pollingTimeInMs);

The first parameter is an already existing WebDriver element. The second parameter is how long we want to wait for the element to be present. If we set it to say 30 it will wait for 30 seconds before it throws an error. The third parameter is how often you want to check for the element. If you set it really low, e.g. 10, it will check every 10 milliseconds. However this could put a load on the test machine. If you know it typically takes 55 milliseconds then maybe waiting for 60 would be best. You would not want to wait for 1000 milliseconds as it would make your test too slow.

This just creates the WebDriverWait object. It doesn't actually do the waiting. Let's say you click the ACCEPT checkbox. This causes Javascript to make the Next button visible. If you just click the ACCEPT checkbox then click the Next button it might fail because the Next button isn't visible as quickly as you can click it. So you need to click the ACCEPT checkbox, wait for the Next button to be visible then click the Next button. Here is some example code:

long timeoutInSeconds = 30;
long pollingTimeInMs = 250;
WebDriverWait wdw = new WebDriverWait(driver, timeoutInSeconds, pollingTimeInMs);
wdw.until(visibilityOfElementLocated(By.id("next"))).click();

This will check every 250 milliseconds to see if the element located by id='next' is visible. If the element does not become visible in 30 seconds it will fail the step and throw an error.

The list of conditions you can wait for are:

presenceOfElementLocated(by);
visibilityOf(driver.findElement(by));
alertIsPresent();
elementSelectionStateToBe(by, true);
elementSelectionStateToBe(we, true);
elementToBeClickable(by);
elementToBeSelected(by);
frameToBeAvailableAndSwitchToIt(frameLocator);
invisibilityOfElementLocated(by);
invisibilityOfElementWithText(by,text);
presenceOfAllElementsLocatedBy(by);
textToBePresentInElement(by, text);
textToBePresentInElementValue(by, text);
titleContains(title);
titleIs(title);
visibilityOf(we);
visibilityOfElementLocated(by);

The WebDriverWait also lets you alter the polling time, timeout, message displayed when it times out and exceptions it should ignore while waiting:

wdw.pollingEvery(delayBetweenPolling, TimeUnit.MILLISECONDS)
.ignoring(NoSuchElementException.class)
.withTimeout(timeoutInSeconds, TimeUnit.SECONDS)
.withMessage(message)
.until(visibilityOfElementLocated(By.id("next"))).click();

Because you can set the unit of measure for polling time and timeout, you can refine these after you instantiate the WebDriverWait object.

.