CSS for WebDriver revisited

I always liked using XPath for locators in Selenium and WebDriver. It was more procedural than using Cascading Style Sheets (CSS) and felt more like programming. However, using CSS for locators is much faster than using XPath. I haven't done any tests to see just how much faster but it is significant enough that I can see the difference.

I run a test suite using XPath locators. I can see significant delays where there appears to be nothing happening on the screen. I refactor the code to use CSS and the delays disappear. I would estimate that 3 to 5 second delays (XPath) become too small to measure with the naked eye (CSS).

The only feature I have been unable to do in CSS are locators which use the XPath text() function. CSS seems to be all about the tag and its attributes but not the text contained within the tag.

The one thing it took me a little while to get used to when switching from XPath to CSS was compound statements. For example, I might have an XPath like:

    "//div[contains(@class,'prefspanel')]/table[@id='sizer1']"

So how do I convert this to CSS?

    "div[class~='prefspanel']>table#sizer1"

Shorter and pretty much the same attributes. The ~= isn't quite the same as the contains() function. If I have:

    class='prefspanel foo bar'

the ~= will find prefspanel because it is a 'word' separated by whitespace. The contains() function will find 'prefs', 'prefspan', 'prefspanel' and even 'panel' because these are all substrings of the original. In CSS3 they introduced *= which is just like contains().

I also notice that many attributes will change dynamically. However, the start of the attribute or the end of the attribute will be consistent. In XPath I have to use the contains() function to find the attributes which end with the unique identifier. With CSS I can use ^= to see if the attribute begins with a substring or $= to see if the attribute ends with a substring.

One other place that XPath seems to work better is when you are reversing up the DOM. For example, if you have the following:

    <div id='pair'>
        <span>
            <p>Label:</p>
            <input size='25' type='text' value=''/>
        </span>
    </div>


I would typically find the input using:

    "//p[contains(text(),'Label:')]/../input"

There is no easy way to do the same thing with CSS. With CSS I would find the input from the div tag. However, I usually find the p and input are paired together and the XPath always works but the input relative to the div changes occasionally.

Keeping data and code in one shell script file

In the past I have written scripts which required a lot of data. I could have the script read a second file with all the data in it but occasionally I'd lose the data file and still have the script. Or I'd have dozens of scripts and data files but no idea which data went with which script.

Solution: put the script and the data into one file. The script could then read the data from itself. Here is an example script:

#!/bin/sh

for entry in $(sed -n -e '/^exit/,$ p' $0 | grep -v exit | sed -e '/^$/d' | sed -e '/^#.*/d' | sed -e 's/ /_/g'); do
        entry=$(echo $entry | sed -e 's/_/ /g')
        firstname=$(echo $entry | awk -F, '{print $1}')
        lastname=$(echo $entry | awk -F, '{print $2}')
        number=$(echo $entry | awk -F, '{print $3}')
        echo "$firstname $lastname's phone number is $number"
done
exit

#FirstName,LastName,Phone
Darrell,Grainger,(416) 555-1212
John,Doe,(323) 555-1234
Jessica,Alba,(909) 555-9999

In this example, the data is a list of comma separated fields. Let's examine the list in the for statement. The $0 is the file currently executing, i.e. the file with this script and the data.

The sed command prints everything from the line which starts with exit to the end of file. The grep command gets rid of the line which starts with exit. The next sed command discards all blank lines. The third sed command discards all lines which start with #. This allows us to start a line with # if we want to add comments or comment out a line of data.

The final sed command on the for statement replaces all spaces with underscores. The reason for this is because if I have a line with a space, the for statement will process it as two separate records. I don't want that. I want to read one line as one record.

Inside the body of the for loop, the first line converts all the underscores back to spaces. If you want to have underscore in your data, this will not work. The solution is to pick a character which is not part of your data set. You can pick anything so long as the character you pick is the same in the for statement and the first line of the for loop body. The g in the sed statement is important in case there is more than one space.

The next three lines show how to break the line apart by commas. If you need to use commas in your data then pick a different character to separate the fields. The -F switch in the awk statement sets the field separator. So if you use exclamation mark as a field separator, you need to change the awk statement to -F!.

The echo statement is just an example of using the data.

Getting string from the clipboard in Java

Recently needed Java code to get the contents of the clipboard. For your reading pleasure:


public static String getStringFromClipboard() {
String s = null;
Clipboard c = Toolkit.getDefaultToolkit().getSystemClipboard();
Transferable contents = c.getContents(null);
if(contents != null && contents.isDataFlavorSupported(DataFlavor.stringFlavor)) {
try {
Object o = contents.getTransferData(DataFlavor.stringFlavor);
if (o instanceof String)
s = (String)o;
} catch(UnsupportedFlavorException ufe) {
ufe.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
return s;
}

This method assumes you are getting a string from the clipboard.

waitForElement

If you have Ajax calls or javascript which updates the page after the page has finished loading, you may have to wait for that code to finish before you can interact with the page. Essentially, waiting for page load is not sufficient.

The best way to deal with this is watch for something in the DOM which signals the script has finished executing. However, this is not always the easiest solution. If you are not the developer or familiar with javascript, it could take you days to figure out what event or change you need to wait for.

A better solution is to wait for whatever you want to interact with to be done. If you are waiting for a SELECT tag to be loaded with OPTION tags via Ajax there is no need to wait for the entire page to finish loading. You can just wait for the OPTION you wait to appear on the SELECT.

The best wait to do this is:

1. wait for a short period of time (250ms)
2. check for element
3. if element does not exist go to 1

There is one problem with this algorithm. If the element never appears, this becomes an infinite loop. So we add a time out:

1. maxTime = 5 seconds
2. timeSlice = 250 ms
3. elapsedTime = 0
4. wait for timeSlice
5. elapsedTime += timeSlice
6. check for element
7. if element does not exist AND elapsedTime < maxTime go to 4

Finally, this method must return true or false, where false means we did not find the element:

1. maxTime = 5 seconds
2. timeSlice = 250 ms
3. elapsedTime = 0
4. wait for timeSlice
5. elapsedTime += timeSlice
6. check for element
7. if element does not exist AND elapsedTime < maxTime go to 4
8. if elapsedTime < maxTime return true // found it
9. else return false // timed out

In Selenium and Java this might look like:

public WebElement waitForElement(By by) {
    WebElement result = null;
    long maxTime = 5 * 1000; // time in milliseconds
    long timeSlice = 250;
    long elapsedTime = 0;

    do {
        try{
            Thread.sleep(timeSlice);
            elapsedTime += timeSlice;
            result = driver.findElement(by);
        } catch(Exception e) {
        }
    } while(result == null && elapsedTime < maxTime);

    return result;
}

.