Difference between using current node (.) and text() function in XPath for Selenium locators

Recently someone asked about using the partial link text to find an anchor with an IMG tag in the middle of the text. The HTML snippet was:

<a href="something.html">
&nbsp;
<img src="filename.gif">
&nbsp;
partial link text
</a>

The initial attempt for a locator was:

"//a[contains(text(),'partial link text')]"

Normally I would expect this to work. However, the text() function does not seem to find it. Peter Jeffery Gale (thanks Peter) noticed that the following locator did work:

"//a[contains(.,'partial link text')]"

The . notation is the current node in the DOM. This is going to be an object of type Node. I'm guessing that the Node is getting cast to a string. Something similar to:

"//a[contains(string(.),'partial link text')]"

The end result seems to be that getting the entire Node, convert it to a string and scanning the string for a substring always works. Using the XPath function text() to get the text for an element only gets the text up to the first inner element. If the text you are looking for is after the inner element you must use the current node to search for the string and not the XPath text() function.

Selecting WebDriver locators

When you are using WebDriver for test automation a test case really boils down to:

1. Find an element
2. Perform an action
3. Confirm the expected result

Finding an element and confirming the expected result requires locators. In WebDriver a locator is a way of uniquely identifying an element on the web page, i.e. in the Document Object Model (DOM). The By class is used in WebDriver to locator elements. You have:

• By.className(className);
• By.id(id);
• By.linkText(linkText);
• By.name(name);
• By.partialLinkText(linkText);
• By.tagName(name);
• By.cssSelector(selector);
• By.xpath(xpathExpression);

The most powerful locators are CSS and XPath. All the other selectors can actually be done using CSS or XPath. For example:

Original	CSS	XPath
By.className("foo");	By.cssSelector(".foo");	By.xpath("//*[@class='foo']");
By.id("bar");	By.cssSelector("#bar");	By.xpath("//*[@id='bar']");
By.linkText("Click Me");	N/A	By.xpath("//a[text()='Click Me']");
By.name("fee");	By.cssSelector("[name='fee']");	By.xpath("//*[@name='fee']");
By.partialLinkText("some");	N/A	By.xpath("//a[contains(text(),'some')]");
By.tagname("div");	By.cssSelector("div");	By.xpath("//div");

In addition to the simple locators, CSS and XPath can selector more complex elements. Rather than saying I want all the DIV tags, I can say I want all the DIV tags whose parent is a SPAN. In CSS this would be "span>div" and in XPath this would be "//span/div".

The combinations are endless. For each tag in the XPath or CSS I can add multiple identifiers. So I could have locators for things as complex as "all DIV tags, with name containing 'foo' and class equal 'bar' whose parent is a TD but only in the TABLE with id='summary' and the class equal 'humho'"

The first thing to understand is that CSS will be noticeably faster than XPath when testing against Internet Explorer. Your tests could run as much as 10 times slower (something which runs on a day on Firefox could take a week on Internet Explorer) when using XPath.

So the first thing to remember is CSS is better than XPath. However, some things are easier to express as XPath. So occasionally you might need to use XPath.

If you have a selector like "html>body>table>tbody>tr[2]>td[3]>a" it might work but if the developer finds it does not format nicely on Chrome, they need to throw in a DIV. So the selector changes to "html>body>div>table>tbody>tr[2]>td[3]>a". Later a new version of Internet Explorer comes out and the developer finds they need to add a SPAN to make it look proper on the new Internet Explorer and still look okay on older versions. So the locator becomes "html>body>div>table>tbody>tr[2]>td[3]>span>a".

If we spend all our time maintaining the locators, it could end up that the cost of maintaining the automation is greater than running the tests manually. In which case the automation is deemed a failure.

So you have to start looking for patterns. Is there something I could use on the first version of the application which also works on the second and third version? Can I predict a locator which will work on the fourth and subsequent versions?

Often the underlying technology changes but it continues to look the same to the user. So is there something visual I can use which will not change? In this example, the text for the anchor probably never changed. So I'd use By.linkText("whatever"); locator or By.xpath("//a[text()='whatever']);.

What if I find myself changing locators because sometimes the text is "  whatever", sometimes it is "whatever" and other times it is "whatever  "? Then I'm going to use By.partialLinkText("whatever"); or By.xpath("//a[contains(text(), 'whatever')]");.

The danger is that there might be two links which contain the substring "whatever". I need to make sure I am selecting the correct link. So the locator might need to be more complex. It might need to be partial text and parent information. For example, if the text appears in two different tables and I want the text from table 2. Table 2 has the id='foo2' then the locator might be:

• "table.foo2 a"
• "//table[@id='foo2']/tbody/tr/td/a[contains(text(),'whatever')]"

The first locator assume there is only 1 anchor in the second table. This might not be true in all cases. The second locator finds the second table but it searches all rows (TR) and all columns (TD) for an anchor (A) whose text contains the substring "whatever". This can be extremely slow, especially for large tables.

Finding the balance between locators which are too long and too short can be an art. The trick is to pick something. If it requires maintenance, pick a new locator which works on the previous versions and the new version. As you continue to maintain the locators you will see a pattern. You will start to see that chunks of HTML code never change Outside these chunks change (so keep the locator short enough to stay inside the chunk that does not change). Within the chunk there might be multiple matches if you make the locator too short. So figure out, within that chunk, what makes the element you want different from all the other matches.

So how do I look at the DOM? I need to see what the DOM looks like to be able to see all the possible locators which would work.

If you are using Internet Explorer 8 or higher you can press F12 to open the developer tools. If you are using Firefox you need to install Firebug then F12 will open Firebug. If you are using Chrome then CTRL-SHIFT-I will open the developer tools.

Beyond that, the only tool I use is my brain and the W3 standards.

Reading the W3 standards (or any standards documentation, ISO, ANSI, IEEE, etc.) can be difficult at first. Especially if you have been learning from books like "Web Design in 21 Days" or "Software Testing for Dummies." However, the more you read and understand standards documentation, the easier it gets to read other standards documents. If generating XPath was easy enough for a piece of software then why would they pay you to do the work? There are probably a dozen XPath locators for any given element on a page. Some will work once and need to be revised on the next release of the application. Some will work within the design pattern of the application and might never need updating. There is no way for a piece of software to spot the design pattern and know which locator will work best. This is what they pay you to do.

Excessively long XPath is brittle and will need a great deal of revising from release to release. Extremely short XPath will sometimes find the wrong element between releases. This leads to a test which fails unpredictably and can be difficult to debug. Not something you want in an automation suite. Finding the right balance is your job. The first time you select a locator it might need revising for the next release. You need to look at why you selected the first locator when selecting the revised locator. The second locator should work for the first release and the second release. When the locator fails, you need to select a new locator which would have worked on the first release and all subsequent releases, including the next release. After a while you should start to see the pattern. The pattern is usually derived from some design pattern the application is being developed with. Learn about Design Patterns, it will be extremely helpful in generating good test automation. If the developers change the tools, libraries, design patterns, etc. you should expect the locators to fail. At this point, selecting a locator which works with the next release but does not work with the previous release makes sense. Major change in development usually implies major change in test automation. It would be difficult for a tool to realize when it needs to abandon old locators.

Essentially, automation is all about finding elements (locators), performing actions on them, confirming the expected results (usually involves more locators). Two thirds of the work is about the locators. Learning XPath, CSS and DOM will make your job that much easier.

• XPath 1.0: http://www.w3.org/TR/xpath/
• XPath 2.0: http://www.w3.org/TR/xpath20/
• XPath functions: http://www.w3.org/TR/xpath-functions/
• CSS 1.0: http://www.w3.org/TR/2008/REC-CSS1-20080411/
• CSS 2.0: http://www.w3.org/TR/CSS2/
• CSS 3.0: http://www.w3.org/TR/selectors/
• DOM: http://www.w3.org/TR/DOM-Level-2-HTML/html.html

When possible, use CSS selectors as they are faster. Some things are easier to locate using XPath and XQuery (XPath functions). It is better to have a test run slow and be easy to maintain. So if CSS selectors are complex and unintuitive you might want to use XPath functions instead.

This is essentially how I decide on locators.

.

maintenance versus performance

I was recently sent an example xpath for Selenium automation. The xpath, for a DIV, was:

"//*[@class="val1 val2 val3 val4"

My experience has been, especially with Internet Explorer, that xpath is very slow compared to say CSS. However, I find xpath easier to read (thus easier to maintain, especially in a team enviroinment) and certain things are just not possible in CSS.

So there is always a balance of maintenance versus performance. The golden rule for optimization is to never optimize right away. Right the code for maintainability and if you need to improve performance, then and only then optimize it.

The first thing I notice about the xpath above is the = operator. If the developer adds another value to the class attribute or the order of the values change, this will cause the automation to require maintenance. My experience has been that the values of multiple value attributes do change over time. So from a maintenance perspective I would write this as:

"//*[contains(@class,'val1') and contains(@class,'val2')
 and contains(@class,'val3') and contains(@class,'val4')]"

This way, if the order of the attributes change or the developer adds another value the xpath will still find the element.

The other things I'd change about the xpath is the use of //*. This will search every element for a match. Rather inefficient. It does have the nice feature that if a developer changes the element from a DIV to something else, the xpath will require no maintenance. However, and this is just from years of experience, I find it rare that a developer changes a DIV to something else. If they do, there is a huge amount of refactoring and using //* will not save you from any maintenance.

Since the likelihood of maintenance is low but it is such an obvious performance hit, I would actually use //div for the xpath.

Understanding how xpath relates to web pages

When using automation tools like Selenium or Watij you often find yourself creating an xpath to find an element. Talking to a few people there seems to be a lack of understanding of how an xpath relates to a web page.

I think the step which is missing for most people is understanding how to look at a web page.

A web page is merely a group of blocks inside blocks. To illustrate I have the following image:

Image the outer block is the <BODY> of the web page. Inside the outer block, i.e. the BODY are two rectangles. Let's say they are <TABLE> elements. The top table, i.e. /HTML/BODY/TABLE[1], has one row and three columns. The lower table, i.e. /HTML/BODY/TABLE[2], has three rows and four columns.

Let's say that both tables have one row of cells where the class was 'foo', i.e. <TD class='foo'>. If I wanted to find all cells with class='foo' and the text contained 'bar' I would use:

    //TD[@class='foo' and contains(text(), 'bar')]

But what if I wanted to search only the second table? Then I would use:

    //TABLE[2]/TBODY/TR/TD[@class='foo' and contains(text(), 'bar')]

Essentially, the longer the xpath the small the area of the web page I am searching. Using //BODY will search the largest square in my example. Using //BODY/TABLE[2] will search the lower table or the second level in.

If you look at the third row of the lower table you can see the 'cells' contain another level of element. Let's say that the cells, i.e. <TD>, contains a <DIV>. Using //TABLE[2]/TR[3]/TD/DIV[1] focuses on the first div in the last row of the lower table.

How to pick identifiers

When doing software automation pretty much all automation tools require some way of finding the elements in the application. If it is a desktop application, the tool needs a way to find the menus, buttons, text fields, etc. If it is a web application, the tool needs a way to find the various HTML elements, e.g. input, select, ul, li, table.

The trick to easily maintainable automation is to pick something unique and unlikely to change. Take for example a web page with a table and I want to find the contents of a cell. Tools like Selenium using 'locators'. A locator can be: id, name, dom, xpath, css. There are others but these five are the ones common to most web automation tools. I like to use these because if I switch to a different tool (new project, current tool no longer supported, different company, etc.) my knowledge is more transferable. If I use a recognition method unique to a tool I am tying myself to that tool.

If you read the HTML standard, you will find that the ID of an element must be unique. Being unique is paramount to automation. If two elements on a page can have the same value then there is the chance at some point your automation will fail. Some automation languages will fail with a helpful error, i.e. it will tell you there was more than one element which matched. Many will just select the first match, or the last match, or fail farther down in the test case. The problem with ID is not all elements have an ID attribute. There is nothing in the standard indicating all elements have to have an ID. But if an ID exists, this is the best choice.

The NAME attribute does not have to be unique and therefore could cause problems in the future. I will occasionally use the NAME attribute but only if I am working closely with the development team and have a feel for how they select attributes. If the attribute is selected by some automated tool (like struts) and I know this tool is guaranteed to make all the names unique, I'll use it but only if there is no better choice.

The DOM is going to be something a web developer will understand. However, the structure of the DOM might change. If the UI is fairly simple or the application is very stable then using the DOM might be okay. But of they are adding menus, moving around navigation bars, changing tables to div/span combinations, adding divs or spans to deal with new web browsers, etc. then the DOM will be changing and you will be required to update your automation. So what works today might given you a lot of headaches in the future. The whole idea behind automation is to spend more time creating the automation and reap the benefit of running it again and again. If you spend 4 times more time automating compared to manual testing and have to spend the same amount of time maintaining the automation as you would manual testing, you'll never re-coop the initial cost of automation. The more the DOM is changing the less beneficial it will be to use the DOM.

The XPATH is my preferred method of identifying elements on a web page. How you use it can make a world of difference however. If the full xpath to the cell in a table was /html/body/div[2]/span[4]/table[3]/tbody/tr[7]/td[9] I could use that. It would definitely be unique. But if the document is in flux, this xpath will change. Alternative xpaths would be //body/div[2]/span[4]/table[3]/tbody/tr[7]/td[9] or even //table[3]/tbody/tr[7]/td[9]. The shorter I can get the xpath the less likely it will change. Even with the last xpath will break if they add another table above the current one. What if they add or subtract a column? Then the td becomes td[8] or td[10].

Basically, using magic numbers is a bad thing. This is taught in first year computer programming. If we look closer at xpath we find there are functions we can use. If the cell has ID=foo I can use //TD[@id='foo']. What if today it is a cell in a table but in the future they change it to a set of DIV/SPAN. I could use //*[@id='foo'].

What if the automated tool occasionally adds a space to the attribute strings (I see this a lot). I'd have to figure this out and change it to //*[@id='foo ']. But there is a better solution, try using: //*[contains(@id,'foo')]. The danger of this is that there might be an ID='foo' and a second element with ID='foobar'. At this point you need to use your judgement. Are there attribute values who are subsets of other attribute values? If yes, don't use contains(). Is it common for spaces to get added to attribute values? If yes, do use contains(). What if both conditions occur? This is where it gets hard. There is no one right choice.

Sometimes you want to add some more in. For example, if the table has ID='bar' and the structure of the table is fairly solid, I might be okay to use //table[@id='bar']/tbody/tr[7]/td[9].

Another solution is if things are relative to one another. For example, if I have a table and the third column has an element which is unique (the text, the id, whatever) and I was the seventh column on that row, I can use a relative path. For example, say I want the input field on the same row with the text 'Enter the Quantity:' then I could use:

    //td[contains(text(),'Enter the Quantity:')]/../td[7]/input

This starts from the cell with the known text, goes up one to be on the row then down into the 7th column and finally down into the input element.

The CSS is very much the same as XPATH. The only difference is that understanding how you can match things with CSS requires knowledge of CSS. If you are a developer who uses CSS to identify and target elements for say AJAX then obviously you are going to be comfortable using CSS to identify the elements. Just like with XPATH you can do things like TD#foo (e.g. //TD[@id='foo']), #foo (e.g. //*[@id='foo']), *[id*='foo'] (e.g. //*[contains(@id,'foo')]). NOTE: I'm not an CSS expert so take these examples with a grain of salt.

Summary, pick identifiers which are (1) unique, (2) unlikely to change and (3) you understand and can maintain.

xpath

I have been doing a lot of web testing. The general idea behind all UI test automation tools is to locate an element on a page then do something with it, set it, clear it, read it, etc.

For the web automation tools you can use:

(a) the position on the screen (x,y coordinations)
(b) the position in the DOM (e.g. /html/body/table/tbody/tr[2]/td[4]
(c) a unique attribute

The position on the screen never works. Different browsers, fonts, screen resized, etc. will change the layout and ultimately, change the screen position of elements. I wouldn't use this. Working with development to provide alternative means will be less work than maintaining automation with x,y coordinates.

The position within the DOM is a little brittle. When a browser is patched or a new browser needs to be supported it is not uncommon for the developers to throw in some span or div elements to help with layout. So the element /html/body/table might change to /html/body/div/span/table. The more precise the positioning information the more brittle it will be.

A unique attribute would be something like the id of a tag. For example:

Darrell

I can find this via the class or id attribute, or both. This is where xpath comes in handy. The automation tools I have been using (Watij, Selenium) can use xpath to locate an element. For my td element I can use:

//TD[@class='username-cell']

or

//TD[@id='username']

or

//TD[@class='username-cell' and @id='username']

The id attribute is required to be unique. So if the element has an id, that is the attribute to use. If you start an xpath with // it tells the tool to start searching anywhere in the DOM. Starting with a single / will start at the root. For a web page that will always be /html.

Xpath can be quite powerful in identifying elements. You have a few 'tricks' you want to use. First is that id=' foo' is not the same as id='foo '. The whitespace makes a difference. To get around this I would use:

//TD[contains(@id,'foo')]

Now the whitespace does not matter. you have to be careful with this however. If there are two matches, it depends on the automation tool as to what will happen. If I have:

darrellDarrell

then:

//TD[contains(@id,'user')]

will have unpredictable results. Not something you want in test automation. So how to get around this?

//TD[contains(@id,'user') and not(contains(@id,'username'))]

Any attribute inside the tag can be used via the @ symbol. You can do things like look at the style attribute using @style. Because the order of the things in style does not matter to the browser, the contains() function helps a lot.

Finally, the text between the open and close tag can be found using the text() function. So if I had:

Google

I can find it using:

//A[contains(text(),'Google')]

What about the difference between Google and google? For matching either you can use:

//A[contains(lower-case(text()),'google')]

This will take the text in the anchor (e.g. Google) and changing it to lower case (e.g. google) then comparing that resulting string to 'google'.

In addition to and there is an or keyword as well but I usually find it better to narrow things down (and will filter out) rather than build up the matches (or will combine).

There is a lot more to know about xpath. If you are curious, ask.

Additionally, I find a developer will put an id on something because he uses it from javascript to find it and all the elements underneath it. So if there is:

Darrell

Jerome

Mark

I would tend to use:

//DIV[contains(@id,'users')]/UL/LI[contains(text(),'Darrell')]

to find the element which contains 'Darrell'. The developer will tend to not want to change the structure under the id'd tag because it will cause them a lot of maintenance as well.