I always liked using XPath for locators in Selenium and WebDriver. It was more procedural than using Cascading Style Sheets (CSS) and felt more like programming. However, using CSS for locators is much faster than using XPath. I haven't done any tests to see just how much faster but it is significant enough that I can see the difference.
I run a test suite using XPath locators. I can see significant delays where there appears to be nothing happening on the screen. I refactor the code to use CSS and the delays disappear. I would estimate that 3 to 5 second delays (XPath) become too small to measure with the naked eye (CSS).
The only feature I have been unable to do in CSS are locators which use the XPath text() function. CSS seems to be all about the tag and its attributes but not the text contained within the tag.
The one thing it took me a little while to get used to when switching from XPath to CSS was compound statements. For example, I might have an XPath like:
"//div[contains(@class,'prefspanel')]/table[@id='sizer1']"
So how do I convert this to CSS?
"div[class~='prefspanel']>table#sizer1"
Shorter and pretty much the same attributes. The ~= isn't quite the same as the contains() function. If I have:
class='prefspanel foo bar'
the ~= will find prefspanel because it is a 'word' separated by whitespace. The contains() function will find 'prefs', 'prefspan', 'prefspanel' and even 'panel' because these are all substrings of the original. In CSS3 they introduced *= which is just like contains().
I also notice that many attributes will change dynamically. However, the start of the attribute or the end of the attribute will be consistent. In XPath I have to use the contains() function to find the attributes which end with the unique identifier. With CSS I can use ^= to see if the attribute begins with a substring or $= to see if the attribute ends with a substring.
One other place that XPath seems to work better is when you are reversing up the DOM. For example, if you have the following:
<div id='pair'>
<span>
<p>Label:</p>
<input size='25' type='text' value=''/>
</span>
</div>
I would typically find the input using:
"//p[contains(text(),'Label:')]/../input"
There is no easy way to do the same thing with CSS. With CSS I would find the input from the div tag. However, I usually find the p and input are paired together and the XPath always works but the input relative to the div changes occasionally.
Tuesday, January 31, 2012
Tuesday, January 17, 2012
Keeping data and code in one shell script file
In the past I have written scripts which required a lot of data. I could have the script read a second file with all the data in it but occasionally I'd lose the data file and still have the script. Or I'd have dozens of scripts and data files but no idea which data went with which script.
Solution: put the script and the data into one file. The script could then read the data from itself. Here is an example script:
In this example, the data is a list of comma separated fields. Let's examine the list in the for statement. The $0 is the file currently executing, i.e. the file with this script and the data.
The sed command prints everything from the line which starts with exit to the end of file. The grep command gets rid of the line which starts with exit. The next sed command discards all blank lines. The third sed command discards all lines which start with #. This allows us to start a line with # if we want to add comments or comment out a line of data.
The final sed command on the for statement replaces all spaces with underscores. The reason for this is because if I have a line with a space, the for statement will process it as two separate records. I don't want that. I want to read one line as one record.
Inside the body of the for loop, the first line converts all the underscores back to spaces. If you want to have underscore in your data, this will not work. The solution is to pick a character which is not part of your data set. You can pick anything so long as the character you pick is the same in the for statement and the first line of the for loop body. The g in the sed statement is important in case there is more than one space.
The next three lines show how to break the line apart by commas. If you need to use commas in your data then pick a different character to separate the fields. The -F switch in the awk statement sets the field separator. So if you use exclamation mark as a field separator, you need to change the awk statement to -F!.
The echo statement is just an example of using the data.
Solution: put the script and the data into one file. The script could then read the data from itself. Here is an example script:
#!/bin/sh
for entry in $(sed -n -e '/^exit/,$ p' $0 | grep -v exit | sed -e '/^$/d' | sed -e '/^#.*/d' | sed -e 's/ /_/g'); do
entry=$(echo $entry | sed -e 's/_/ /g')
firstname=$(echo $entry | awk -F, '{print $1}')
lastname=$(echo $entry | awk -F, '{print $2}')
number=$(echo $entry | awk -F, '{print $3}')
echo "$firstname $lastname's phone number is $number"
done
exit
#FirstName,LastName,Phone
Darrell,Grainger,(416) 555-1212
John,Doe,(323) 555-1234
Jessica,Alba,(909) 555-9999
In this example, the data is a list of comma separated fields. Let's examine the list in the for statement. The $0 is the file currently executing, i.e. the file with this script and the data.
The sed command prints everything from the line which starts with exit to the end of file. The grep command gets rid of the line which starts with exit. The next sed command discards all blank lines. The third sed command discards all lines which start with #. This allows us to start a line with # if we want to add comments or comment out a line of data.
The final sed command on the for statement replaces all spaces with underscores. The reason for this is because if I have a line with a space, the for statement will process it as two separate records. I don't want that. I want to read one line as one record.
Inside the body of the for loop, the first line converts all the underscores back to spaces. If you want to have underscore in your data, this will not work. The solution is to pick a character which is not part of your data set. You can pick anything so long as the character you pick is the same in the for statement and the first line of the for loop body. The g in the sed statement is important in case there is more than one space.
The next three lines show how to break the line apart by commas. If you need to use commas in your data then pick a different character to separate the fields. The -F switch in the awk statement sets the field separator. So if you use exclamation mark as a field separator, you need to change the awk statement to -F!.
The echo statement is just an example of using the data.
Getting string from the clipboard in Java
Recently needed Java code to get the contents of the clipboard. For your reading pleasure:
public static String getStringFromClipboard() {
String s = null;
Clipboard c = Toolkit.getDefaultToolkit().getSystemClipboard();
Transferable contents = c.getContents(null);
if(contents != null && contents.isDataFlavorSupported(DataFlavor.stringFlavor)) {
try {
Object o = contents.getTransferData(DataFlavor.stringFlavor);
if (o instanceof String)
s = (String)o;
} catch(UnsupportedFlavorException ufe) {
ufe.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
return s;
}
public static String getStringFromClipboard() {
String s = null;
Clipboard c = Toolkit.getDefaultToolkit().getSystemClipboard();
Transferable contents = c.getContents(null);
if(contents != null && contents.isDataFlavorSupported(DataFlavor.stringFlavor)) {
try {
Object o = contents.getTransferData(DataFlavor.stringFlavor);
if (o instanceof String)
s = (String)o;
} catch(UnsupportedFlavorException ufe) {
ufe.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
return s;
}
This method assumes you are getting a string from the clipboard.
Tuesday, January 10, 2012
waitForElement
If you have Ajax calls or javascript which updates the page after the page has finished loading, you may have to wait for that code to finish before you can interact with the page. Essentially, waiting for page load is not sufficient.
The best way to deal with this is watch for something in the DOM which signals the script has finished executing. However, this is not always the easiest solution. If you are not the developer or familiar with javascript, it could take you days to figure out what event or change you need to wait for.
A better solution is to wait for whatever you want to interact with to be done. If you are waiting for a SELECT tag to be loaded with OPTION tags via Ajax there is no need to wait for the entire page to finish loading. You can just wait for the OPTION you wait to appear on the SELECT.
The best wait to do this is:
.
The best way to deal with this is watch for something in the DOM which signals the script has finished executing. However, this is not always the easiest solution. If you are not the developer or familiar with javascript, it could take you days to figure out what event or change you need to wait for.
A better solution is to wait for whatever you want to interact with to be done. If you are waiting for a SELECT tag to be loaded with OPTION tags via Ajax there is no need to wait for the entire page to finish loading. You can just wait for the OPTION you wait to appear on the SELECT.
The best wait to do this is:
- wait for a short period of time (250ms)
- check for element
- if element does not exist go to 1
There is one problem with this algorithm. If the element never appears, this becomes an infinite loop. So we add a time out:
- maxTime = 5 seconds
- timeSlice = 250 ms
- elapsedTime = 0
- wait for timeSlice
- elapsedTime += timeSlice
- check for element
- if element does not exist AND elapsedTime < maxTime go to 4
Finally, this method must return true or false, where false means we did not find the element:
- maxTime = 5 seconds
- timeSlice = 250 ms
- elapsedTime = 0
- wait for timeSlice
- elapsedTime += timeSlice
- check for element
- if element does not exist AND elapsedTime < maxTime go to 4
- if elapsedTime < maxTime return true // found it
- else return false // timed out
In Selenium and Java this might look like:
public WebElement waitForElement(By by) {
WebElement result = null;
long maxTime = 5 * 1000; // time in milliseconds
long timeSlice = 250;
long elapsedTime = 0;
do {
try{
Thread.sleep(timeSlice);
elapsedTime += timeSlice;
result = driver.findElement(by);
} catch(Exception e) {
}
} while(result == null && elapsedTime < maxTime);
return result;
}
.
Wednesday, December 28, 2011
The 9 Oddest Job Interview Questions Asked at Tech Companies in 2011
I recent read an article on Mashable.com about the 9 oddest job interview questions asked at tech companies in 2011. Here they are:
- How many people are using Facebook in San Francisco at 2:30 p.m. on a Friday?
- If Germans were the tallest people in the world, how would you prove it?
- Given 20 ‘destructible’ light bulbs (which break at a certain height), and a building with 100 floors, how do you determine the height that the light bulbs break?
- How would you cure world hunger?
- You’re in a row boat, which is in a large tank filled with water. You have an anchor on board, which you throw overboard (the chain is long enough so the anchor rests completely on the bottom of the tank). Does the water level in the tank rise or fall?
- Please spell ‘diverticulitis’.
- You have a bouquet of flowers. All but two are roses, all but two are daisies, and all but two are tulips. How many flowers do you have?
- How do you feel about those jokers at Congress?
- If you were a Microsoft Office program, which one would you be?
I wondered how would I have answered these.
The first one was immediately obvious to me. The answer is "all of them." Okay a little facetious but that is how I'd answer it at first. Sort of an ice-breaker. If they had a problem with that answer and moved on I'd have a bit of a problem working there. If that got a chuckle I'd see if they were looking for something else. I think I'd ask for further clarification. What is the motivation for this question? What frame of mind should someone be in to properly answer this question? Is there a specific answer you are looking for?
Number 2 seems to be a little vague for me. As a software tester, ambiguity doesn't work for me. What does "Germans were the tallest people in the world" mean? Is the combined height of all Germans greater than the combined height of all other people nations? By Germans do you mean people born in German? People who hold German citizenship? What about immigrants to Germany? What about people who hold dual citizenship? What about former Germans who immigrated to another country and no longer consider themselves German? In short, the question needs further clarification to be answered.
Number 3 is also a little vague. How accurate does the answer for each bulb have to be? Can we open a window on each floor in the building? Do we know the exact height of each window? Are we to assume all the bulbs break at the same height and there is one answer for all 20 bulbs? If this is the case and we really want to know from which floor the bulbs will break, drop one from the first floor. If it does not break, go to the second floor and repeat. When if no bulb has broken and you make it to the 20th floor, go down and collect all 20 unbroken bulbs, start again from the 21st floor. Once you find the floor the bulb breaks on, you will have used only 1 bulb. If I make it to the 100th floor and no bulb has broken. I'd have to devise some way to go above 100 floors and continue the test.
On the other hand, if the bulbs might break at different heights, I'd have to drop all of them from the first floor. then all surviving bulbs from the second floor and so on.
For number 4, if I had an answer to this I won't be sitting in the job interview at a tech company. I would be implementing my answer.
Number 5 seems like a question about displacement. A real physics question. If I am in a boat with an anchor, the boat myself and the anchor have weight. The amount of water displaced by the boat is less than the weight of myself, the boat and the anchor. If I throw the anchor overboard, the boat will rise in the water (it weighs less and displaces just as much water, therefore it will be more buoyant). However as the boat rises it will displace less water. Thus the water level will fall. The anchor dropped in the water has volume and will displace some water. This the water level will rise. The unanswered question is whether the displacement of the anchor is greater than, less than or equal to the reduced displacement of the boat. I believe the answer is equal to. So the water level in the tank will neither rise or fall.
Kind of hard to mess up number 6. This is a question which only works verbally. Since I can see the spelling it is pointless. Not sure what they are trying to test here. They like people who are good at spelling bees?
Number 7 would have to be 3 flowers. If I have 1 rose then all but two are roses (3 - 2 = 1 rose), 1 daisy then all but two are daisies (3 - 2 = 1 daisy) and 1 tulip then all but two are tulips (3 - 2 = 1 tulip).
Number 8 seems to assume I care about American politics and have an option on Congress. The question is a leading question. Personally, I'd answer it with, "I don't believe in mixing politics, religion and work. Since this is a job interview no politics or religion please."
And for number 9, I'd have to take a moment and think about it. I'd need to know myself then I'd have to relay how a Microsoft Office program could be analogous to the traits I like about myself. Outlook is good for communication, Word is probably the most popular program and good at the most jobs. Excel is great for finance, budgetting, invoicing. Powerpoint is good to convey ideas and used in presentations which inform and teach. Do we include Messenger? I think the key to this one is knowing yourself. If you can describe any Microsoft Office application as exhibiting the same traits you're probably giving a good answer. On the other hand you might want to say you could never limit yourself to one Microsoft Office program. Like the full Office solution, you do it all.
Personally, I try to avoid interview questions like these. Often the interview just thinks the answer is creative or smart and if you can get it you must be creative or smart. It does not take into account cultural differences, training background, already heard the question, etc. In other cases the interview thinks by making the question cryptic, it will be harder for the interviewee to know what the interviewer is looking for and they'll get an honest answer. Realistically, that doesn't work. If you look at research surveys, they will ask a 100 questions. Of those, 20 questions are probably related. The candidate might think they know what I'm look for on 2 or 3 of those question but the majority of the questions will give me what the honest answer to the questions are.
In the end, these are all games and statistically, employers should find a good candidate. They might not find the best candidate but they'll never know because the person they hire will be okay and possibly even great.
.
Selecting WebDriver locators
When you are using WebDriver for test automation a test case really boils down to:
In addition to the simple locators, CSS and XPath can selector more complex elements. Rather than saying I want all the DIV tags, I can say I want all the DIV tags whose parent is a SPAN. In CSS this would be "span>div" and in XPath this would be "//span/div".
The combinations are endless. For each tag in the XPath or CSS I can add multiple identifiers. So I could have locators for things as complex as "all DIV tags, with name containing 'foo' and class equal 'bar' whose parent is a TD but only in the TABLE with id='summary' and the class equal 'humho'"
The first thing to understand is that CSS will be noticeably faster than XPath when testing against Internet Explorer. Your tests could run as much as 10 times slower (something which runs on a day on Firefox could take a week on Internet Explorer) when using XPath.
So the first thing to remember is CSS is better than XPath. However, some things are easier to express as XPath. So occasionally you might need to use XPath.
If you have a selector like "html>body>table>tbody>tr[2]>td[3]>a" it might work but if the developer finds it does not format nicely on Chrome, they need to throw in a DIV. So the selector changes to "html>body>div>table>tbody>tr[2]>td[3]>a". Later a new version of Internet Explorer comes out and the developer finds they need to add a SPAN to make it look proper on the new Internet Explorer and still look okay on older versions. So the locator becomes "html>body>div>table>tbody>tr[2]>td[3]>span>a".
If we spend all our time maintaining the locators, it could end up that the cost of maintaining the automation is greater than running the tests manually. In which case the automation is deemed a failure.
So you have to start looking for patterns. Is there something I could use on the first version of the application which also works on the second and third version? Can I predict a locator which will work on the fourth and subsequent versions?
Often the underlying technology changes but it continues to look the same to the user. So is there something visual I can use which will not change? In this example, the text for the anchor probably never changed. So I'd use By.linkText("whatever"); locator or By.xpath("//a[text()='whatever']);.
What if I find myself changing locators because sometimes the text is " whatever", sometimes it is "whatever" and other times it is "whatever "? Then I'm going to use By.partialLinkText("whatever"); or By.xpath("//a[contains(text(), 'whatever')]");.
The danger is that there might be two links which contain the substring "whatever". I need to make sure I am selecting the correct link. So the locator might need to be more complex. It might need to be partial text and parent information. For example, if the text appears in two different tables and I want the text from table 2. Table 2 has the id='foo2' then the locator might be:
If you are using Internet Explorer 8 or higher you can press F12 to open the developer tools. If you are using Firefox you need to install Firebug then F12 will open Firebug. If you are using Chrome then CTRL-SHIFT-I will open the developer tools.
Beyond that, the only tool I use is my brain and the W3 standards.
Reading the W3 standards (or any standards documentation, ISO, ANSI, IEEE, etc.) can be difficult at first. Especially if you have been learning from books like "Web Design in 21 Days" or "Software Testing for Dummies." However, the more you read and understand standards documentation, the easier it gets to read other standards documents. If generating XPath was easy enough for a piece of software then why would they pay you to do the work? There are probably a dozen XPath locators for any given element on a page. Some will work once and need to be revised on the next release of the application. Some will work within the design pattern of the application and might never need updating. There is no way for a piece of software to spot the design pattern and know which locator will work best. This is what they pay you to do.
Excessively long XPath is brittle and will need a great deal of revising from release to release. Extremely short XPath will sometimes find the wrong element between releases. This leads to a test which fails unpredictably and can be difficult to debug. Not something you want in an automation suite. Finding the right balance is your job. The first time you select a locator it might need revising for the next release. You need to look at why you selected the first locator when selecting the revised locator. The second locator should work for the first release and the second release. When the locator fails, you need to select a new locator which would have worked on the first release and all subsequent releases, including the next release. After a while you should start to see the pattern. The pattern is usually derived from some design pattern the application is being developed with. Learn about Design Patterns, it will be extremely helpful in generating good test automation. If the developers change the tools, libraries, design patterns, etc. you should expect the locators to fail. At this point, selecting a locator which works with the next release but does not work with the previous release makes sense. Major change in development usually implies major change in test automation. It would be difficult for a tool to realize when it needs to abandon old locators.
Essentially, automation is all about finding elements (locators), performing actions on them, confirming the expected results (usually involves more locators). Two thirds of the work is about the locators. Learning XPath, CSS and DOM will make your job that much easier.
When possible, use CSS selectors as they are faster. Some things are easier to locate using XPath and XQuery (XPath functions). It is better to have a test run slow and be easy to maintain. So if CSS selectors are complex and unintuitive you might want to use XPath functions instead.
This is essentially how I decide on locators.
.
- Find an element
- Perform an action
- Confirm the expected result
Finding an element and confirming the expected result requires locators. In WebDriver a locator is a way of uniquely identifying an element on the web page, i.e. in the Document Object Model (DOM). The By class is used in WebDriver to locator elements. You have:
- By.className(className);
- By.id(id);
- By.linkText(linkText);
- By.name(name);
- By.partialLinkText(linkText);
- By.tagName(name);
- By.cssSelector(selector);
- By.xpath(xpathExpression);
The most powerful locators are CSS and XPath. All the other selectors can actually be done using CSS or XPath. For example:
| Original | CSS | XPath |
|---|---|---|
| By.className("foo"); | By.cssSelector(".foo"); | By.xpath("//*[@class='foo']"); |
| By.id("bar"); | By.cssSelector("#bar"); | By.xpath("//*[@id='bar']"); |
| By.linkText("Click Me"); | N/A | By.xpath("//a[text()='Click Me']"); |
| By.name("fee"); | By.cssSelector("[name='fee']"); | By.xpath("//*[@name='fee']"); |
| By.partialLinkText("some"); | N/A | By.xpath("//a[contains(text(),'some')]"); |
| By.tagname("div"); | By.cssSelector("div"); | By.xpath("//div"); |
In addition to the simple locators, CSS and XPath can selector more complex elements. Rather than saying I want all the DIV tags, I can say I want all the DIV tags whose parent is a SPAN. In CSS this would be "span>div" and in XPath this would be "//span/div".
The combinations are endless. For each tag in the XPath or CSS I can add multiple identifiers. So I could have locators for things as complex as "all DIV tags, with name containing 'foo' and class equal 'bar' whose parent is a TD but only in the TABLE with id='summary' and the class equal 'humho'"
The first thing to understand is that CSS will be noticeably faster than XPath when testing against Internet Explorer. Your tests could run as much as 10 times slower (something which runs on a day on Firefox could take a week on Internet Explorer) when using XPath.
So the first thing to remember is CSS is better than XPath. However, some things are easier to express as XPath. So occasionally you might need to use XPath.
If you have a selector like "html>body>table>tbody>tr[2]>td[3]>a" it might work but if the developer finds it does not format nicely on Chrome, they need to throw in a DIV. So the selector changes to "html>body>div>table>tbody>tr[2]>td[3]>a". Later a new version of Internet Explorer comes out and the developer finds they need to add a SPAN to make it look proper on the new Internet Explorer and still look okay on older versions. So the locator becomes "html>body>div>table>tbody>tr[2]>td[3]>span>a".
If we spend all our time maintaining the locators, it could end up that the cost of maintaining the automation is greater than running the tests manually. In which case the automation is deemed a failure.
So you have to start looking for patterns. Is there something I could use on the first version of the application which also works on the second and third version? Can I predict a locator which will work on the fourth and subsequent versions?
Often the underlying technology changes but it continues to look the same to the user. So is there something visual I can use which will not change? In this example, the text for the anchor probably never changed. So I'd use By.linkText("whatever"); locator or By.xpath("//a[text()='whatever']);.
What if I find myself changing locators because sometimes the text is " whatever", sometimes it is "whatever" and other times it is "whatever "? Then I'm going to use By.partialLinkText("whatever"); or By.xpath("//a[contains(text(), 'whatever')]");.
The danger is that there might be two links which contain the substring "whatever". I need to make sure I am selecting the correct link. So the locator might need to be more complex. It might need to be partial text and parent information. For example, if the text appears in two different tables and I want the text from table 2. Table 2 has the id='foo2' then the locator might be:
- "table.foo2 a"
- "//table[@id='foo2']/tbody/tr/td/a[contains(text(),'whatever')]"
The first locator assume there is only 1 anchor in the second table. This might not be true in all cases. The second locator finds the second table but it searches all rows (TR) and all columns (TD) for an anchor (A) whose text contains the substring "whatever". This can be extremely slow, especially for large tables.
Finding the balance between locators which are too long and too short can be an art. The trick is to pick something. If it requires maintenance, pick a new locator which works on the previous versions and the new version. As you continue to maintain the locators you will see a pattern. You will start to see that chunks of HTML code never change Outside these chunks change (so keep the locator short enough to stay inside the chunk that does not change). Within the chunk there might be multiple matches if you make the locator too short. So figure out, within that chunk, what makes the element you want different from all the other matches.
So how do I look at the DOM? I need to see what the DOM looks like to be able to see all the possible locators which would work.
If you are using Internet Explorer 8 or higher you can press F12 to open the developer tools. If you are using Firefox you need to install Firebug then F12 will open Firebug. If you are using Chrome then CTRL-SHIFT-I will open the developer tools.
Beyond that, the only tool I use is my brain and the W3 standards.
Reading the W3 standards (or any standards documentation, ISO, ANSI, IEEE, etc.) can be difficult at first. Especially if you have been learning from books like "Web Design in 21 Days" or "Software Testing for Dummies." However, the more you read and understand standards documentation, the easier it gets to read other standards documents. If generating XPath was easy enough for a piece of software then why would they pay you to do the work? There are probably a dozen XPath locators for any given element on a page. Some will work once and need to be revised on the next release of the application. Some will work within the design pattern of the application and might never need updating. There is no way for a piece of software to spot the design pattern and know which locator will work best. This is what they pay you to do.
Excessively long XPath is brittle and will need a great deal of revising from release to release. Extremely short XPath will sometimes find the wrong element between releases. This leads to a test which fails unpredictably and can be difficult to debug. Not something you want in an automation suite. Finding the right balance is your job. The first time you select a locator it might need revising for the next release. You need to look at why you selected the first locator when selecting the revised locator. The second locator should work for the first release and the second release. When the locator fails, you need to select a new locator which would have worked on the first release and all subsequent releases, including the next release. After a while you should start to see the pattern. The pattern is usually derived from some design pattern the application is being developed with. Learn about Design Patterns, it will be extremely helpful in generating good test automation. If the developers change the tools, libraries, design patterns, etc. you should expect the locators to fail. At this point, selecting a locator which works with the next release but does not work with the previous release makes sense. Major change in development usually implies major change in test automation. It would be difficult for a tool to realize when it needs to abandon old locators.
Essentially, automation is all about finding elements (locators), performing actions on them, confirming the expected results (usually involves more locators). Two thirds of the work is about the locators. Learning XPath, CSS and DOM will make your job that much easier.
- XPath 1.0: http://www.w3.org/TR/xpath/
- XPath 2.0: http://www.w3.org/TR/xpath20/
- XPath functions: http://www.w3.org/TR/xpath-functions/
- CSS 1.0: http://www.w3.org/TR/2008/REC-CSS1-20080411/
- CSS 2.0: http://www.w3.org/TR/CSS2/
- CSS 3.0: http://www.w3.org/TR/selectors/
- DOM: http://www.w3.org/TR/DOM-Level-2-HTML/html.html
When possible, use CSS selectors as they are faster. Some things are easier to locate using XPath and XQuery (XPath functions). It is better to have a test run slow and be easy to maintain. So if CSS selectors are complex and unintuitive you might want to use XPath functions instead.
This is essentially how I decide on locators.
.
Friday, December 23, 2011
Your automation must not dictate your test plan
One of the things I see people getting into automation doing is selecting what to automate or how to test an application based on what the automation tool will let them do. This is a dangerous approach to automation.
The test cases I create for an application are based on what is important to the customer. I want to make sure that the customer experience is a good experience. If I create a set of test cases or use stories which reflect real customer usage of the application then I am most likely to find issues which will affect the customer.
I remember working on a project for 4 years. After 4 years of testing and improving the application we were at a point that over 90% of the features important to the customer were tested and bug free. Of the remaining 10% we knew of most the defects and had a work-around. We were at a point where we were testing extreme edge cases. At this point I found a major defect. The developer looked at the fix and realized the defect had been there since the beginning. In 4 years not one single customer reported this defect. The defect was easy to automate but really added zero value to the application. This is NOT a test case you want to start with when automating.
On another project someone found a defect in a desktop application. The steps to reproduce were:
At this point the modal dialog is hidden behind the window with focus and the window with focus appears to be frozen. It is really waiting for you to respond to the modal dialog. This was a design flaw in the operating system. It was virtually impossible to fix in the application under test without a major re-design. It was highly unlikely a customer would be able to reproduce this defect. When the tester showed me the 'locked' state it only took me a few minutes to figure out what was going on. Our customer was typically a software developer with 7+ years of experience.
This was a useless test case. In both this and the previous test case it was a bad test case regardless of creating it manually or automating it. My point is, the test case came first. Even before we attempted to automate it, we decided whether or not it was a good test case.
Test automation NEVER proceeds test case creation or test planning.
Once you know what you want to test and the steps to testing it, you automate those steps.
This is the second mistake I see people getting into automation making. They decide WHAT they want to test but when they start automating it, the steps they generate with the automation are not the same steps as they would do manually. In this case you have taken the time to create a good set of test cases and thrown them out the door when you start automating. This is not a good idea.
Rather than changing the test case to something which is easy to automate, you need to figure out how to automate the test steps. This is what separates good automation from bad automation.
Many times I have seen a test case automated. It gets run and passes. We ship the application to the customer. He uses the same feature and it fails horribly. Why? Because the steps he used to get to the defect where not the steps we automated. We had a good test case. If an experienced tester had executed the test case manually, they would have found the defect. The person automating the test case just found it easier to automate something close to but not equivalent to the test case.
I am currently using Selenium 2.x with WebDriver. One of the changes from Selenium 1.x to 2.x is that you cannot interact with invisible elements. For example, a common trick on a website is to have an Accept checkbox on a download page. If you accept the terms the Download button becomes visible. In Selenium 1.x I could click on the Download button without clicking the Accept checkbox. The REAL test case was:
What someone would automate with Selenium 1.x was:
The idea is that it saves a step. One less step means quicker to code, runs quicker, one less thing to maintain. You do this a thousand times and it adds up. HOWEVER, the customer would never click on the invisible Download button.
In Selenium 2.x you would get a failure with the shortened test case. People occasionally complain that Selenium 2.x has removed an important feature. They want to know how they can click on the invisible Download button. They come up with these tricky Javascript snippets which will allow Selenium 2.x to 'see' and click the Download button. Is a customer going to create a Javascript snippet, inject it into the page, run it just so they can click the Download button? Is a manually tester going to do this? If the answer is no, then why is our automation doing this? If the manual test case calls for clicking the Accept checkbox then our automation should as well. If clicking the Accept checkbox does not enable the Download button, file a bug and move on to something else.
Finally, automation is all about finding elements on a page, interacting with them (clicking, right clicking, typing, etc.) and checking what happened. As a manual tester you are going to use your eyes and hands to do everything. The test case might have a step like, "Locate the folder called 'My Documents' and double click it." This is really two steps. The automation should locate the folder called 'My Documents', this is step 1. It should double click the element, this is step 2. As a manual tester I find the element by looking for the text 'My Documents'. If this is a web page and the HTML is:
<div id='lsk499s'><a href="...">My Documents</a></div>
I am not going to use the div id to find the element. I'm going to use the text. As a manual tester I used the text to find the element. There is no reason to do anything differently with the automation tool.
What if the web page is made to look like Window Explorer. On the left is a tree view with one of the nodes being 'My Documents' and on the right is a thumbnail view with a 'My Documents' folder. In the manual test case, does it specify which 'My Documents' to double click? If yes, follow the test case. If no, how would you as a tester decide? Do you always go with the thumbnail view? Do you pick randomly? Do you change ever test run? If you are a good manual tester, we want that experience captured by the automation. If I would normally change every test run but I never test the same feature twice in one day, it might be sufficient to say, if the day of the year is even, double click thumbnail else double click tree view. If the automation gets run daily, it will pick a different way each day.
The important thing in all this is that I am a good tester. I write good test plans and test cases. When I decide to automation my good test cases, I should not compromise the quality of my testing just because I am testing it with an automation tool rather than manually.
Happy testing!
.
The test cases I create for an application are based on what is important to the customer. I want to make sure that the customer experience is a good experience. If I create a set of test cases or use stories which reflect real customer usage of the application then I am most likely to find issues which will affect the customer.
I remember working on a project for 4 years. After 4 years of testing and improving the application we were at a point that over 90% of the features important to the customer were tested and bug free. Of the remaining 10% we knew of most the defects and had a work-around. We were at a point where we were testing extreme edge cases. At this point I found a major defect. The developer looked at the fix and realized the defect had been there since the beginning. In 4 years not one single customer reported this defect. The defect was easy to automate but really added zero value to the application. This is NOT a test case you want to start with when automating.
On another project someone found a defect in a desktop application. The steps to reproduce were:
- Run an application not involved in the test case at all (Outlook for example)
- Hand edit a project file using notepad or something other than the application it was intended for
- Make a very specific change to the file
- Run the application under test
- Open the corrupted file
- Select a feature which relies on the corrupted file
- A modal dialog appears telling you the file is corrupt, do you wish to repair it.
- Ignore the dialog
- Use CTRL-TAB to switch to a different application not involved in the test case at all
- Click on the application under test in a very specific location on the MDI client window
At this point the modal dialog is hidden behind the window with focus and the window with focus appears to be frozen. It is really waiting for you to respond to the modal dialog. This was a design flaw in the operating system. It was virtually impossible to fix in the application under test without a major re-design. It was highly unlikely a customer would be able to reproduce this defect. When the tester showed me the 'locked' state it only took me a few minutes to figure out what was going on. Our customer was typically a software developer with 7+ years of experience.
This was a useless test case. In both this and the previous test case it was a bad test case regardless of creating it manually or automating it. My point is, the test case came first. Even before we attempted to automate it, we decided whether or not it was a good test case.
Test automation NEVER proceeds test case creation or test planning.
Once you know what you want to test and the steps to testing it, you automate those steps.
This is the second mistake I see people getting into automation making. They decide WHAT they want to test but when they start automating it, the steps they generate with the automation are not the same steps as they would do manually. In this case you have taken the time to create a good set of test cases and thrown them out the door when you start automating. This is not a good idea.
Rather than changing the test case to something which is easy to automate, you need to figure out how to automate the test steps. This is what separates good automation from bad automation.
Many times I have seen a test case automated. It gets run and passes. We ship the application to the customer. He uses the same feature and it fails horribly. Why? Because the steps he used to get to the defect where not the steps we automated. We had a good test case. If an experienced tester had executed the test case manually, they would have found the defect. The person automating the test case just found it easier to automate something close to but not equivalent to the test case.
I am currently using Selenium 2.x with WebDriver. One of the changes from Selenium 1.x to 2.x is that you cannot interact with invisible elements. For example, a common trick on a website is to have an Accept checkbox on a download page. If you accept the terms the Download button becomes visible. In Selenium 1.x I could click on the Download button without clicking the Accept checkbox. The REAL test case was:
- Go to download page
- Click Accept checkbox
- Click Download button
- Confirm download
What someone would automate with Selenium 1.x was:
- Go to download page
- Click Download button
- Confirm download
The idea is that it saves a step. One less step means quicker to code, runs quicker, one less thing to maintain. You do this a thousand times and it adds up. HOWEVER, the customer would never click on the invisible Download button.
In Selenium 2.x you would get a failure with the shortened test case. People occasionally complain that Selenium 2.x has removed an important feature. They want to know how they can click on the invisible Download button. They come up with these tricky Javascript snippets which will allow Selenium 2.x to 'see' and click the Download button. Is a customer going to create a Javascript snippet, inject it into the page, run it just so they can click the Download button? Is a manually tester going to do this? If the answer is no, then why is our automation doing this? If the manual test case calls for clicking the Accept checkbox then our automation should as well. If clicking the Accept checkbox does not enable the Download button, file a bug and move on to something else.
Finally, automation is all about finding elements on a page, interacting with them (clicking, right clicking, typing, etc.) and checking what happened. As a manual tester you are going to use your eyes and hands to do everything. The test case might have a step like, "Locate the folder called 'My Documents' and double click it." This is really two steps. The automation should locate the folder called 'My Documents', this is step 1. It should double click the element, this is step 2. As a manual tester I find the element by looking for the text 'My Documents'. If this is a web page and the HTML is:
<div id='lsk499s'><a href="...">My Documents</a></div>
I am not going to use the div id to find the element. I'm going to use the text. As a manual tester I used the text to find the element. There is no reason to do anything differently with the automation tool.
What if the web page is made to look like Window Explorer. On the left is a tree view with one of the nodes being 'My Documents' and on the right is a thumbnail view with a 'My Documents' folder. In the manual test case, does it specify which 'My Documents' to double click? If yes, follow the test case. If no, how would you as a tester decide? Do you always go with the thumbnail view? Do you pick randomly? Do you change ever test run? If you are a good manual tester, we want that experience captured by the automation. If I would normally change every test run but I never test the same feature twice in one day, it might be sufficient to say, if the day of the year is even, double click thumbnail else double click tree view. If the automation gets run daily, it will pick a different way each day.
The important thing in all this is that I am a good tester. I write good test plans and test cases. When I decide to automation my good test cases, I should not compromise the quality of my testing just because I am testing it with an automation tool rather than manually.
Happy testing!
.
Subscribe to:
Posts (Atom)