Google Analytics

Search

To search for specific articles you can use advanced Google features. Go to www.google.com and enter "site:darrellgrainger.blogspot.com" before your search terms, e.g.

site:darrellgrainger.blogspot.com CSS selectors

will search for "CSS selectors" but only on my site.


Wednesday, November 30, 2011

Interface versus implementation

I've seen a few people posting Java code for Selenium automation where they have things like:

FirefoxDriver driver = new FirefoxDriver();

If you look at the code for Selenium you will see they have an interface called WebDriver. FirefoxDriver, InternetExplorerDriver, HtmlUnitDriver, RemoteWebDriver all implement WebDriver.

If I'm writing a test suite, I ultimately want to run the suite on Firefox, Internet Explorer, Chrome, etc. but if I implement my framework using FirefoxDriver, I have to edit the code to make it work on InternetExplorerDriver. The proper solution is to write your code using:

WebDriver driver = new FirefoxDriver();

You might argue that I need to change my code still. Okay, so you would implement it using:

WebDriver driver;
    if(browserType.equals("firefox")) {
        driver = new FirefoxDriver();
    } else if(browserType.equals("ie")) {
        driver = new InternetExplorerDriver();
    } else {
        driver = new RemoteWebDriver();
    }

Now all subsequent code will use driver as if it was merely a WebDriver. However, if you look at RemoteWebDriver you will see that it also implements JavascriptExecutor. So if I wanted to use:

driver.executeScript(script, args);

I'll get an error because WebDriver does not support executeScript. My driver is REALLY a RemoteWebDriver and does support executeScript. So how do I access that functionality? Quite simple:

((RemoteWebDriver)driver).executeScript(script, args);


Even better would be:

if(driver instanceof RemoteWebDriver) {
        ((RemoteWebDriver)driver).executeScript(script, args);
    } else if(driver instanceof FirefoxDriver) {
        ((FirefoxDriver)driver).executeScript(script, args);
    } else if(driver instanceof InternetExplorerDriver) {
        ((InternetExplorerDriver)driver).executeScript(script, args);
    }


Now if you run the test suite against a browser that does not support a particular feature, it will skip that part of the code automatically.

And that is proper use if polymorphism.

Monday, November 28, 2011

Random crashes

New application under test is a web based application. We have Windows and Mac OS X customers. All our Mac OS X customers are using Safari 5.0 or 5.1. Most our Windows customers are using Internet Explorer. We use the meta tag to force Internet Explorer 8 compatibility if you are using Internet Explorer 9. A handful of our Windows customers are using Firefox (whatever is latest).

So I need to create an automated test suite for:
  • Safari 5.0 on Mac OS X 10.6 (Snow Leopard) 32-bit OS
  • Safari 5.1 on Mac OS X 10.7 (Lion) 64-bit OS
  • Internet Explorer 8 on Windows XP 32-bit OS
  • Internet Explorer 9 on Windows 7 64-bit OS
  • Firefox 8 on Windows 7 64-bit OS
With this combination I feel I have adequately covered the combinations our customers will care about. If time allowed and everything was written with one automation technology, there is no reason I couldn't run it on other combinations. For example Internet Explorer 9 on Windows XP 32-bit OS or Firefox 7 on Windows XP 64-bit OS.

The assumption is that the above 5 combinations will find most the defects.

My problem is that Safari on Mac OS X is as equally important as Internet Explorer on Windows.

After poking around the web I found Selenium 2.0 (WebDriver) supports all the platforms, sort of. There is no Safari driver for WebDriver but I have a snippet here for using Selenium 1.0 to create a Safari instance, from that I can create a SeleneseCommandExecutor and with the Executor I can create a RemoteWebDriver. A bit hacky but it works.

So I started creating my test automation. I got up to 39 test cases when I decided to run them as a single suite. Ran them on Firefox 8 and everything worked perfectly. Ran them on Internet Explorer. Around 25 test cases in and Internet Explorer pops up a dialog telling me something went wrong with Internet Explorer. I can debug it (because I have Visual Studio installed) or just close it. Regardless, my automation is dead. If my automation dies after the 25th test case and I build it up to 45,000 that would be totally useless. I wouldn't be completely happy with it but if it at least recovered and continued on to the next test case it would be better. But it does not.

So the hunt goes on. I have found a commercial product which claims to work for all the above platforms and more. I'll split up time between evaluating the commercial product and debugging Selenium. Hopefully by  the end of the week I'll have a decision one way or the other.

Stay tuned...

Monday, November 7, 2011

Using Oracle SQL Developer with SQL Server

If you are looking for a SQL Client for both Oracle DB and Microsoft SQL Server you can use the Oracle SQL Developer with the jTDS plugin.

Get Oracle SQL Developer from the Oracle web site. Just use your favourite search engine to search for 'Oracle SQL Developer'. These instructions have been verified to work with SQL Developer 3.0.

Once you have installed Oracle SQL Developer, run it and go to the Help menu. There you should find the Check For Updates... option. There will be a dialog with Third Party SQL Developer Extensions as an option. Make sure this is selected.

On the next page there will be an option to select the jTDS extension. Select this for SQL Server and Sybase support.

Once you accept the license agreement and restart SQL Developer, you will now have a tab for SQL Server when creating a new connection.

Monday, September 5, 2011

maintenance versus performance

I was recently sent an example xpath for Selenium automation. The xpath, for a DIV, was:

"//*[@class="val1 val2 val3 val4"

My experience has been, especially with Internet Explorer, that xpath is very slow compared to say CSS. However, I find xpath easier to read (thus easier to maintain, especially in a team enviroinment) and certain things are just not possible in CSS.

So there is always a balance of maintenance versus performance. The golden rule for optimization is to never optimize right away. Right the code for maintainability and if you need to improve performance, then and only then optimize it.

The first thing I notice about the xpath above is the = operator. If the developer adds another value to the class attribute or the order of the values change, this will cause the automation to require maintenance. My experience has been that the values of multiple value attributes do change over time. So from a maintenance perspective I would write this as:

"//*[contains(@class,'val1') and contains(@class,'val2') and contains(@class,'val3') and contains(@class,'val4')]"

This way, if the order of the attributes change or the developer adds another value the xpath will still find the element.

The other things I'd change about the xpath is the use of //*. This will search every element for a match. Rather inefficient. It does have the nice feature that if a developer changes the element from a DIV to something else, the xpath will require no maintenance. However, and this is just from years of experience, I find it rare that a developer changes a DIV to something else. If they do, there is a huge amount of refactoring and using //* will not save you from any maintenance.

Since the likelihood of maintenance is low but it is such an obvious performance hit, I would actually use //div for the xpath.

Thursday, September 1, 2011

How to do focused testing

After years of software testing I have found one constant... there is never enough time to test everything.

So how do I pick what I'm going to test? Here are a few tricks I use to focus what I'm going to test.

First, test what changed. I like to get a snapshot of the last release (source code) and a snapshot of the current release candidate then compare them. Often the source code is readable enough for any tester to understand what has changed. If you are not a programmer, not familiar with the particular language or the code requires a little indepth knowledge only the programmers have then you might want to talk with some of the developers about what you are looking at.

For example, if I am testing a shopping cart for an online web store and I see changes to a class called TaxRules then I'm going to look at the requirements for tax rules and see if there is anything new. Even if there aren't requirements for tax rules, I can talk to the person who wanted this feature (or improvement) and I can look at the code. Comments will be for humans and I'm human (no really, I am). Good code should also be self documenting. In other words, the variables and method names should almost read like English.

The next trick is to get someone to add a code coverage tool to the web site (or whatever it is you are testing). As you test the application (automated or manually) the coverage tool will record which parts of the code you executed. Look to see of the code which has changed was executed. Set the coverage tool so it records actions within the methods and not just method level data. This way you can see which branch conditions were called. If there is a loop which never gets entered or a portion of an if/elsif/else which does not executed, you know you need another test case. You could even ask the developer to relate the statements missed to user actions. Essentially, get them to tell you what test case you missed. :)

Finally, look at the inputs to the methods. Can you see a way to affect the inputs? Does the method handle situations where the data is bad? Developers think in terms of "make the program do what it is supposed to do." They always make it do the right thing when given the right data. As a tester you need to think in terms of "make sure the program doesn't do what it is not supposed to do." In other words, if you input bad data, it should report a nice, helpful message to the user or just restrict the user's ability to input the bad data.

As you do code coverage, you can keep notes with your test cases. Just a comment about how test case XXX should be run with class YYY is altered.

Monday, July 25, 2011

Tools to determine how to use Selenium

Being able to view the source code to a page is helpful when trying to use Selenium. Many of the methods in Selenium require you to locate the element via something related to the source code. For example, if I wanted to click on an element I need to look at the source code. From the user's point of view they might see:

Click Me

But the underlying source code could be:

<span>
    <div>
        <table>
            <tbody>
                <tr><td><strong onclick="javascript:clickme();">Click Me</strong></td></tr>
            </tbody>
        </table>
    </div>
<span>

I might assume there is an anchor and I want to find the <a> element with the text 'Click Me'.  By looking at the source code I can see that I actually want to find a <strong> element and click on it.

Just viewing the source code isn't always sufficient however. AJAX and Web 2.0 means that the source code gets changed in memory after it is loaded. So what is actually available in the DOM (and Selenium) might be different than what you see when viewing the source code.

So how do you see what Selenium sees? A simple solution is to write your Selenium code to get to a point in the program then print out the source code using Selenium. Selenium 1.0 and 2.0 both have methods to return the HTML source code as a string, which you can print.

The problem with this is that you might have to write a program to print the source. Then with the source, find the code you want to click. Then run the program so it clicks the element then prints the new source. Then run the program to click the first element, click the second element and print the source. For n steps this will take O(n^2) run time. Not very efficient.

So how can you view the dynamically changing source? The answer is in browser tools.

If you are using IE7, search for "microsoft dev toolbar". When you install it, it gives you development tools which can view the page in real time. If you are using IE8, the tool is built in. Just press F12 and it will display the development tools.

If you are using Firefox, install the Firebug extension.

If you are using Google Chrome, right click an element and select Inspect to bring up the development tools.

Thursday, May 26, 2011

Defensive programming

Had an interesting conversation with a programmer today. He created a program for updating pricing on books. It allowed you to update the price of one book at a time or to load a list of volume ids (sort of like ISBNs) and batch update the prices.

Many of our partners use Excel to manage their list of books. So the programmer was told the batch screen had to load Excel files.

My first thought when testing the application was, "How can users mess up an Excel file?"

The application required a header row with specific text. What if:
  • there were extra columns
  • a column was missing
  • there was whitespace before or after the text
  • there was no header row
  • the text was all lowercase
  • the text was all uppercase
  • the text was camel cased (ThisIsCamelCase)
Tried all these and each one crashed the program. The programmer was using a third party library to access the Excel spreadsheet. I had used similar libraries in the past and knew these to be problems with those libraries. Essentially, you need to guard against bad data as these libraries don't.

When LAMP (Linux/Apache/MySQL/Php|Perl|Python) programmers access the database, they never assume the data coming in is safe. They check all the data before using it to access the database. This is because hackers will do things like input the following for First Name: "Hacker; DROP TABLE ..." or some other destructive text.

The lesson learned from this is to be a defensive programmer. LAMP programmers know this. I believe this to be true for people using third party libraries but also when using the code from someone else on your own team. If you assume the library does not handle edge cases and ensure the data has been checked before getting passed to the other programmer's code.

If you assume all code (third party libraries, your old code, code from other programmers on your team, etc.) does not handle code properly and check the inputs before passing them along to that code, your applications will be much more solid.

To the testers reading this, assume all programmers will assume someone else will take care of guarding against bad inputs. So think of all the bad inputs you can and send them in. You'll usually find they make it quite far before someone handles them. Often it will be the operating system that handles this with a crypt error dialog and a crash.