Google Analytics


To search for specific articles you can use advanced Google features. Go to and enter "" before your search terms, e.g. CSS selectors

will search for "CSS selectors" but only on my site.

Wednesday, June 4, 2008

A monthly thing

Seems life is turning my site into a monthly event. This month I'm going to talk about boundary test cases and a little about resumes.

When you are testing an application you want to be systematic. Even the simplest of applications can have so many possible combinations that testing all of them does not make sense. What you need to figure out is what subset is sufficient.

For example, a web site I was testing had security and a timeout. You had to log in and the system would automatically log you out after a certain period of inactivity.

Recently, they made the timeout period configurable. You would edit a config file and set a property. If the application.timeout property existed and had a value of 30 it would automatically log users out after 30 minutes of inactivity.

So, how do you test this. I could try all possible values but the application server takes 10 minutes to power up and 5 minutes to power down. Each value I'd use then means 15 minutes plus the timeout period. Even if it only supported up to 99 minutes it would take me one solid week to test it. If I took this long for every feature it would take me decades to test the application. In other words, before I finished testing it, the computer would be obsolete.

What I need to do is pick a subset. If 1 worked and 2 worked, I'd guess that 3, 4, 5, 6, etc. will work. If I understand the language the application was written in, I might be able to figure out special cases. It was written in Java. Is the timeout value stored in an int? long? Integer? If the data type is 8 bits then I know for Java there are no unsigned char so the data range is -127 to 128. What happens if I use 129? What happens if I use a negative number? The value 0 will be fine for the char but what will the timeout code do with a 0? Does 0 mean 'disable timeout'?

Turns out the programmer used a 32 bit number. This means it ranges from 2147483648 to -2147483647. So I could try a value of 2147483649 (MAX+1). If they are using the input as a String then using Integer class to convert it, the Integer class will throw an Exception. What about if I set the value to "twenty". Did they think to handle that. For the application I'm testing, the user is a Application Server Administrator for LARGE enterprise environments. I didn't test for "twenty" because our users wouldn't try that.

At this point you might be wonder, what about the resume thing? I see a lot of testers putting programming languages on their resumes. I test development tools. All my staff are programmers and I expect anyone I hire to be a programmer. So for me putting a language on your resume means you know how to program in that language. If you don't I have to put you on a different team (e.g. testing the user interface for application monitoring solutions). If I was hiring for someone testing a non-development application, the language on the resume might mean you understand the limitations of the language and can apply that to your testing.

For example, if you note you test Java and C applications, it means you understand that C has unsigned data types and Java does not. In Java the boundary cases are going to be 2^7, 2^15 and 2^31 but in C language it will be all the Java boundary cases PLUS 2^8, 2^16 and 2^32 (there are actually more but this shows the difference between Java and C applications). All the people I've interviewed had no idea why the language the application was written with made a difference. When I asked them why they put it on their resume they had no idea. If you don't know why something is worth putting on your resume ask. If no one can tell you, don't put it on your resume until someone can explain why.

No comments: