Google Analytics

Search

To search for specific articles you can use advanced Google features. Go to www.google.com and enter "site:darrellgrainger.blogspot.com" before your search terms, e.g.

site:darrellgrainger.blogspot.com CSS selectors

will search for "CSS selectors" but only on my site.


Friday, June 20, 2014

Desktop automation tools

Talking to one of my colleagues today. He asked me about testing desktop applications. As someone who predates the Internet, testing desktop applications was not foreign to me. However I realized that many testers today have only worked on web or mobile device applications.

Testing things like Microsoft Word, Notepad, Calendar or Eclipse isn't something many testers have done.

You wouldn't use something like Selenium, cucumber or Watir to test a desktop application.

So where would you go to find desktop application test tools? The first place I look is http://en.wikipedia.org/wiki/List_of_GUI_testing_tools. At this time this page notes:


  • AutoIt: http://www.autoitscript.com/site/autoit/ (free)
  • IBM/Rational Functional Tester: http://www-03.ibm.com/software/products/en/functional (commercial)
  • Quality First Software: http://www.qfs.de/en/index.html (commercial)
  • Sikuli: http://www.sikuli.org/ (free)
  • SilkTest: http://www.borland.com/products/silktest/ (commercial)
  • Test Automation FX: http://www.testautomationfx.com/ (commercial)
  • Telerik TestStudio: http://www.telerik.com/teststudio (commercial)
In addition to these links I would also check out:
  • http://www.ministryoftesting.com/resources/software-testing-tools/
  • http://sourceforge.net/directory/development/development-testing/freshness:recently-updated/
The first site has a wide range of links and it currently actively maintained. Unfortunately, it has more links for web and mobile testing but there are some desktop application tools listed.

The second link is to sourceforge. It will take you to the recently updated testing links. It includes open source applications for all sorts of testing. You can remove the "recently-updated" filter and see more but I usually restrict myself to recently updated tools. 

If the tool is for desktop testing and it has not been recently updated, there is a strong chance it does not work with modern applications or a current operating system. The older the tool the higher the risk it will not work on Windows 7 or 8, Mac OS X 10.8 or 10.9.

If you enter in a search term, like "testing" then a whole set of menus will appear below the search term and you can narrow things down even further. For example, after entering "testing" into the search text box I can select OS to be "Mac" and category "Quality Assurance".

I can also clear all the filters and search for things which might help narrow it down in different ways. For example, if I know how to program Java and I think I might want to add to the open source tool then I can filter for things with Programming Language "Java". Additionally, if the desktop application is written using Swing and Java I could search for Programming Language "Java", Category "Testing" then enter "swing". From the results I find jrobot (http://jayrobot.sourceforge.net/) and gtt (GUI Test Tool; http://gtt.sourceforge.net/). 

It is also worth just poking around sourceforge and keeping a mental note about things you find. This is because you'll be surprised what you stumble. For example, JFCUnit is a good tool for testing Java Swing applications but it didn't come with JFCUnit (http://jfcunit.sourceforge.net/).


Tuesday, June 10, 2014

What is agile testing?

There are books on agile testing and there are people who claim to be agile testers (as opposed to??). But what does this actually mean?

In recent years I have worked at companies/clients who claim to be 'agile' or I hear of companies who say they have a prescription for 'being' agile.

But agile is a concept and a way of doing software development. To me it is almost as if, when you start creating lists or writing down a checklist of activities you need to do in order to be agile is right when you stop being agile.

Doing agile software development is constantly thinking about how you can be better, faster, more efficient in what you are doing to get your software to the people who can use it or who will be affected by it.

I started being agile probably around 1998. Actually I was probably trying to be agile before then but around that time was when I started working with other like mind individuals.

The first thing I started noticing was that the developer to tester ratio caused testing to be the bottleneck in the software development life cycle. So I had to start thinking about how I can reduce risk, test more with less effort.

One possible aid was automating tests. Writing automation was not faster than manually testing an application but if I need to test the application on Windows 95, Windows 98, Windows 98 Second Edition, Windows ME, Windows NT4 and Windows 2000 then I could test it 6 times, I could test it fully once and some limited testing 5 times (what I was doing before automation) or I could take 3 times longer automating it (than manually testing it once) and run it 6 times.

Before test automation I would test one platform completely then pick the highest priority tests and run them on the other 5 platforms. This made testing faster but because I didn't test everything on every platform it increased risk.

Writing automation and running it 6 times maintained the risk but allowed me to test everything faster. It was this last idea that made test automation the best way to do all this repetitive testing.

I wasn't agile because I was writing test automation. I wrote test automation because I wanted to be agile. So if you write test automation it doesn't make you agile. If your test automation makes you able to test more without increasing risk then you are being agile.

Now I have seen people who write automation which takes days or even weeks to run. This is not faster than manual testing in some cases. Or running the tests and analysing the results might take longer than manually testing everything. It is automated but this is not agile.

What if we have an emergency fix? Someone finds a defect in production code. They figure out what is wrong and fix it. I am then tasked with testing the new build before we deploy to production. I could run the entire test suite on the new build. Or I could use the source control system to find out the difference between the production code and the fix. I could examine the source code, understand what has changed and manually test only what I determine would be effected by the fix.

The best part is that the fix might expose a new problem that the test automation isn't even testing for but my brain should quickly see the potential danger. It is faster to do this focused testing and I probably have a better chance of spotting new problems an automation test suite would not detect. This is agile, faster and reduce risk. I could run the test suite as well but if the tests are still running when we are ready to go to production, we could release to production and decide if it needed to be rolled back should something in the test suite fail. Getting to production versus the likelihood the tests will fail. I'm not the gatekeeper trying to stop the code from going to production. I am part of the development team trying to improve the quality of the code in production. It is no longer a black and white issue. If it is better than what is in production but still not perfect, we should still release it.

How about testing the right thing and understand what the right thing REALLY is. Old school testing was when you are given a list of requirements and you simple checked off the requirements were met. This does not mean you implemented a good, high quality piece of software.

As part of agile software development, we have stories. Everyone needs to understand what the details of a software are. The developer needs to understand the story so they can implement the requirements (or acceptance criteria). A tester needs to understand the story so they can confirm the requirements have been implemented correctly. The key is that the developer, tester, business analyst and possibly the product owner all need to confirm with each other what the requirements are before they are ready. We are essentially testing the requirements before we test they have been met.

Cam we go even further? At one company I was testing software for a retail store. It was a store I shopped at. I could have claimed to be representative of the customer because I was a customer. But was I really the typical customer? I was not the typical customer. How did I determine this? I would go to retail stores after work and just observe other customers. I would go to different stores on different days and at different times. Where people on their smart phones? Did anyone use the computer terminals at the entrance? I found the typical customer was much older or a mom with kids. I saw people with ads and coupons printed out on a 8.5" x 11" paper.

I would chat with people in the store. I didn't tell them who I was. I asked them how they found out about sales, where did they get coupons, did other stores do stuff they wished this store did on the website, etc. I found if people thought they could get you implementing their personal agenda or they were being surveyed that their answers weren't honest. If I was just a chatty shopper in the store I feel I got a more honest response.

So next time you are thinking about being an agile tester, do something different. Visit the customer in their natural environment. Visit different stores and see if different neighbourhoods have different customers or does the store always attract the same sort of person.