Generating a file of a specific size

Every once in a while someone is looking for a file of a specific size. Occasionally, it must be real data. If you are transmitting the file and the data will be compressed then the type of data will make a difference.

However, if you just need a file to fill some space or there will be no compression then Windows has a neat little utility called FSUTIL.

The FSUTIL file can be used for a number of things but the nicest feature is creating a new file filled with zero bytes. First, you need to know how many bytes. If you want a file which is 38 gigabytes then you need to figure out how many bytes that is. Technically, it is 1024*1024*1024*38. If you want  a rough idea you can just use 38,000,000,000 but a 38 gigabyte file is really 40,802,189,312. Next is which file you want to hold the data. Let's say you want to create C:\DELETEME.TXT then the full FSUTIL command is:

fsutil file createnew C:\DELETEME.TXT 40802189312

This will create a 38G file is a matter of seconds.

For UNIX, Mac OS X or Linux you can use DD. The DD command is for converting and copying files. If we wanted to create the 38G file with DD the command would be:

dd of=deleteme.txt oseek=79691776

The oseek is the magic. It will seek n*512 bytes. The standard size of a block is 512. So we take 79691776*512, which is 38G. Even easier would be:

dd of=deleteme.txt oseek=38m obs=1024

This will generate a file of 38M * 1024 or 38G. Much easier to figure things on working with values like these. That is, no need for a calculator.

IMPORTANT: after you press enter on the DD command it will take the standard input as its input. So you need to enter CONTROL-D.

The other option for UNIX/Linux/Mac OS X is to copy a file of a set size. The nice thing about this is you can take a file with real data and copy it enough times to make a single file of the correct size. For example, if I have a text file with 512 bytes of real data I can set the if= to that file and make multiple copies of that one file into the output file.

Removing logout button

When I set up a Selenium Server it only works when someone is logged in. In a previous article I wrote how to make a user automatically log in. For this article I'll write how to prevent that user from logging out.

1. Run regedit.exe
2. Go to HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies
3. Create a new Key called Explorer
4. Go to Explorer folder
5. Create the DWORD key StartMenuLogoff.
6. Set it to 1.
7. Go to the Start Menu and use Shutdown or Reboot (so Logoff is not the last option used).

When you log in as that user now, they will not have a Logoff button available to them. This has been tested on Windows 7.

automatic log in, automatic run

When writing test scripts for testing an installer, on Windows, one of the challenges is when the installer requires a reboot. To handle this I use automatic log in and the run once features of Windows.

To create automatic log in:

1. Run regedit.exe
2. Go to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon
3. Set DefaultUserName to the name of the person you want to automatically log in. If the key does not exist add it as REG_SZ.
4. Set DefaultPassword to the user's password. If the key does not exist add it as REG_SZ. Note: anyone who can read the registry can see the user's name and password. Create a local user with no permission on your network.
5. Set AutoAdminLogon to 1. If the key does not exist add it as REG_SZ.
6. Set ForceAutoLogon to 1. If the key does not exist add it as REG_SZ.

And that is all you need to have the computer automatically log in after a reboot.

The second part is to create a script which will start up the automation at the right place. What usually happens is the installer will put something in place so that after the reboot it will continue with the install the moment the user logs in. If this is the case for your installer, write a script which continues the automation and add it to the RunOnce key. Assuming you have a batch file which starts part 2 of your automation:

1. Run regedit.exe
2. Go to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce
3. Add a string value
4. The Name of the should be something like zzz_Automation
5. The Data value will be the full path to the batch file. It is important to note that the current directory will not be the location of the batch file. Do not assume it will be in your script. Either use full paths for everything in the script or change to the directory you think you should be in then continue with the script.

Enjoy.

Generating a screen capture when using RemoteWebDriver

Recently I was asked if all the different implementations of WebDriver allow for screen captures. I had a look at the source code and see that almost all of them "implement TakesScreenshot". If the Java class for the driver implements TakeScreenshot then it allows for screen captures. This means a previous posting (http://darrellgrainger.blogspot.ca/2011/02/generating-screen-capture-on-exception.html) there isn't really any need for using Robot to generate the screen capture.

The nicest thing about using the built-in screen capture is that Robot will capture the visible screen. The built-in screen capture will capture the entire window in the browser, including the parts not visible, i.e. things you have to scroll to see.

Shortly after this someone asked of RemoteWebDriver supported screen capture. This was very important because Robot would not capture even the visible part of a REMOTE screen.

A quick check of RemoteWebDriver.java showed that it did NOT implement TakesScreenshot.

Recently however I found a solution. Assuming you have the code:

DesiredCapabilities dc = DesiredCapabilities.firefox();
URL url = new URL("http://localhost:4444/wd/hub");
WebDriver driver = new RemoteWebDriver(url,dc);

You can change the driver to:

WebDriver driver = new Augmenter().augment(new RemoteWebDriver(url,dc));

Now it has the capability to take a screen shot. To make the call, here are the three different outputs:

File f = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
String s = ((TakesScreenshot)driver).getScreenshotAs(OutputType.BASE64);
byte[] b = ((TakesScreenshot)driver).getScreenshotAs(OutputType.BYTES);

And that is all there is to taking a screenshot with RemoteWebDriver. I have tried this with Firefox and InternetExplorer. I assume it works with the other browsers but I leave that for you to explore.

Creating a screen capture on every action

Someone recently commented on an article I wrote about generating a screen capture when an exception is thrown (see Generating a screen capture on exception thrown with Selenium 2).

Performing an action when an exception is thrown is built into the Selenium framework. You just need to create the action to generate a screen capture and hook it into the framework.

The Selenium framework comes with a WebDriverEventListener interface. In my article above I created an implementation of the WebDriverEventListener interface. To do this properly, you need to implement the following methods:

beforeNavigateTo(String url, WebDriver driver);
afterNavigateTo(String url, WebDriver driver);
beforeNavigateBack(WebDriver driver);
afterNavigateBack(WebDriver driver);
beforeNavigateForward(WebDriver driver);
afterNavigateForward(WebDriver driver);
beforeFindBy(By by, WebElement element, WebDriver driver);
afterFindBy(By by, WebElement element, WebDriver driver);
beforeClickOn(WebElement element, WebDriver driver);
afterClickOn(WebElement element, WebDriver driver);
beforeChangeValueOf(WebElement element, WebDriver driver);
afterChangeValueOf(WebElement element, WebDriver driver);
beforeScript(String script, WebDriver driver);
afterScript(String script, WebDriver driver);
onException(Throwable throwable, WebDriver driver);

In my screen capture when an exception is thrown, I put the necessary code to generate a screen capture in the onException method. You can see from the list above that you can have Selenium perform an action when other events occur. For example, if you wanted to do a screen capture after each click() action, you could write:

public void afterClickOn(WebElement element, WebDriver driver) {
    String filename = generateRandomFilenameFromWebElementAndDriver(element, driver);
    createScreenCaptureJPEG(filename);
}

I'll leave it to the reader to figure out how to create the generateRandomFilenameFromWebElementAndDriver. You might do something like use getCurrentUrl(), convert things like colon, slash, etc. into valid filename characters then append the web element getTagName() to the end of the filename.

Or you could do something tricky like have the beforeClickOn generate the filename and the afterClickOn use that filename to generate the screen capture.

If the event you want to trigger a screen capture is not listed above, there is no easy answer for how you would do it.

Initially, you might considering changing the WebDriverEventListener interface. But this would require a change to how Selenium works. Every time there is a new release of Selenium, you would have to merge your changes back in.

I would recommend submitting the changes as a new feature to the Selenium project to see if they'll incorporate it or wrap the Selenium methods with your own methods. For example, you would call your sendKeys() which could do a driver.sendKeys then do a screen capture.

StaleElementException

Most automation tools depend on the concept of the page has finished loading. With AJAX and Web 2.0 this has become a grey area. META tags can refresh the page and Javascript can update the DOM at regular intervals.

For Selenium this means that StaleElementException can occur. StaleElementException occurs if I find an element, the DOM gets updated then I try to interact with the element.

Actions like:

driver.findElement(By.id("foo")).click();

are not atomic. Just because it was all entered on one line, the code generated is no different than:

By fooID = By.id("foo");
WebElement foo = driver.findElement(fooID);
foo.click();

If Javascript updates the page between the findElement call and the click call then I'll get a StaleElementException. It is not uncommon for this to occur on modern web pages. It will not happen consistently however. The timing has to be just right for this bug to occur.

Generally speaking, if you know the page has Javascript which automatically updates the DOM, you should assume a StaleElementException will occur. It might not occur when you are writing the test or running it on your local machine but it will happen. Often it will happen after you have 5000 test cases and haven't touched this code for over a year. Like most developers, if it worked yesterday and stopped working today you'll look at what you changed recently and never find this bug.

So how do I handle it? I use the following click method:

public boolean retryingFindClick(By by) {
        boolean result = false;
        int attempts = 0;
        while(attempts < 2) {
            try {
                driver.findElement(by).click();
                result = true;
                break;
            } catch(StaleElementException e) {
            }
            attempts++;
        }
        return result;
}

This will attempt to find and click the element. If the DOM changes between the find and click, it will try again. The idea is that if it failed and I try again immediately the second attempt will succeed. If the DOM changes are very rapid then this will not work. At that point you need to get development to slow down the DOM change so this works or you need to make a custom solution for that particular project.

The method takes as input a locator for the element you want to click. If it is successful it will return true. Otherwise it returns false. If it makes it past the click call, it will return true. All other failures will return false.

Personally, I would argue this should always work. If the developers are refreshing the page too quickly then it will be overloading the browser on the client machine.

When trial and error is a bad thing

I'm a hacker. Not the breaking into systems and doing damage hacker. To me a hacker is someone who learns things by trial and error. I will systematically poke away at something which for all intents and purpose is a black box. I hack to learn. I hack things I own. I'll create an instance of something and hack away at it. People I work with will create development, test or staging environments which I will hack.

I do not hack sites I don't own or have permission to hack. This is what differentiates good hackers and bad hackers.

What I do is poke at something. Maybe I'll try changing an input or altering the environment slightly and see how that changes things. I'll keep doing this until a pattern merges. From trying different things I start forming a hypothesis of what is happening inside the black box. If I try something and the result does not fit in my hypothesis, I form a new hypothesis. At some point I usually get a clear understanding of what is happening in the black box without ever seeing what is in the black box.

I essentially look at the symptoms and narrow down what the cause would be.

This is a good use of trial and error. The goal is not to find the input which gives me the desired output. If I stopped the moment I got the desired output, I might think I have the solution but I don't. Case in point, I input 2 and 2 and get 4. My hypothesis is that the black box does addition. At this point my hypothesis is correct. However, if I poke further I might find that inputing 2 and 3 gives me 6. Now I see that it is not addition. New hypothesis is that it is multiplication.

Hacking is really empirical. Unless I try every possible input, I cannot be certain my hypothesis is correct. For example, I might input 1 to a function and it returns 43, I input 2 and get 47, I input 3 and get 53. After inputting the numbers from 1 to 20 I notice all the numbers are prime! My hypothesis is that the function is a prime number generator. However, if I input 41 I get 1763. This is not prime (43 * 41 = 1763). Turns out the function is Euler's formula for finding prime numbers, i.e. n^2 + n + 41. This has been proven to only produce prime numbers when n is less than 40.

Still hacking can be a good thing. Trail and error to find THE answer is never a good thing.

I see a lot of people solving problems as follows:

1. Program or computer not functioning correctly.
2. Change something.
3. If program or computer not functioning correctly go to step 2.
4. Problem solved.

Now maybe they did find the right solution but most often they don't. Later the problem will come back with different symptoms. If I purchased a program from you and you used this method to solve the problem, here is how I see this as a consumer of software:

My car is running slower than normal. I bring it to my mechanic and he does the following:

1. He changes the spark plugs and charges me for that.
2. Car is still running slow.
3. He adjusts the valve on the carburetor and charges me for that.
4. Car is still running slow.
5. He rotates the tires and charges me for that.
6. Car is still running slow.
7. He changes all the fluids and charges me for that.
8. Car is still running slow.
9. Cars today have a lot of electronics, so he disconnects the battery for a week.
10. All my programming, bluetooth, radio stations, clock, GPS, etc. are gone.
11. The car is no longer running slow.
12. Three months later the car is running slow again.
13. My mechanic disconnects the battery for a week.
14. All my programming, bluetooth, radio stations, clock, GPS, etc. are gone.
15. My car is still running slow.

Would you pay for all the work the mechanic did? I think it is safe to say that NO ONE would put up with this. Some people might put up with it until step 12 then find a new mechanic. Others would put up with this until just step 2 or 4. Most of us would not pay for anything after step 2.

I've worked in industries where EVERYONE programs like this. There might be 4 or 5 different vendors and you really don't have any other choice. However, it just takes one guy to write quality software and everyone switches to that other guy. Trying to win back those customers means you have to make up for all the poor software issues PLUS give them some incentive to switch away from the guy who has always given them good software.