Test Web APIs

Testing web APIs essentially breaks down into:

• Building an HTTP Request
• Sending the HTTP Request
• Get the HTTP Response
• Verify the HTTP Response

If you look at RFC 2616 it will tell you everything you need to know about HTTP/1.1. However, if you are not used to reading RFC, this document can be quite confusing.

Here some highlights to an HTTP Request...

If you go to a website and they have a FORM, submitting the FORM will do an HTTP Request. The FORM tag will have an action attribute. In side the FORM will be INPUT, SELECT, BUTTON, etc. tags.

For example, the Google home page is a FORM. The action for the FORM is "/search". The INPUT has a name attribute of "q". If I enter "HTTP Request" into the INPUT field, my web browser builds an HTTP Request and sends it to Google. You can type the following URL into an address bar:

http://www.google.com/search?q=HTTP%20Request&hl=en

This is the same as entering 'HTTP Request' into the input field and clicking the Search button. The one oddity is the %20. A proper HTTP Request has no spaces in it. So you need to convert spaces to a hexidecimal value. The hexidecimal value 20 is ASCII for a space. Also symbols like / : & etc. have to be convert as well. If the symbol is part of the Request and not a value being passed you don't convert it. The general format is:

http://hostname:port/action?key=value&key=value&key=value

If you leave out the port it defaults to 80 (443 for https).

At this point you might be wondering, what does this have to do with API testing? A lot. Many APIs are implemented using HTTP Request/Response. So you need to send them an HTTP Request just like your web browser creates an HTTP Request. With the web browser, it adds a lot more you might not be aware of. The HTTP Request has the URL, a header and sometimes data. Your web browser will secretly add header information like:

Host: www.google.com
Accept-Language: en-us,en;q=0.5

Finally, the FORM can be a POST or a GET form. If it sends data using the URL, like the above examples, it is a GET request. If it sends the data in such a way the user doesn't see it, it is a POST request. For example, if I was logging into a website, I would use a POST so the password isn't visible in the address bar.

So now we have talked about all three parts. The URL, the HEADER and the DATA.

Let's say I design an API such that I expect some data to be in the URL, some in the header and some in the data. So lets say the URL will be:

https://www.mywebsite.com:8443/accounts/admin/requestAccount?company=Acme&auth=Darrell

In the header I want them to identify which machine the request is coming from and an API Key:

Request-Host: 127.0.0.1
Company-API-Key: foobar7

And finally, the data will be an XML file with the following information:

    <?xml version=\"1.0\" encoding=\"utf-8\"?>
    <RequestAccount>
    <CompanyName>Acme</CompanyName>
    <EmployeeID>10318630</EmployeeID>
    <FirstName>Bob</FirstName>
    <LastName>Johnson</LastName>
    </RequestAccount>

So how do I take this knowledge and using it? Let's say the XML data is saved in the text file foo.txt and I have curl installed on my machine (comes default on UNIX, Linux and Mac OS X machines; you can get and install a Windows version for free).

The curl command to use this would be:

curl -k -d @foo.txt -H "Request-Host: 127.0.0.1" -H "Company-API-Key: foobar7" -o response.txt https://www.mywebsite.com:8443/accounts/admin/requestAccount?company=Acme&auth=Darrell

The -k makes it so we don't worry about SSL certificates and authentication, the -d is used to send the data, the -H adds header information, the -o saves the response to the file.

The great thing about curl is you can try something, make a slight change, try again, make a slight change, try again. So if you wanted to try a bunch of different data files, you can do this quickly and easily with a shell for loop. You can play with the header information or leave/change some of the URL fields.

You can quickly probe and test a set of APIs using curl. There are more options to curl but this is the basic usage.

Upgrading Selenium 2.0b2 to 2.0b3 - Python

Selenium 2.0b3 was recently released. My current company is using Selenium 2.0 for web testing and the preferred language for the team is python. If you are familiar with python you might know that upgrading a site-package (like Selenium) is as simple as:

pip install selenium

One of the staff did this and immediately the test suite broke. The line of code which broke was:

host="localhost"
port=4444
browser="firefox"
wd = webdriver.Remote(command_executor='http://' + host + ':' + str(port) + '/wd/hub', browser_name=browser, platform='ANY')

One of the nice things about using Eclipse (my IDE of choice) is you can hold down the CTRL key and click a method to jump to the source code. So I held down the CTRL key (actually it was the Command key because I'm on a Mac) and clicked on Remote. This took me to the class for Remote. The __init__ method is the constructor.

When I looked at the constructor for 2.0b3 it was immediately obvious that the constructor had changed. The old constructor (2.0b2) was:

def __init__(self, command_executor='http://localhost:4444/wd/hub', browser_name='', platform='', version='', javascript_enabled=True)

but the new constructor (2.0b3) is:

def __init__(self, command_executor='http://127.0.0.1:4444/wd/hub', desired_capabilities=None, browser_profile=None)

So the first parameter, command_executor, is the same but all the other parameters have been changed to a desired_capabilities. So what is desired_capabilities? It is a python dictionary. The format for a dictionary is:

dc = { "browser_name" : browser, "platform" : 'ANY' }

This one little change almost fixed things. A little more digging into the source code and I found that the dictionary should be using "browserName" and there doesn't seem to be any support for platform anymore. Since we only care about the browser, I changed the dictionary to:

dc = { "browserName" : browser }

and this solved everything.

Lesson Learned: you have to dig into the source code to figure out what is going on.

P.S. reading the comments in the source code indicated that the start_session method, which uses the desired_capabilities, it still talks about the parameters browser_name and platform. It is clear that the comments have not been updated to reflect the code.

Knowledge sharing

Joining a new company is always difficult at first. You spend a lot of time learning what you don't know about the company and the culture. Here are some ideas to make this normally unproductive time more productive.

Most companies have a wiki or central web site like Sharepoint. Create an employee page with pictures of the employees, there name, contact information (office/location, email, phone, etc.).

Next create a Subject Matter Expert (SME) table. Put the areas of expertise and who is the main and secondary contact. You could even work this from the other direction. As a manager, think about the needs of your department, create a table with one row for each subject your department needs an expert in. In the second column put the name of the person who is the SME for that subject. In the third column put the name of who would be your second choice for SME. Any row which does not have a second name means you need to create a backup for that area. Otherwise, when your one and only SME goes on vacation you could be in trouble. Or worse, they leave the company. Any row which has no SME at all means you need to either get someone to become that SME or you have a case for hiring someone.

Whenever someone new joins the company, take their picture, add an entry to the employee page and a link to the SME table. If they are an SME for a subject you have no one, make them the SME. If they are the SME for a subject you have one SME, make them the backup.

The next step in the process is for the SME to document what they know. Some people will believe documenting what they know will allow the company to get rid of them. The truth is that an SME's practical experience can never really be captured in a document. Most often this knowledge comes into play because the SME has left the company, is away on vacation or ill. If they don't document their knowledge, they will be critical to the company. Going on vacation will be discouraged. They will most likely be put under a great deal of stress. Sooner or later they will fall ill or quit.

The ultimate goal is to create a backup for the SME. If the SME is promoted or put on other activities, the company will require him to be available to help out the backup/replacement. If someone is replacing the SME, the SME will most likely become the backup until a suitable backup can be found. By the time the replacement is fully up to speed and a suitable backup has been found, the SME will probably be an SME for something new or they can at least make sure they are becoming the SME for something new and critical to the company.

Bottom line, if they are trying to phase you out it would be obvious. So don't worry and help your company share that knowledge.