Dealing with things that disappear from the DOM when you look at them

From time to time I encounter a javascript feature which does not let me examine it. For example, auto-complete features on input boxes.

We have all seen this feature if you go to http://www.google.ca. You start typing something and Google gives you suggestions on what you are going to type. For example, I enter "WebD" and Google suggests "WebDriver".

The problem automating some of these is finding the list of suggestions. For example, go to http://www.asp.net/ajaxlibrary/ajaxcontroltoolkitsamplesite/autocomplete/autocomplete.aspx with Chrome, Inspect the auto-complete input box (right click in the input box and select "Inspect") and type something into the input box.

You will see a list of suggestions appear as you type but looking at the DOM in the Inspect window there is no list of suggestions attached to the input element.

They have to exist somewhere in the DOM but you have to find them. So while the suggestion list is open I click on the Inspect window and press CTRL-F to find one of the suggestions.

The problem is the moment I click in the Inspect window, the list of suggestions disappears. So the CTRL-F in the Inspect window finds nothing.

If I type a word in the input box then press CTRL-F (don't change focus to the Inspect window) the auto-complete list disappears. Even if it didn't disappear, it would find the suggestions in the browser window, not in the Inspect window.

So here is how I find the suggestions in the DOM...

WebDriver driver = new ChromeDriver();
driver.get("http://www.asp.net/ajaxlibrary/ajaxcontroltoolkitsamplesite/autocomplete/autocomplete.aspx");
driver.findElements(By.cssSelector("input[name$='myTextBox']")).sendKeys("test");
String src = driver.getPageSource();
driver.quit();

then I set a breakpoint on the driver.quit(); and run in Debug mode. When it stops on the breakpoint I start using the Watch Window of my IDE.

If I enter "test" and one of the suggestions is "testASG" then I'm trying to find the list with "testASG" as one of the suggestions. So I will add:

src.indexOf("testASG");

to the Watch Window. Let's say it returns 34923. So now I know the text it at 34923 and that the element wrapping the suggestion is BEFORE 34923. So I can try:

src.substring(34500);

and examine the results. From this I might see "<li>testASG</li>" and some other stuff before the <li>. This tells me that the list of suggestion is a <ul> element (<ol> would be numbered). So I can start using:

src.indexOf("<ul");

I know it will start with <ul but there would be different, unknown attributes after the ul. When I did this it returned something like 5723. So I looked at:

src.substring(5720);

I see it is finding a "null". Not what I was looking for. So I might start looking for ul but using the index of "testASG". So I might try:

src.substring(33000).indexOf("<ul");

If that returns a number higher than 34923 (after the li) then I need a smaller number. If it gives me a number between 33000 and 34923 then I can look at the substring and see if that was the ul I was looking for.

By narrowing down (binary search) where the ul is for the list of suggestions I can finally see what the DOM looks like by examining the src variable. At some point I will figure out the locators for the ul, li, etc. I can even trying the following locator in the Watch Window:

driver.findElements(By.cssSelector("ul#AutoCompleteEx_completionListElem>li");

To get a list of all the suggestions.