Selenium Webdriver: Wait for Complex Page with JavaScript to Load

Selenium WebDriver: Wait for complex page with JavaScript to load

If anyone actually knew a general and always-applicable answer, it would have been implemented everywhere ages ago and would make our lives SO much easier.

There are many things you can do, but every single one of them has a problem:

1. As Ashwin Prabhu said, if you know the script well, you can observe its behaviour and track some of its variables on window or document etc. This solution, however, is not for everyone and can be used only by you and only on a limited set of pages.

2. Your solution by observing the HTML code and whether it has or hasn't been changed for some time is not bad (also, there is a method to get the original and not-edited HTML directly by WebDriver), but:

   • It takes a long time to actually assert a page and could prolong the test significantly.
   • You never know what the right interval is. The script might be downloading something big that takes more than 500 ms. There are several scripts on our company's internal page that take several seconds in IE. Your computer may be temporarily short on resources - say that an antivirus will make your CPU work fully, then 500 ms may be too short even for a noncomplex scripts.
   • Some scripts are never done. They call themselves with some delay (setTimeout()) and work again and again and could possibly change the HTML every time they run. Seriously, every "Web 2.0" page does it. Even Stack Overflow. You could overwrite the most common methods used and consider the scripts that use them as completed, but ... you can't be sure.
   • What if the script does something other than changing the HTML? It could do thousands of things, not just some innerHTML fun.

3. There are tools to help you on this. Namely Progress Listeners together with nsIWebProgressListener and some others. The browser support for this, however, is horrible. Firefox began to try to support it from FF4 onwards (still evolving), IE has basic support in IE9.

And I guess I could come up with another flawed solution soon. The fact is - there's no definite answer on when to say "now the page is complete" because of the everlasting scripts doing their work. Pick the one that serves you best, but beware of its shortcomings.

Selenium how to manage wait for page load?

To wait for document.readyState to be complete isn't a full proof approach to ensure presence, visibility or interactibility of an element.

Hence, the function:

JavascriptExecutor js = (JavascriptExecutor) driver.getWebDriver();
String result = js.executeScript("return document.readyState").toString();
    if (!result.equals("complete")) {
     Thread.sleep(1000)
    } 
}

And even waiting for jQuery.active == 0:

public void WaitForAjax2Complete() throws InterruptedException
{
    while (true)
    {
        if ((Boolean) ((JavascriptExecutor)driver).executeScript("return jQuery.active == 0")){
            break;
    }
    Thread.sleep(100);
    }
}

Will be a pure overhead.

You can find a couple of relevant discussions in:

• Selenium IE WebDriver only works while debugging
• Do we have any generic function to check if page has completely loaded in Selenium

---

Solution

The effective approach will be to induce WebDriverWait inconjunction with the ExpectedConditions either for:

• presence of element
• visibility of element
• interactibility of element

You can find a couple of relevant discussions in:

• Selenium: How selenium identifies elements visible or not? Is is possible that it is loaded in DOM but not rendered on UI?
• WebDriverWait not working as expected

---

More than one thread to crawl

WebDriver is not thread-safe. Having said that, if you can serialise access to the underlying driver instance, you can share a reference in more than one thread. This is not advisable. But you can always instantiate one WebDriver instance for each thread.

Ideally the issue of thread-safety isn't in your code but in the actual browser bindings. They all assume there will only be one command at a time (e.g. like a real user). But on the other hand you can always instantiate one WebDriver instance for each thread which will launch multiple browsing tabs/windows. Till this point it seems your program is perfect.

Now, different threads can be run on same Webdriver, but then the results of the tests would not be what you expect. The reason behind is, when you use multi-threading to run different tests on different tabs/windows a little bit of thread safety coding is required or else the actions you will perform like click() or send_keys() will go to the opened tab/window that is currently having the focus regardless of the thread you expect to be running. Which essentially means all the test will run simultaneously on the same tab/window that has focus but not on the intended tab/window.

Wait for page to fully load with webdriverjs

I found that this works for what I needed.

driver.get('http://www.google.com');

driver.wait(function() {
  return driver.executeScript('return document.readyState').then(function(readyState) {
    return readyState === 'complete';
  });
});

// Do stuff after page load here

How to make Selenium WebDriver wait for page to load when new page is loaded via JS event

Explicit waits are what you need;

http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp

You can directly add this to your test or you may want to DRY it up, especially if there is a common wait expectation such as the disappearance of a spinning icon.

You could extend the click method to always wait after clicking or if following page objects, add a wait_until_loaded method to a base page class. There many other valid approaches but dependent on how the AUT is implemented

Selenium wait until document is ready

Try this code:

  driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);

The above code will wait up to 10 seconds for page loading. If the page loading exceeds the time it will throw the TimeoutException. You catch the exception and do your needs. I am not sure whether it quits the page loading after the exception thrown. i didn't try this code yet. Want to just try it.

This is an implicit wait. If you set this once it will have the scope until the Web Driver instance destroy.

See the documentation for WebDriver.Timeouts for more info.

Wait for a page to fully load in Selenium

WebDriverWait inherits methods like wait until.

So something like

webDriverWait.until(ExpectedConditions.visibilityOfElementLocated( elementLocator)

should work. You can use ExpectedConditions, it would make things simpler. You can also use the method visibilityOfAllElements

---

Related Topics