Python Selenium Wait for Several Elements to Load

Python - Selenium - implicit wait for multiple elements

The problem here is that it is simpler for a single item. It just has to wait until your locator returns a single, or more, elements.

When you deal with multiple elements, WebDriver cannot possibly know how long to wait because it has no idea about how many elements you expect to be there.

So you'll have to use an explicit wait instead.

In this explicit wait, you should:

Run find_elements_by_path
Check the result from the step is a collection that contains the amount of elements you need. If this isn't equal to the number you expect, you can let the "waiter" fail fast and go round again.
If the above is true, you can exit your "waiting", otherwise, let the "waiter" go round again.

http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp

How to get selenium to wait on multiple elements to load

Rather than trying to combine them all into a single wait, you can have a separate wait for each.

...

try:
    wait = WebDriverWait(browser, browser_delay)
    wait.until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, CSSSelector1_toWaitOn))
    wait.until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, CSSSelector2_toWaitOn))
    wait.until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, CSSSelector3_toWaitOn))
    wait.until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, CSSSelector4_toWaitOn))))
except TimeoutException:
    print("Selenium timeout")

Just be aware, there are 3 levels of interaction within Selenium for elements:

present - the element is in the DOM. If you attempt to click on or get text from, etc. a present (but not visble) element, an ElementNotInteractable exception will be thrown.
visible - the element is in the DOM and visible (e.g. not invisible, display: none, etc.)
clickable - the element is visible and enabled. For most cases, this is just... is it visible? The special cases would be elements like an INPUT button that is marked as disabled. An element that is styled as disabled (greyed out) using CSS, is not considered disabled.

Python Selenium wait for several elements to load

Keeping in mind comments of Mr.E. and Arran I made my list traversal fully on CSS selectors. The tricky part was about my own list structure and marks (changing classes, etc.), as well as about creating required selectors on the fly and keeping them in memory during traversal.

I disposed waiting for several elements by searching for anything that is not loading state. You may use ":nth-child" selector as well like here:

#in for loop with enumerate for i    
selector.append(' > li:nth-child(%i)' % (i + 1))  # identify child <li> by its order pos

This is my hard-commented code solution for example:

def parse_crippled_shifted_list(driver, frame, selector, level=1, parent_id=0, path=None):
    """
    Traversal of html list of special structure (you can't know if element has sub list unless you enter it).
    Supports start from remembered list element.

    Nested lists have classes "closed" and "last closed" when closed and "open" and "last open" when opened (on <li>).
    Elements themselves have classes "leaf" and "last leaf" in both cases.
    Nested lists situate in <li> element as <ul> list. Each <ul> appears after clicking <a> in each <li>.
    If you click <a> of leaf, page in another frame will load.

    driver - WebDriver; frame - frame of the list; selector - selector to current list (<ul>);
    level - level of depth, just for console output formatting, parent_id - id of parent category (in DB),
    path - remained path in categories (ORM objects) to target category to start with.
    """

    # Add current level list elements
    # This method selects all but loading. Just what is needed to exclude.
    selector.append(' > li > a:not([class=loading])')

    # Wait for child list to load
    try:
        query = WebDriverWait(driver, WAIT_LONG_TIME).until(
            EC.presence_of_all_elements_located((By.CSS_SELECTOR, ''.join(selector))))

    except TimeoutException:
        print "%s timed out" % ''.join(selector)

    else:
        # List is loaded
        del selector[-1]  # selector correction: delete last part aimed to get loaded content
        selector.append(' > li')

        children = driver.find_elements_by_css_selector(''.join(selector))  # fetch list elements

        # Walk the whole list
        for i, child in enumerate(children):

            del selector[-1]  # delete non-unique li tag selector
            if selector[-1] != ' > ul' and selector[-1] != 'ul.ltr':
                del selector[-1]

            selector.append(' > li:nth-child(%i)' % (i + 1))  # identify child <li> by its order pos
            selector.append(' > a')  # add 'li > a' reference to click

            child_link = driver.find_element_by_css_selector(''.join(selector))

            # If we parse freely further (no need to start from remembered position)
            if not path:
                # Open child
                try:
                    double_click(driver, child_link)
                except InvalidElementStateException:
                        print "\n\nERROR\n", InvalidElementStateException.message(), '\n\n'
                else:
                    # Determine its type
                    del selector[-1]  # delete changed and already useless link reference
                    # If <li> is category, it would have <ul> as child now and class="open"
                    # Check by class is priority, because <li> exists for sure.
                    current_li = driver.find_element_by_css_selector(''.join(selector))

                    # Category case - BRANCH
                    if current_li.get_attribute('class') == 'open' or current_li.get_attribute('class') == 'last open':
                        new_parent_id = process_category_case(child_link, parent_id, level)  # add category to DB
                        selector.append(' > ul')  # forward to nested list
                        # Wait for nested list to load
                        try:
                            query = WebDriverWait(driver, WAIT_LONG_TIME).until(
                                EC.presence_of_all_elements_located((By.CSS_SELECTOR, ''.join(selector))))

                        except TimeoutException:
                            print "\t" * level,  "%s timed out (%i secs). Failed to load nested list." %\
                                                 ''.join(selector), WAIT_LONG_TIME
                        # Parse nested list
                        else:
                            parse_crippled_shifted_list(driver, frame, selector, level + 1, new_parent_id)

                    # Page case - LEAF
                    elif current_li.get_attribute('class') == 'leaf' or current_li.get_attribute('class') == 'last leaf':
                        process_page_case(driver, child_link, level)
                    else:
                        raise Exception('Damn! Alien class: %s' % current_li.get_attribute('class'))

            # If it's required to continue from specified category
            else:
                # Check if it's required category
                if child_link.text == path[0].name:
                    # Open required category
                    try:
                        double_click(driver, child_link)

                    except InvalidElementStateException:
                            print "\n\nERROR\n", InvalidElementStateException.msg, '\n\n'

                    else:
                        # This element of list must be always category (have nested list)
                        del selector[-1]  # delete changed and already useless link reference
                        # If <li> is category, it would have <ul> as child now and class="open"
                        # Check by class is priority, because <li> exists for sure.
                        current_li = driver.find_element_by_css_selector(''.join(selector))

                        # Category case - BRANCH
                        if current_li.get_attribute('class') == 'open' or current_li.get_attribute('class') == 'last open':
                            selector.append(' > ul')  # forward to nested list
                            # Wait for nested list to load
                            try:
                                query = WebDriverWait(driver, WAIT_LONG_TIME).until(
                                    EC.presence_of_all_elements_located((By.CSS_SELECTOR, ''.join(selector))))

                            except TimeoutException:
                                print "\t" * level, "%s timed out (%i secs). Failed to load nested list." %\
                                                     ''.join(selector), WAIT_LONG_TIME
                            # Process this nested list
                            else:
                                last = path.pop(0)
                                if len(path) > 0:  # If more to parse
                                    print "\t" * level, "Going deeper to: %s" % ''.join(selector)
                                    parse_crippled_shifted_list(driver, frame, selector, level + 1,
                                                                parent_id=last.id, path=path)
                                else:  # Current is required
                                    print "\t" * level,  "Returning target category: ", ''.join(selector)
                                    path = None
                                    parse_crippled_shifted_list(driver, frame, selector, level + 1, last.id, path=None)

                        # Page case - LEAF
                        elif current_li.get_attribute('class') == 'leaf':
                            pass
                else:
                    print "dummy"

        del selector[-2:]

What is the command in selenium python that says wait for all elements on page to load?

webdriver will wait for a page to load by default.

It does not wait for loading inside frames or for ajax requests. It means when you use .get('url'), your browser will wait until the page is completely loaded and then go to the next command in the code. But when you are posting an ajax request, webdriver does not wait and it's your responsibility to wait an appropriate amount of time for the page or a part of page to load; so there is a module named expected_conditions.

Wait until page is loaded with Selenium WebDriver for Python

The webdriver will wait for a page to load by default via .get() method.

As you may be looking for some specific element as @user227215 said, you should use WebDriverWait to wait for an element located in your page:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException

browser = webdriver.Firefox()
browser.get("url")
delay = 3 # seconds
try:
    myElem = WebDriverWait(browser, delay).until(EC.presence_of_element_located((By.ID, 'IdOfMyElement')))
    print "Page is ready!"
except TimeoutException:
    print "Loading took too much time!"

I have used it for checking alerts. You can use any other type methods to find the locator.

EDIT 1:

I should mention that the webdriver will wait for a page to load by default. It does not wait for loading inside frames or for ajax requests. It means when you use .get('url'), your browser will wait until the page is completely loaded and then go to the next command in the code. But when you are posting an ajax request, webdriver does not wait and it's your responsibility to wait an appropriate amount of time for the page or a part of page to load; so there is a module named expected_conditions.

Python selenium : Explicitly wait for one of two elements to be loaded

find_element raises NoSuchElementException exception if no element is found.

If element with the id a does not exist, driver.find_element(By.ID,"a") will raises the exception and the driver.find_element(By.ID,"b") will not be executed.

A simple way to solve the problem is using find_elements which return empty list instead of raising the exception:

WebDriverWait(driver,5).until(
    lambda driver: driver.find_elements(By.ID,"a") or driver.find_elements(By.ID,"b"))

Selenium wait for all class elements

WebDriverWait inconjunction with the expected_conditions as presence_of_element_located() will wait for the very first matched WebElement.

To wait until all the elements of the same class e.g. foo class are present, instead of presence_of_element_located() you need to induce WebDriverWait for the presence_of_all_elements_located() and your effective code block will be:

WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.CLASS_NAME, 'foo')))

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC