Some websites dont fully load/render in selenium headless mode
from selenium import webdriver
from time import sleep
options = webdriver.ChromeOptions()
options.add_argument("--window-size=1920,1080")
options.add_argument("--headless")
options.add_argument("--disable-gpu")
options.add_argument(
"user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36")
browser = webdriver.Chrome(options=options)
some websites uses user-agent to detect whether the browser is in headless mode or not as headless browser uses a different user-agent than normal browser. So explicitly set user agent.Headless browser detection
stop page loading with Selenium Webdriver
https://www.selenium.dev/documentation/en/webdriver/page_loading_strategy/#:~:text=Defines%20the%20current%20session's%20page,loading%20takes%20lot%20of%20time.
chromeOptions.setPageLoadStrategy(PageLoadStrategy.EAGER);
WebDriver driver = new ChromeDriver(chromeOptions);
use page loading strategy Eager to wait only till initial html is loaded , you can also use none , but make sure you have explicit/implicit waits for elements if timing issue comes upIn python its working wierdly only the diseried capability is working
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
caps = DesiredCapabilities().CHROME
# caps["pageLoadStrategy"] = "normal" # Waits for full page load
caps["pageLoadStrategy"] = "none"
options = Options()
driver = webdriver.Chrome(desired_capabilities=caps, options=options)
url = 'https://www.gm-trucks.com/'
driver.get(url)
print(driver.title)
print("hi")
input()
Or :options = Options()
options.set_capability("pageLoadStrategy", "none")
driver = webdriver.Chrome(options=options)
Update
The documentation is updated as per selenium 4.0.0-alpha-7so use above solution or update to selenium v4 for future protection
pip install selenium==4.0.0.a7
Bughttps://github.com/SeleniumHQ/seleniumhq.github.io/issues/627
Headless Chrome with Selenium not loading the web-page elements correctly
On some more research, I discovered adding
chrome_options.add_argument('--lang=en_US')
this code snippet to my program does the trick.Headless chrome does not support all incoming languages and thus some pages do not respond well to that. Supporting the language that page outputs in, loads the page correctly.
Related Topics
Will_Paginate Find Out If I'M on The Last Page
How to Use Reform to Prepopulate for Featured Objects
No Such File to Load - Mechanize
Ruby a Clever Way to Execute a Function on a Condition
How Does The "#Map(&Proc)" Idiom Work When Introspecting Module Classes
How to Test Helpers Blocks in Sinatra, Using Rspec
What Is Ruby's Stringio Class Really
Deleting a Specific Line in a Text File
Why Does Array.To_S Return Brackets
How to Downgrade My Rails Version
Set Ruby 2.0 Keyword Arguments with Attr_Accessor on Initialize
Belongs_To Presence in Rails 5 Not Working
Sorting a Hash in Ruby Based on Value and Then Key
Multi Step Form with Image Uploader
How to Parse Days/Hours/Minutes/Seconds in Ruby