Change User-Agent for Selenium Web-Driver

Change user-agent for Selenium web-driver

There is no way in Selenium to read the request or response headers. You could do it by instructing your browser to connect through a proxy that records this kind of information.

Setting the User Agent in Firefox

The usual way to change the user agent for Firefox is to set the variable "general.useragent.override" in your Firefox profile. Note that this is independent from Selenium.

You can direct Selenium to use a profile different from the default one, like this:

from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference("general.useragent.override", "whatever you want")
driver = webdriver.Firefox(profile)

Setting the User Agent in Chrome

With Chrome, what you want to do is use the user-agent command line option. Again, this is not a Selenium thing. You can invoke Chrome at the command line with chrome --user-agent=foo to set the agent to the value foo.

With Selenium you set it like this:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
opts = Options()
opts.add_argument("user-agent=whatever you want")

driver = webdriver.Chrome(chrome_options=opts)

Both methods above were tested and found to work. I don't know about other browsers.

Getting the User Agent

Selenium does not have methods to query the user agent from an instance of WebDriver. Even in the case of Firefox, you cannot discover the default user agent by checking what general.useragent.override would be if not set to a custom value. (This setting does not exist before it is set to some value.)

Once the browser is started, however, you can get the user agent by executing:

agent = driver.execute_script("return navigator.userAgent")

The agent variable will contain the user agent.

How to change the User Agent using Selenium and Python

Your code is just perfect. You simply have to write the line of code to change the user-agent in the next line. As an example:

• Code Block:

  from selenium import webdriver
  
  driver = webdriver.Chrome(executable_path=r'C:\WebDrivers\chromedriver.exe')
  print(driver.execute_script("return navigator.userAgent;"))
  # Setting user agent as Chrome/83.0.4103.97
  driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36'})
  print(driver.execute_script("return navigator.userAgent;"))
  # Setting user agent as Chrome/83.0.4103.53
  driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
  print(driver.execute_script("return navigator.userAgent;"))
  driver.get('https://www.httpbin.org/headers')
  

• Console Output:

  Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36
  Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36
  Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36
  

• Browser Snapshot:

---

Reference

You can find a couple of relevant detailed discussions in:

• How to rotate various user agents using selenium python on each request
• Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection

Way to change Google Chrome user agent in Selenium?

A simple way to use a random User Agent would be using Python's fake_useragent module as follows :

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from fake_useragent import UserAgent

options = Options()
ua = UserAgent()
userAgent = ua.random
print(userAgent)
options.add_argument(f'user-agent={userAgent}')
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\WebDrivers\ChromeDriver\chromedriver_win32\chromedriver.exe')
driver.get("https://www.google.co.in")
driver.quit()

Result of 3 consecutive execution is as follows :

1. First Execution :

   Mozilla/5.0 (Windows NT 4.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36
   

2. Second Execution :

   Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.517 Safari/537.36
   

3. Third Execution :

   Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1309.0 Safari/537.17
   

Change user agent using selenium wire in Chrome

from seleniumwire import webdriver  # Import from seleniumwire

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--user-agent="Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Microsoft; Lumia 640 XL LTE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Mobile Safari/537.36 Edge/12.10166"')
browser = webdriver.Chrome(chrome_options=chrome_options)
user_agent = browser.execute_script("return navigator.userAgent;")
print(str(user_agent))
# Go to the Google home page
browser.get('https://www.google.com')

The same Chrome options that are mentioned in this question will also work here. For the printing of user-agent string see this question.

How to change at runtime the user-agent of a Remote Selenium Driver?

Found a way. Instead of giving as first argument (command_executor) just the url of the Selenium hub, you need to wrap it inside a ChromeRemoteConnection like this:

from selenium.webdriver.chrome.remote_connection import ChromeRemoteConnection

driver = Remote(
    ChromeRemoteConnection(remote_server_addr=url),
    desired_capabilities=DesiredCapabilities.CHROME
)

After this, the user-agent can be changed dinamically; however, instead of calling execute_cdp_cmd (Remote doesn't have this function, resulting in an AttributeError), the inner line of code can be executed safely.

cmd = "Network.setUserAgentOverride"
ua = "My brand new user agent!"
cmd_args = dict(userAgent=ua)

driver.execute("executeCdpCommand", {"cmd": cmd, "params": cmd_args})
assert ua == driver.execute_script("return navigator.userAgent;")

Changing the user agent using selenium webdriver in Java

DesiredCapabilities would help you to change user agent.

You can achieve this by calling these methods:

• setBrowserName(java.lang.String browserName)
• setPlatform(Platform platform)
• setVersion(java.lang.String version)

Or

• static DesiredCapabilities chrome()
• static DesiredCapabilities firefox()
• static DesiredCapabilities iphone()
• ...

More here.

Selenium userAgent

User Agent

A User Agent is a software which retrieves, renders and facilitates end user interaction with the Web content.

Within the UserAgent:

Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.75 Mobile/14E5239e Safari/602.1

• Mozilla/5.0: is the general token that says that the browser is Mozilla-compatible. For historical reasons, almost every browser today sends it.

• Mac OS X: The Chrome (or Chromium/Blink-based engines) user agent string is similar to Firefox. For compatibility, it adds strings like KHTML, like Gecko and Safari. Similarly, older Presto-based Opera releases used Mac OS.

---

Related Topics