Running Selenium Webdriver with a Proxy in Python

Running Selenium Webdriver with a proxy in Python not changing IP

If you use Firefox browser,

...
# set proxy
firefox_capabilities = webdriver.DesiredCapabilities.FIREFOX
firefox_capabilities['marionette'] = True
firefox_capabilities['proxy'] = {
    'proxyType': "MANUAL",
    'httpProxy': PROXY,
    'ftpProxy': PROXY,
    'sslProxy': PROXY
}
driver = webdriver.Firefox(options=options, executable_path=r'geckodriver.exe', capabilities = firefox_capabilities)

If you use Chrome,

...
chrome_options = Options()
chrome_options.add_argument('--proxy-server=' + PROXY)
chrome_options.add_argument("--headless") 
driver = webdriver.Chrome(executable_path = 'chromedriver.exe', options=chrome_options)

Selenium Python Running a Browser with a Proxy

Try below solution:

from selenium import webdriver

PROXY = "96.70.52.227:48324" #  HOST:PORT

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)
chrome_options.add_argument("ignore-certificate-errors")

chrome = webdriver.Chrome(options=chrome_options)
chrome.get("https://www.ipchicken.com/")

Error running Selenium Webdriver with a proxy in Python

Google Restriction

You are getting an error message because of the proxy,

Google blocks all free proxy's

---

Proxy Spider

I also coded a proxy spider that tests all proxy's on: https://free-proxy-list.net/

Code: https://github.com/xtekky/proxy-spider

---

Setup Own Proxy

Here is an article that tells you how to create your own proxy server.

---

Setup Selenium with proxy

• Try this script:

Github Code: https://github.com/xtekky/selenium-tutorials/tree/main/selenium%20proxy

from selenium import webdriver
from selenium.webdriver.common.proxy import ProxyType, Proxy #proxy module
import time

proxy_ip = 'ip:port' #get a free proxy from the websites in the description

#setting up proxy
proxy =Proxy()
proxy.proxy_type = ProxyType.MANUAL
proxy.http_proxy = proxy_ip
proxy.ssl_proxy = proxy_ip

#linking proxy and setting up driver
capabilities = webdriver.DesiredCapabilities.CHROME
proxy.add_to_capabilities(capabilities)
driver = webdriver.Chrome('CHROMEDRIVER PATH', desired_capabilities=capabilities) # replace the chromedriver path

#loading test page
driver.get('https://httpbin.org/ip')
time.sleep(8)
driver.quit()

How to update the Proxy Server within the same session using Selenium and Python

No, you won't be able to change the proxy server using Selenium after starting the driver and the Browsing Context.

When you configure an instance of a ChromeDriver with ChromeOptions() to span a new Chrome Browsing Context the configuration gets baked within the chromedriver executable which will persist for the lifetime of the WebDriver and being uneditable. So you can't modify/add any existing/new configuration through ChromeOptions() class to the WebDriver instance which is currently in execution.

Even if you are able to extract the ChromeDriver and ChromeSession attributes e.g. Session ID, Cookies, UserAgent and other session attributes from the already initiated ChromeDriver and Chrome Browsing Session still you won't be able to change the set of attributes of the ChromeDriver.

A cleaner way would be to quit() the existing ChromeDriver and Chrome Browser instances gracefully and then span a new set of ChromeDriver and Chrome Browser instance with the new set of proxy configuration as follows:

options = webdriver.ChromeOptions()
options.add_argument('--no-sandbox')
options.add_argument('--disable-setuid-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--window-size=600,400')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--disable-accelerated-2d-canvas')
options.add_argument('--disable-gpu')
options.add_argument('--headless')
urls_to_visit = ['https://www.google.com/', 'https://stackoverflow.com/']
proxies = open("proxy.txt", "r", encoding="utf-8", errors="ignore").readlines()
for i in range(0, len(urls_to_visit)):
    proxy = ((random.choice(proxies)).replace("\n", ""))
    options.add_argument('--proxy-server=%s' % proxy)
    browser = webdriver.Chrome(Path, options=options)
    browser.get("{}".format(urls_to_visit[i]))
    # perform the tasks
    driver.quit()

---

References

You can find a couple of relevant discussions in:

• How to rotate Selenium webrowser IP address

Python Selenium Proxy in Chrome

There's nothing wrong with your code. That proxy is just not available/not working anymore.
Try to find another proxy that a better uptime. Keep it mind that public proxies have a noticeable latency so the page will load pretty slow.

Selenium Python Proxy

The following code works for me in chrome.

PROXY = '91.214.31.234:8080'
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)
chrome = webdriver.Chrome(options=chrome_options)
chrome.get("https://www.icanhazip.com")

---

Related Topics