Python: Requests.Exceptions.Connectionerror. Max Retries Exceeded With Url

Max retries exceeded with url in Requests Python

Add headers.

Disguise the browser.

import requests

headers = {
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36'}

url_1 = 'http://www.dsit.org.ir/?cmd=page&Cid=92&title=Kontakt&lang=fa'

print(requests.get(url=url_1, headers=headers).text)

flask application brings error requests.exceptions.ConnectionError requests.exceptions.ConnectionError Max retries exceeded with url: /chain

This could happen for various reasons. Maybe the port you are trying to run flask is being used from another application. Try the structure below to change the port that the flask is running.

from flask import Flask

app = Flask(__name__)

@app.route('/')
def index():
    return 'something'

app.run(host='127.0.0.1', port=5000)

Try to use different ports like 5000 or 8080.

Python: requests.exceptions.ConnectionError. Max retries exceeded with url

Looking at stack trace you've provided your error is caused by httplib.BadStatusLine exception, which, according to docs, is:

Raised if a server responds with a HTTP status code that we don’t understand.

In other words something that is returned (if returned at all) by proxy server cannot be parsed by httplib that does actual request.

From my experience with (writing) http proxies I can say that some implementations may not follow specs too strictly (rfc specs on http aren't easy reading actually) or use hacks to fix old browsers that have flaws in their implementation.

So, answering this:

Could it be a bad proxy?

... I'd say - that this is possible. The only real way to be sure is to see what is returned by proxy server.

Try to debug it with debugger or grab packet sniffer (something like Wireshark or Network Monitor) to analyze what happens in the network. Having info about what exactly is returned by proxy server should give you a key to solve this issue.

urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=59587): Max retries exceeded with url using Selenium GeckoDriver Firefox

This error message...

MaxRetryError(_pool, url, error or ResponseError(cause))urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=59587): Max retries exceeded with url: /session/b38be2fe-6d92-464f-a096-c43183aef6a8/element (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000173145EF520>: Failed to establish a new connection: [WinError 10061] No connections could be made because the target machine actively refused them'))

...implies that the GeckoDriver was unable to initiate/spawn a new Browsing Context i.e. firefox session.

Root cause

The root cause of this error can be either of the following:

This error may surface if have closed the Browsing Context manually with brute force when the driver have already initiated a lookout for element/elements.
There is a possibility that the application you are trying to access is throttling the requests from your system/machine/ip-address/network.
There is also a possibility that the application have identified the Selenium driven GeckoDriver initiated firefox Browsing Context as a bot and is denying any access.

Solution

Ensure that:

To evade the detection as a bot, pass the argument --disable-blink-features=AutomationControlled as follows:

from selenium.webdriver.firefox.options import Options

options = Options()
options.add_argument('--disable-blink-features=AutomationControlled')

Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.
Induce WebDriverWait to synchronize the fast moving WebDriver along with the Browsing Context.