Timeout for Python Requests.Get Entire Response

Timeout for python requests.get entire response

What about using eventlet? If you want to timeout the request after 10 seconds, even if data is being received, this snippet will work for you:

import requests
import eventlet
eventlet.monkey_patch()

with eventlet.Timeout(10):
    requests.get("http://ipv4.download.thinkbroadband.com/1GB.zip", verify=False)

Why doesn't requests.get() return? What is the default timeout that requests.get() uses?

What is the default timeout that get uses?

The default timeout is None, which means it'll wait (hang) until the connection is closed.

Just specify a timeout value, like this:

r = requests.get(
    'http://www.example.com',
    proxies={'http': '222.255.169.74:8080'},
    timeout=5
)

Python requests.get() only responds if adding a time out

I think this had to do with some IPv6 problems - the timeout forced the usage of IPv4? Disabling IPv6 in the router menu did the job.

HTTP requests.post timeout

Use the timeout parameter:

r = requests.post(url, data=payload, timeout=1.5)

Note: timeout is not a time limit on the entire response download;
rather, an exception is raised if the server has not issued a response
for timeout seconds (more precisely, if no bytes have been received on
the underlying socket for timeout seconds). If no timeout is specified
explicitly, requests do not time out.

How to know for sure if requests.get has timed out?

If a response is not received from the server at all within the given time, then an exception requests.exceptions.Timeout is thrown, as per Exa's link from the other answer.

To test if this occurred we can use a try, except block to detect it and act accordingly, rather than just letting our program crash.

Expanding on the demonstration used in the docs:

import requests

try:
    requests.get('https://github.com/', timeout=0.001)
except requests.exceptions.Timeout as e:
    # code to run if we didn't get a reply
    print("Request timed out!\nDetails:", e)
else:
    # code to run if we did get a response, and only if we did.
    print(r.headers)

Just substitute your url and timeout where appropriate.

Timeout within session while sending requests

I'm not sure this is the right way as I could not find the usage of timeout in this documentation.

Scroll to the bottom. It's definitely there. You can search for it in the page by pressing Ctrl+F and entering timeout.

You're using timeout correctly in your code example.

You can actually specify the timeout in a few different ways, as explained in the documentation:

If you specify a single value for the timeout, like this:

r = requests.get('https://github.com', timeout=5)

The timeout value will be applied to both the connect and the read timeouts. Specify a tuple if you would like to set the values separately:

r = requests.get('https://github.com', timeout=(3.05, 27))

If the remote server is very slow, you can tell Requests to wait forever for a response, by passing None as a timeout value and then retrieving a cup of coffee.

r = requests.get('https://github.com', timeout=None)

Try using https://httpstat.us/200?sleep=5000 to test your code.

For example, this raises an exception because 0.2 seconds is not long enough to establish a connection with the server:

import requests

link = "https://httpstat.us/200?sleep=5000"

with requests.Session() as s:
    try:
        r = s.get(link, timeout=(0.2, 10))
        print(r.text)
    except requests.exceptions.Timeout as e:
        print(e)

Output:

HTTPSConnectionPool(host='httpstat.us', port=443): Read timed out. (read timeout=0.2)

This raises an exception because the server waits for 5 seconds before sending the response, which is longer than the 2 second read timeout set:

import requests

link = "https://httpstat.us/200?sleep=5000"

with requests.Session() as s:
    try:
        r = s.get(link, timeout=(3.05, 2))
        print(r.text)
    except requests.exceptions.Timeout as e:
        print(e)

Output:

HTTPSConnectionPool(host='httpstat.us', port=443): Read timed out. (read timeout=2)

You specifically mention using a timeout within a session. So maybe you want a session object which has a default timeout. Something like this:

import requests

link = "https://httpstat.us/200?sleep=5000"

class EnhancedSession(requests.Session):
    def __init__(self, timeout=(3.05, 4)):
        self.timeout = timeout
        return super().__init__()

    def request(self, method, url, **kwargs):
        print("EnhancedSession request")
        if "timeout" not in kwargs:
            kwargs["timeout"] = self.timeout
        return super().request(method, url, **kwargs)

session = EnhancedSession()

try:
    response = session.get(link)
    print(response)
except requests.exceptions.Timeout as e:
    print(e)

try:
    response = session.get(link, timeout=1)
    print(response)
except requests.exceptions.Timeout as e:
    print(e)

try:
    response = session.get(link, timeout=10)
    print(response)
except requests.exceptions.Timeout as e:
    print(e)

Output:

EnhancedSession request
HTTPSConnectionPool(host='httpstat.us', port=443): Read timed out. (read timeout=4)
EnhancedSession request
HTTPSConnectionPool(host='httpstat.us', port=443): Read timed out. (read timeout=1)
EnhancedSession request
<Response [200]>

python requests timeout doesn't timeout

From the requests docs:

timeout is not a time limit on the entire response download; rather, an exception is raised if the server has not issued a response for timeout seconds (more precisely, if no bytes have been received on the underlying socket for timeout seconds). If no timeout is specified explicitly, requests do not time out.

The time you are printing is the total time of the process, but the timeout parameter concerns the time elapsed for the underlying socket in which no bytes were recieved as explained above.

---

Related Topics