Correct Way to Try/Except Using Python Requests Module

Correct way to try/except using Python requests module?

Have a look at the Requests exception docs. In short:

In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception.

In the event of the rare invalid HTTP response, Requests will raise an HTTPError exception.

If a request times out, a Timeout exception is raised.

If a request exceeds the configured number of maximum redirections, a TooManyRedirects exception is raised.

All exceptions that Requests explicitly raises inherit from requests.exceptions.RequestException.

To answer your question, what you show will not cover all of your bases. You'll only catch connection-related errors, not ones that time out.

What to do when you catch the exception is really up to the design of your script/program. Is it acceptable to exit? Can you go on and try again? If the error is catastrophic and you can't go on, then yes, you may abort your program by raising SystemExit (a nice way to both print an error and call sys.exit).

You can either catch the base-class exception, which will handle all cases:

try:
    r = requests.get(url, params={'s': thing})
except requests.exceptions.RequestException as e:  # This is the correct syntax
    raise SystemExit(e)

Or you can catch them separately and do different things.

try:
    r = requests.get(url, params={'s': thing})
except requests.exceptions.Timeout:
    # Maybe set up for a retry, or continue in a retry loop
except requests.exceptions.TooManyRedirects:
    # Tell the user their URL was bad and try a different one
except requests.exceptions.RequestException as e:
    # catastrophic error. bail.
    raise SystemExit(e)

---

As Christian pointed out:

If you want http errors (e.g. 401 Unauthorized) to raise exceptions, you can call Response.raise_for_status. That will raise an HTTPError, if the response was an http error.

An example:

try:
    r = requests.get('http://www.google.com/nothere')
    r.raise_for_status()
except requests.exceptions.HTTPError as err:
    raise SystemExit(err)

Will print:

404 Client Error: Not Found for url: http://www.google.com/nothere

Python requests exception handling

Assuming you did import requests, you want requests.ConnectionError. ConnectionError is an exception defined by requests. See the API documentation here.

Thus the code should be :

try:
   requests.get('http://www.google.com')
except requests.ConnectionError:
   # handle the exception

Try/except when using Python requests module

If you want the response to raise an exception for a non-200 status code use response.raise_for_status(). Your code would then look like:

testURL = 'http://httpbin.org/status/404'

def return_json(URL):
    response = requests.get(testURL)

    try:
        response.raise_for_status()
    except requests.exceptions.HTTPError as e:
        # Whoops it wasn't a 200
        return "Error: " + str(e)

    # Must have been a 200 status code
    json_obj = response.json()
    return json_obj

You can tell that this is clearly simpler than the other solutions here and doesn't require you to check the status code manually. You would also just catch an HTTPError since that is what raise_for_status will raise. Catching RequestsException is a poor idea. That will catch things like ConnectionErrors or TimeoutErrors, etc. None of those mean the same thing as what you're trying to catch.

Python requests errors out and I don't know how to catch the error

Python exception classes hierarchy is:

• BaseException
  
  • Exception
    
    • OSError
      
      • RequestException *
      • TimeoutError

＊ IOError that is a base class for RequestException have been merged with OSError after python version 3.3:

Changed in version 3.3: EnvironmentError, IOError, WindowsError, socket.error, select.error and mmap.error have been merged into OSError, and the constructor may return a subclass.

So as you can see RequestException is not a parent class of TimeoutError exception and can't be used to catch this type of errors:

import requests

try:
    raise TimeoutError('TIMEOUT ERROR')

except requests.exceptions.RequestException as e:
    # catch RequestException type errors that are specific for request library only
    # do something
    print("RequestExceptions will be caught")
 
except TimeoutError as e:
    # catch TimeoutError type errors that has same level in hierarchy as RequestException errors
    # do something
    print("TimeoutErrors will be caught")

except OSError as e:
    # catch all OSError type errors. Little bit wider than previous exceptions
    # do something
    print("TimeoutErrors or RequestExceptions or any other OSErrors will be caught")

except Exception as e:
    # catch any python errors, because all errors in python are children (sub types) for Exception class. Most wider exception type
    # do something
    print("TimeoutErrors or RequestExceptions or OSErrors or any other python errors will be caught")

How to debug on exceptions inside try except block?

In the breakpoints settings (Either the icon in the debug toolbar, or ctrl+shft+F8), you can set exception breakpoints.

The "Activation Policy" is usually set by default to "On termination". But since you handle the error, there is no termination. To activate the breakpoint immediately, even if the error is handled, you need to set the activation policy to "On raise":

Note: that warning sign which says: "This option may slow down the debugger"

---

Related Topics