How to Log Server Errors on Django Sites

How do you log server errors on django sites

Well, when DEBUG = False, Django will automatically mail a full traceback of any error to each person listed in the ADMINS setting, which gets you notifications pretty much for free. If you'd like more fine-grained control, you can write and add to your settings a middleware class which defines a method named process_exception(), which will have access to the exception that was raised:

http://docs.djangoproject.com/en/dev/topics/http/middleware/#process-exception

Your process_exception() method can then perform whatever type of logging you'd like: writing to console, writing to a file, etc., etc.

Edit: though it's a bit less useful, you can also listen for the got_request_exception signal, which will be sent whenever an exception is encountered during request processing:

http://docs.djangoproject.com/en/dev/ref/signals/#got-request-exception

This does not give you access to the exception object, however, so the middleware method is much easier to work with.

Location of Django logs and errors

Logs are set in your settings.py file. A new, default project, looks like this:

# A sample logging configuration. The only tangible logging
# performed by this configuration is to send an email to
# the site admins on every HTTP 500 error when DEBUG=False.
# See http://docs.djangoproject.com/en/dev/topics/logging for
# more details on how to customize your logging configuration.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}

By default, these don't create log files. If you want those, you need to add a filename parameter to your handlers

    'applogfile': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': os.path.join(DJANGO_ROOT, 'APPNAME.log'),
'maxBytes': 1024*1024*15, # 15MB
'backupCount': 10,
},

This will set up a rotating log that can get 15 MB in size and keep 10 historical versions.

In the loggers section from above, you need to add applogfile to the handlers for your application

'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
'APPNAME': {
'handlers': ['applogfile',],
'level': 'DEBUG',
},
}

This example will put your logs in your Django root in a file named APPNAME.log

Log all errors to console or file on Django site

It's a bit extreme, but for debugging purposes, you can turn on the DEBUG_PROPAGATE_EXCEPTIONS setting. This will allow you to set up your own error handling. The easiest way to set up said error handling would be to override sys.excepthook. This will terminate your application, but it will work. There may be things you can do to make this not kill your app, but this will depend on what platform you're deploying this for. At any rate, never use this in production!

For production, you're pretty much going to have to have extensive error handling in place. One technique I've used is something like this:

>>> def log_error(func):
... def _call_func(*args, **argd):
... try:
... func(*args, **argd)
... except:
... print "error" #substitute your own error handling
... return _call_func
...
>>> @log_error
... def foo(a):
... raise AttributeError
...
>>> foo(1)
error

If you use log_error as a decorator on your view, it will automatically handle whatever errors happened within it.

The process_exception function is called for some exceptions (eg: assert(False) in views.py) but process_exception is not getting called for other errors like ImportErrors (eg: import thisclassdoesnotexist in urs.py). I'm new to Django/Python. Is this because of some distinction between run-time and compile-time errors?

In Python, all errors are run-time errors. The reason why this is causing problems is because these errors occur immediately when the module is imported before your view is ever called. The first method I posted will catch errors like these for debugging. You might be able to figure something out for production, but I'd argue that you have worse problems if you're getting ImportErrors in a production app (and you're not doing any dynamic importing).

A tool like pylint can help you eliminate these kinds of problems though.

With DEBUG=False, how can I log django exceptions to a log file

Here is a full working logging configuration. Critical errors are logged to sentry, warnings are sent to admins by emails, normal notice errors are logged to syslog, and debug messages are prompted on the standard output.

LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'formatters': {
'verbose': {
'format': '[contactor] %(levelname)s %(asctime)s %(message)s'
},
},
'handlers': {
# Send all messages to console
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
},
# Send info messages to syslog
'syslog':{
'level':'INFO',
'class': 'logging.handlers.SysLogHandler',
'facility': SysLogHandler.LOG_LOCAL2,
'address': '/dev/log',
'formatter': 'verbose',
},
# Warning messages are sent to admin emails
'mail_admins': {
'level': 'WARNING',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler',
},
# critical errors are logged to sentry
'sentry': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'raven.contrib.django.handlers.SentryHandler',
},
},
'loggers': {
# This is the "catch all" logger
'': {
'handlers': ['console', 'syslog', 'mail_admins', 'sentry'],
'level': 'DEBUG',
'propagate': False,
},
}
}

Django Framework : How to catch all occuring server errors centrally and trigger events on such errors

While writing an app, you can use the following approach:

Whenever your app raises an Exception, use custom, app-specific exceptions like this

class FooException(Exception):
"""Base class for all app specific exceptions"""
pass

classFooPermissionException(FooException):
"""Specific class for permission related exceptions"""
pass

Now your app can provide a custom middleware

class FooExceptionMiddleware(object):
"""App specific middleware to handle app specific errors"""

def process_exception(self, request, e):
"""Processes exceptions"""
if isinstance(e, FooException):

if isinstance(e, FooPermissionException):
# handle PermissionExceptions here...

return None

You can read more about Django middleware here.

Edit:

I think you can use the above approach, to catch all Exceptions: Have a look at Django's exception source code. Django's internal exceptions are siblings of Exception. So with something like the following code, you can possibly catch all exceptions, wherever they are raised in your project:

class AllExceptionMiddleware(object):
def process_exception(self, request, e):
if isinstance(e, Exception):
# do stuff
return None

How to log to file using Django and Gunicorn? Using TimedRotatingFileHandler misses logs

It's probably because Python doesn't support coordination between writing to one file from multiple processes, so stuff could get overwritten. I would use either a SocketHandler or a QueueHandler for all the Django and your webapp modules, and have a separate process listening {either a socket receiver or a QueueListener) which writes to the TimedRotatingFileHandler. The official documentation has a cookbook which shows recipes for the multiple process scenario, including working code for socket receivers etc. which you can adapt to your specific needs. I use Supervisor to keep the listener process running.



Related Topics



Leave a reply



Submit