How do you log server errors on django sites
Well, when DEBUG = False
, Django will automatically mail a full traceback of any error to each person listed in the ADMINS
setting, which gets you notifications pretty much for free. If you'd like more fine-grained control, you can write and add to your settings a middleware class which defines a method named process_exception()
, which will have access to the exception that was raised:
http://docs.djangoproject.com/en/dev/topics/http/middleware/#process-exception
Your process_exception()
method can then perform whatever type of logging you'd like: writing to console, writing to a file, etc., etc.
Edit: though it's a bit less useful, you can also listen for the got_request_exception
signal, which will be sent whenever an exception is encountered during request processing:
http://docs.djangoproject.com/en/dev/ref/signals/#got-request-exception
This does not give you access to the exception object, however, so the middleware method is much easier to work with.
Location of Django logs and errors
Logs are set in your settings.py
file. A new, default project, looks like this:
# A sample logging configuration. The only tangible logging
# performed by this configuration is to send an email to
# the site admins on every HTTP 500 error when DEBUG=False.
# See http://docs.djangoproject.com/en/dev/topics/logging for
# more details on how to customize your logging configuration.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
By default, these don't create log files. If you want those, you need to add a filename
parameter to your handlers
'applogfile': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': os.path.join(DJANGO_ROOT, 'APPNAME.log'),
'maxBytes': 1024*1024*15, # 15MB
'backupCount': 10,
},
This will set up a rotating log that can get 15 MB in size and keep 10 historical versions.
In the loggers
section from above, you need to add applogfile
to the handlers
for your application
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
'APPNAME': {
'handlers': ['applogfile',],
'level': 'DEBUG',
},
}
This example will put your logs in your Django root in a file named APPNAME.log
Log all errors to console or file on Django site
It's a bit extreme, but for debugging purposes, you can turn on the DEBUG_PROPAGATE_EXCEPTIONS
setting. This will allow you to set up your own error handling. The easiest way to set up said error handling would be to override sys.excepthook. This will terminate your application, but it will work. There may be things you can do to make this not kill your app, but this will depend on what platform you're deploying this for. At any rate, never use this in production!
For production, you're pretty much going to have to have extensive error handling in place. One technique I've used is something like this:
>>> def log_error(func):
... def _call_func(*args, **argd):
... try:
... func(*args, **argd)
... except:
... print "error" #substitute your own error handling
... return _call_func
...
>>> @log_error
... def foo(a):
... raise AttributeError
...
>>> foo(1)
error
If you use log_error as a decorator on your view, it will automatically handle whatever errors happened within it.
The process
_
exception function is called for some exceptions (eg: assert(False) in views.py) but process_
exception is not getting called for other errors like ImportErrors (eg: import thisclassdoesnotexist in urs.py). I'm new to Django/Python. Is this because of some distinction between run-time and compile-time errors?
In Python, all errors are run-time errors. The reason why this is causing problems is because these errors occur immediately when the module is imported before your view is ever called. The first method I posted will catch errors like these for debugging. You might be able to figure something out for production, but I'd argue that you have worse problems if you're getting ImportErrors in a production app (and you're not doing any dynamic importing).
A tool like pylint can help you eliminate these kinds of problems though.
With DEBUG=False, how can I log django exceptions to a log file
Here is a full working logging configuration. Critical errors are logged to sentry, warnings are sent to admins by emails, normal notice errors are logged to syslog, and debug messages are prompted on the standard output.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'formatters': {
'verbose': {
'format': '[contactor] %(levelname)s %(asctime)s %(message)s'
},
},
'handlers': {
# Send all messages to console
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
},
# Send info messages to syslog
'syslog':{
'level':'INFO',
'class': 'logging.handlers.SysLogHandler',
'facility': SysLogHandler.LOG_LOCAL2,
'address': '/dev/log',
'formatter': 'verbose',
},
# Warning messages are sent to admin emails
'mail_admins': {
'level': 'WARNING',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler',
},
# critical errors are logged to sentry
'sentry': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'raven.contrib.django.handlers.SentryHandler',
},
},
'loggers': {
# This is the "catch all" logger
'': {
'handlers': ['console', 'syslog', 'mail_admins', 'sentry'],
'level': 'DEBUG',
'propagate': False,
},
}
}
Django Framework : How to catch all occuring server errors centrally and trigger events on such errors
While writing an app, you can use the following approach:
Whenever your app raises an Exception, use custom, app-specific exceptions like this
class FooException(Exception):
"""Base class for all app specific exceptions"""
pass
classFooPermissionException(FooException):
"""Specific class for permission related exceptions"""
pass
Now your app can provide a custom middleware
class FooExceptionMiddleware(object):
"""App specific middleware to handle app specific errors"""
def process_exception(self, request, e):
"""Processes exceptions"""
if isinstance(e, FooException):
if isinstance(e, FooPermissionException):
# handle PermissionExceptions here...
return None
You can read more about Django middleware here.
Edit:
I think you can use the above approach, to catch all Exceptions
: Have a look at Django's exception source code. Django's internal exceptions are siblings of Exception
. So with something like the following code, you can possibly catch all exceptions, wherever they are raised in your project:
class AllExceptionMiddleware(object):
def process_exception(self, request, e):
if isinstance(e, Exception):
# do stuff
return None
How to log to file using Django and Gunicorn? Using TimedRotatingFileHandler misses logs
It's probably because Python doesn't support coordination between writing to one file from multiple processes, so stuff could get overwritten. I would use either a SocketHandler
or a QueueHandler
for all the Django and your webapp modules, and have a separate process listening {either a socket receiver or a QueueListener
) which writes to the TimedRotatingFileHandler
. The official documentation has a cookbook which shows recipes for the multiple process scenario, including working code for socket receivers etc. which you can adapt to your specific needs. I use Supervisor to keep the listener process running.
Related Topics
Creating Lowpass Filter in Scipy - Understanding Methods and Units
Why Does Python's Multiprocessing Module Import _Main_ When Starting a New Process on Windows
Pandas Unique Values Multiple Columns
How to Draw Axis in the Middle of the Figure
Accessing CPU Temperature in Python
How to Call an Async Function Without Await
Is There a Python Module to Solve Linear Equations
Convert Bytes to Bits in Python
Do I Need to Import Submodules Directly
Create a Main Loop with Tkinter
Python Matplotlib Framework Under MACosx