Error Logging, in a Smooth Way

Error logging, in a smooth way

Firstly, I'd like to commend you for looking at the standard error methods within PHP. Unfortunately error_log has some limitations as you found out.

This is a long answer, read on to find out about:

1. Errors
   
   
   • Logging the error directly vs trigger_error and set_error_handler
   • Where good errors go bad - Fatal Errors.
2. Exceptions
   
   
   • SPL
   • What to do with them?
3. Code
   
   
   • Setup
   • Usage

TL;DR Use trigger_error for raising errors and set_error_handler for logging them.

1. Errors

When things don't go as expected in your program, you will often want to raise an error so that someone or something is notified. An error is for a situation where the program may continue, but something noteworthy, possibly harmful or erroneous has occurred. At this point many people want to log the error immediately with their logging package of choice. I believe this is exactly the wrong thing to do. I recommend using trigger_error to raise the error so that it can be handled with a callback set by set_error_handler. Lets compare these options:

Logging the error directly

So, you have chosen your logging package. Now you are ready to spread the calls to your logger wherever an error occurs in your code. Lets look at a single call that you might make (I'll use a similar logger to the one in Jack's answer):

Logger::getLogger('standard')->error('Ouch, this hurts');

What do you need in place to run this code?

    Class:  Logger
    Method: getLogger
    Return: Object with method 'error'

These are the dependencies that are required to use this code. Everyone who wants to re-use this code will have to provide these dependencies. This means that a standard PHP configuration will no longer be sufficient to re-use your code. With the best case, using Dependency Injection you still require a logger object to be passed into all of your code that can emit an error.

Also, in addition to whatever the code is responsible for, it also has responsibility for logging the error. This goes against the Single Responsibility Principle.

We can see that logging the error directly is bad.

trigger_error to the rescue

PHP has a function called trigger_error which can be used to raise an error just like the standard functions do. The error levels that you use with it are defined in the error level constants. As a user you must use one of the user errors: E_USER_ERROR, E_USER_WARNING or the default value E_USER_NOTICE (other error levels are reserved for the standard functions etc.). Using a standard PHP function to raise the error allows the code to be re-used with any standard PHP installation! Our code is no longer responsible for logging the error (only making sure that it is raised).

Using trigger_error we only perform half of the error logging process (raising the error) and save the responsibility of responding to the error for the error handler which will be covered next.

Error Handler

We set a custom error handler with the set_error_handler function (see the code setup). This custom error handler replaces the standard PHP error handler that normally logs messages in the web server error log depending on the PHP configuration settings. We can still use this standard error handler by returning false within our custom error handler.

The custom error handler has a single responsibility: to respond to the error (including any logging that you want to do). Within the custom error handler you have full access to the system and can run any sort of logging that you want. Virtually any logger that uses the Observer design pattern will be ok (I'm not going to go into that as I believe it is of secondary importance). This should allow you to hook in new log observers to send the output to where you need it.

You have complete control to do what you like with the errors in a single maintainable part of your code. The error logging can now be changed quickly and easily from project to project or within a single project from page to page. Interestingly even @ suppressed errors make it to the custom error handler with an errno of 0 which if the error_reporting mask is respected should not be reported.

When Good Errors go Bad - Fatal Errors

It is not possible to continue from certain errors. The following error levels can not be handled from a custom error handler: E_ERROR, E_PARSE, E_CORE_ERROR, E_CORE_WARNING, E_COMPILE_ERROR, E_COMPILE_WARNING. When these sorts of errors are triggered by a standard function call the custom error handler is skipped and the system shuts down. This can be generated by:

call_this_function_that_obviously_does_not_exist_or_was_misspelt();

This is a serious mistake! It is impossible to recover from, and the system is about to shut down. Our only choice is to have a register_shutdown_function deal with the shutdown. However this function is executed whenever a script completes (successful, as well as unsuccessful). Using this and error_get_last some basic information can be logged (the system is almost shutdown at this point) when the last error was a fatal error. It can also be useful to send the correct status code and show an Internal Server Error type page of your choosing.

2. Exceptions

Exceptions can be dealt with in a very similar way to basic errors. Instead of trigger_error an exception will be thrown by your code (manually with throw new Exception or from a standard function call). Use set_exception_handler to define the callback you want to use to handle the exception with.

SPL

The Standard PHP Library (SPL) provides exceptions. They are my preferred way of raising exceptions because like trigger_error they are a standard part of PHP which does not introduce extra dependencies to your code.

What to do with them?

When an exception is thrown there are three choices that can be made:

1. Catch it and fix it (the code then continues as if nothing bad happened).
2. Catch it, append useful information and re-throw it.
3. Let it bubble up to a higher level.

At each level of the stack these choices are made. Eventually once it bubbles up to the highest level the callback you set with set_exception_handler will be executed. This is where your logging code belongs (for the same reasons as the error handling) rather than spread throughout catch statements in your code.

3. Code

Setup

Error Handler

function errorHandler($errno , $errstr, $errfile, $errline, $errcontext)
{
    // Perform your error handling here, respecting error_reporting() and
    // $errno.  This is where you can log the errors.  The choice of logger
    // that you use is based on your preference.  So long as it implements
    // the observer pattern you will be able to easily add logging for any
    // type of output you desire.
}

$previousErrorHandler = set_error_handler('errorHandler');

Exception Handler

function exceptionHandler($e)
{
    // Perform your exception handling here.
}

$previousExceptionHandler = set_exception_handler('exceptionHandler');

Shutdown Function

function shutdownFunction()
{
    $err = error_get_last();

    if (!isset($err))
    {
        return;
    }

    $handledErrorTypes = array(
        E_USER_ERROR      => 'USER ERROR',
        E_ERROR           => 'ERROR',
        E_PARSE           => 'PARSE',
        E_CORE_ERROR      => 'CORE_ERROR',
        E_CORE_WARNING    => 'CORE_WARNING',
        E_COMPILE_ERROR   => 'COMPILE_ERROR',
        E_COMPILE_WARNING => 'COMPILE_WARNING');

    // If our last error wasn't fatal then this must be a normal shutdown.  
    if (!isset($handledErrorTypes[$err['type']]))
    {
        return;
    }

    if (!headers_sent())
    {
        header('HTTP/1.1 500 Internal Server Error');
    }

    // Perform simple logging here.
}

register_shutdown_function('shutdownFunction');

Usage

Errors

// Notices.
trigger_error('Disk space is below 20%.', E_USER_NOTICE);
trigger_error('Disk space is below 20%.'); // Defaults to E_USER_NOTICE

// Warnings.
fopen('BAD_ARGS'); // E_WARNING fopen() expects at least 2 parameters, 1 given
trigger_error('Warning, this mode could be dangerous', E_USER_WARNING);

// Fatal Errors.    
// This function has not been defined and so a fatal error is generated that
// does not reach the custom error handler.
this_function_has_not_been_defined();
// Execution does not reach this point.

// The following will be received by the custom error handler but is fatal.
trigger_error('Error in the code, cannot continue.', E_USER_ERROR);
// Execution does not reach this point.

Exceptions

Each of the three choices from before are listed here in a generic way, fix it, append to it and let it bubble up.

1 Fixable:

try
{
    $value = code_that_can_generate_exception();
}
catch (Exception $e)
{
    // We decide to emit a notice here (a warning could also be used).
    trigger_error('We had to use the default value instead of ' .
                  'code_that_can_generate_exception\'s', E_USER_NOTICE);
    // Fix the exception.
    $value = DEFAULT_VALUE;
}

// Code continues executing happily here.

2 Append:

Observe below how the code_that_can_generate_exception() does not know about $context. The catch block at this level has more information which it can append to the exception if it is useful by rethrowing it.

try
{
    $context = 'foo';
    $value = code_that_can_generate_exception();
}
catch (Exception $e)
{
    // Raise another exception, with extra information and the existing
    // exception set as the previous exception. 
    throw new Exception('Context: ' . $context, 0, $e);
}

3 Let it bubble up:

// Don't catch it.

Manage the errors of a framework

The execution of controller IMHO can generate two exceptions:

• not found: when controller or method is missing
• permission denied: when ACL blocked access

To handle this, i would just go with something like following code. And you can use multiple catch block.

try
{
    $controller->$command($request, $response);
}
catch(AccessDeniedException $e)
{
    $controller = new ErrorController;
    $controller->accessDenied($request, $response);
}
catch(NotFoundException $e)
{
    $controller = new ErrorController;
    $controller->notFound($request, $response);
}

You can let AccessDeniedException to bubble up from Model Layer too, but usually it is a bad practice. Exception should be handles within same level of abstraction, where it was thrown, or, in case of critical exceptions (when object itself is unable to deal with it), the exceptions might penetrate ONE abstraction boundary. And exceptions should NOT leave the Model Layer, instead they should create error state in the layer, and be processed in your current View instance.

The point is this: instead of magical handler for all errors, you should handle errors close to the place where it originated.

Is catch(console.error) a valid way of catching Promise rejections?

Yes, if you just want to log the error and otherwise ignore it, that works.

.catch expects you to pass it in a function, and if the promise rejects, that function will be called with the rejection value. Often you'll create a new function with custom logic and pass that in, but it's fine to pass in a function that already exists, such as console.error.

Is there a way to filter out specific error messages using Django Logging? eg UncompressableFileError

You can set a Filter on the Sentry handler which checks for the type of errors you want to filter out, and return False to drop them. Roughly:

def sentry_filter(record):
    return 'UncompressableFileError' not in record.getMessage()

and then

sentry_handler.addFilter(sentry_filter)

This might need to be tweaked depending on where the string occurs - e.g. in message or traceback

How to log Celebrate validation errors to the console in Express.js?

The simplest way to achieve this seems to be by defining another error middleware before the Celebrate error middleware, that checks whether the error is a Celebrate error (using the isCelebrateError method) and if so it logs it to the console:

// app.js

const { errors, isCelebrateError } = require('celebrate');

...


// middleware to log Celebrate validation errors
app.use((err, req, res, next) => {
    if (isCelebrateError(err)) {
        console.error(err);
    }
    next(err);
});

// Celebrate middleware to return validation errors
app.use(errors());

...

It is important to include the logging middleware before Celebrate's errors() middleware since errors() returns a JSON response and no other middleware is run after it (you can check out the Celebrate source code for the implementation details of errors()).

php session_start general error handling

The regular PHP session functions don't throw exceptions but trigger errors. Try writing an error handler function and setting the error handler before calling session_start.

function session_error_handling_function($code, $msg, $file, $line) {
    // your handling code here
}

set_error_handler('session_error_handling_function');
session_start();
restore_error_handler();

However, this is just for capturing the session errors. A better way would be to create a general error handler that creates exceptions from errors and surround code parts that may throw errors with try ... catch blocks.

php curl multi error handler

I don't think i agree with the with the way you are capturing the error ... you can try

$nodes = array(
        "http://google.com",
        "http://iamdooooooooooown.com",
        "https://gokillyourself.com"
);

echo "<pre>";
print_r(multiplePost($nodes));

Output

Array
(
    [google.com] => #HTTP-OK 48.52 kb returned
    [iamdooooooooooown.com] => #HTTP-ERROR 0  for : http://iamdooooooooooown.com
    [gokillyourself.com] => #HTTP-ERROR 0  for : https://gokillyourself.com
)

Function Used

function multiplePost($nodes) {
    $mh = curl_multi_init();
    $curl_array = array();
    foreach ( $nodes as $i => $url ) {
        $url = trim($url);
        $curl_array[$i] = curl_init($url);
        curl_setopt($curl_array[$i], CURLOPT_RETURNTRANSFER, true);
        curl_setopt($curl_array[$i], CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)');
        curl_setopt($curl_array[$i], CURLOPT_CONNECTTIMEOUT, 5);
        curl_setopt($curl_array[$i], CURLOPT_TIMEOUT, 15);
        curl_setopt($curl_array[$i], CURLOPT_FOLLOWLOCATION, true);
        curl_setopt($curl_array[$i], CURLOPT_SSL_VERIFYHOST, 0);
        curl_setopt($curl_array[$i], CURLOPT_SSL_VERIFYPEER, 0);
        curl_multi_add_handle($mh, $curl_array[$i]);
    }
    $running = NULL;
    do {
        usleep(10000);
        curl_multi_exec($mh, $running);
    } while ( $running > 0 );
    $res = array();

    foreach ( $nodes as $i => $url ) {
        $domain = parse_url($url, PHP_URL_HOST);
        $curlErrorCode = curl_errno($curl_array[$i]);
        if ($curlErrorCode === 0) {
            $info = curl_getinfo($curl_array[$i]);
            $info['url'] = trim($info['url']);
            if ($info['http_code'] == 200) {
                $content = curl_multi_getcontent($curl_array[$i]);
                $res[$domain] = sprintf("#HTTP-OK %0.2f kb returned", strlen($content) / 1024);
            } else {
                $res[$domain] = "#HTTP-ERROR {$info['http_code'] }  for : {$info['url']}";
            }
        } else {
            $res[$domain] = sprintf("#CURL-ERROR %d: %s ", $curlErrorCode, curl_error($curl_array[$i]));
        }
        curl_multi_remove_handle($mh, $curl_array[$i]);
        curl_close($curl_array[$i]);
        flush();
        ob_flush();
    }
    curl_multi_close($mh);
    return $res;
}

How to log uncaught exceptions during Flink job execution

From Flink's perspective, user code errors are expected and, hence, Flink does not log them on WARN or ERROR. WARN and ERROR are reserved for logging statements which indicate that something is wrong with Flink itself.

The best option to capture task failures is to grep for <TASK_NAME> switched from RUNNING to FAILED. That way you will be notified whenever <TASK_NAME> failed. Note, however, that it is not guaranteed that the logging statement will never change.

---

Related Topics