Log4J Does Not Recreate Files on Deletion

Log4j not recreate log file after deleting it

Is there any way to delete the log file (not from the app), create a new one in the same path with the same name, and that it can be written by the application?

Nope. You need to get the application itself to restart logging.

The problem is that the log4j appender still has a handle for the deleted file, and will continue to write to it ... unaware that it has been deleted.

A better approach would be to have the application itself take care of "rotating" the logfile. Look at the classes that implement the log4j Appender interface for some ideas.

Does anybody know how to configure or extend lo4j2 to recreate logs after deletion?

I'm finally find the solution. Thanx @Alexander in comments for tip.

Short: We can manually initialize rollover process when detect file deletition.

Longer:
I implement it this way:

1) Create FileWatchService which will (1) subscribe for the log-file deletiiton events in your log folder and (2) notify you when these events will occur. It can be done by java.nio.file.WatchService (https://docs.oracle.com/javase/tutorial/essential/io/notification.html). I will provide my code below.

2) Create some other class which will initialize rollover when FileWatchService will notify about file deletition. I'm also will provide my full code below, but main magic will be occur this way:

final LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
// You should know only appender name.
RollingFileAppender appender = (RollingFileAppender) ctx.getConfiguration().getAppenders().get(appenderName);
if (appender != null) {
  // Manually start rollover logic.
  appender.getManager().rollover();
}

My code looks like this (not ideal but it's working for me):

FileWatchService:

public class FileWatchService implements Runnable {
    private final org.apache.logging.log4j.Logger logger = LogManager.getLogger(FileWatchService.class);
    private WatchService watchService = null;
    private Map<WatchKey,Path> keys = null;
    private String tempPath;

    public FileWatchService(String tempPath) {
        try {
            this.watchService = FileSystems.getDefault().newWatchService();
            this.keys = new HashMap<WatchKey,Path>();
            this.tempPath = tempPath;
            Path path = Paths.get(tempPath);
            register(path);
            logger.info("Watch service has been initiated.");
        }
        catch (Exception e) {
            logger.error("The error occurred in process of registering watch service", e);
        }
    }

    // Method which register folder to watch service.
    private void register(Path tempPath) throws IOException {
        logger.debug("Registering folder {} for watching.", tempPath.getFileName());
        // Registering only for delete events.
        WatchKey key = tempPath.register(watchService, ENTRY_DELETE);
        keys.put(key, tempPath);
    }

    @Override
    public void run() {
        try {
            Thread.currentThread().setName("FileWatchService");
            this.processEvents();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

    private void processEvents() throws InterruptedException {
            WatchKey key;

            // Waiting until event occur.
            while ((key = watchService.take()) != null) {
                // Poll all events when event occur.
                for (WatchEvent<?> event : key.pollEvents()) {
                    // Getting type of event - delete, modify or create.
                    WatchEvent.Kind kind = event.kind();

                    // We are interested only for delete events.
                    if (kind == ENTRY_DELETE) {
                        // Sending "notification" to appender watcher service.
                        logger.debug("Received event about file deletion. File: {}", event.context());
                        AppenderWatcher.hadleLogFileDeletionEvent(this.tempPath + event.context());
                    }
                }
                key.reset();
            }
    }
}

Another class for initilize rollover (I have called it AppenderWatcher):

public class AppenderWatcher {
    private static final org.apache.logging.log4j.Logger logger = LogManager.getLogger(AppenderWatcher.class);

    public static void hadleLogFileDeletionEvent(String logFile) {
        File file = new File(logFile);
        if (!checkFileExist(file)) {
            logger.info("File {} is not exist. Starting manual rollover...", file.toString());
            // Getting possible appender name by log-file.
            String appenderName = getAppenderNameByFileName(logFile);
            // Getting appender from list of all appender
            RollingFileAppender appender = (RollingFileAppender) getAppender(appenderName);

            if (appender != null) {
                // Manually start rollover logic.
                appender.getManager().rollover();
                logger.info("Rollover finished");
            }
            else {
                logger.error("Can't get appender {}. Please, check lo4j2 config.", appenderName);
            }

        } else {
            logger.warn("Received notification what file {} was deleted, but it exist.", file.getAbsolutePath());
        }
    }

    // Method which checks is file exist. It need to prevent conflicts with Log4J rolling file logic.
    // When Log4j rotate file it deletes it first and create after.
    private static boolean checkFileExist(File logFile) {
        return logFile.exists();
    }

    // Method which gets appender by name from list of all configured appenders.
    private static Appender getAppender(String appenderName) {
        final LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
        return ctx.getConfiguration().getAppenders().get(appenderName);
    }

    // Method which returns name of appender by log file name.
    // ===Here I'm explaining some customer specific moments of log4j config.
    private static String getAppenderNameByFileName(String fileName) {
        return getLoggerNameByFileName(fileName) + "_LOG";
    }

    // This method fully customer specific. 
    private static String getLoggerNameByFileName(String fileName) {
        // File name looks like "../log/temp/uber.log" (example).
        String[] parts = fileName.split("/");

        // Last part should look like "uber.log"
        String lastPart = parts[parts.length - 1];

        // We need only "uber" part.
        String componentName = lastPart.substring(0, lastPart.indexOf("."));
        return componentName.toUpperCase();
    }
}

Log4j FileAppender recreating deleted files

I study the source of log4j and find log4j can't create new log file, it just print the error message to system.err when the log file was deleted

    /** 
     This method determines if there is a sense in attempting to append. 

     <p>It checks whether there is a set output target and also if 
     there is a set layout. If these checks fail, then the boolean 
     value <code>false</code> is returned. */  

  protected   boolean checkEntryConditions() {  
    if(this.closed) {  
      LogLog.warn("Not allowed to write to a closed appender.");  
      return false;  
    }  

    if(this.qw == null) {  
      errorHandler.error("No output stream or file set for the appender named ["+  
            name+"].");  
      return false;  
    }  

    if(this.layout == null) {  
      errorHandler.error("No layout set for the appender named ["+ name+"].");  
      return false;  
    }  
    return true;  
  }  

I think there are two workaround

1. create another cron thread to monitor the log file
2. add judge in getLog or getInstance (singleton), check the log file does exist, if not then init log4j

Configure log4j to Auto create application log files when deleted

I think you are using log4j in web application, so maybe the method below may be helpful:

write a jsp file to reconfigure the log4j. put it in webapp floder. just like this:

reset.jsp

<%
    java.util.Properties pro = new java.util.Properties();
    pro.put("log4j.appender.A1.Threshold","INFO");
    pro.put("log4j.appender.A1","org.apache.log4j.RollingFileAppender");
    pro.put("log4j.appender.A1.File","/var/log/h3cloud.log");
    pro.put("log4j.appender.A1.MaxFileSize","100000KB");
    pro.put("log4j.appender.A1.MaxBackupIndex","5");

/* all the log4j.properties content.. */

    org.apache.log4j.LogManager.resetConfiguration();
    org.apache.log4j.PropertyConfigurator.configure(pro);
%>

After I deleted the "/var/log/tomcat6/h3cloud.log", I open http://*/reset.jsp, the new "/var/log/h3coud.log" is recreated.

How to get log file deletion to work with spring-boot-starter-log4j2?

I have a small application to try using Delete action. So far it is working for me.

Please see the example at https://github.com/bigzidane/spring-boot-delete-log. Check the READMe.md to see how the previous logs got deleted.

Please enable TRACE in your log4j configuration (<Configuration status="TRACE" monitorInterval="30">), it will help you detect why the Delete action is not working for you by just following logs which you see in my README.md file.

It may come from the path you are settings are wrong (Maybe I'm not correct here) but with TRACE option I believe you can find out information by yourself.

Logs with Trace option example

2019-05-25 09:38:33.756 INFO WINDOWS-ESDA5FC --- [ main] c.e.SpringBootDeleteLogApp : Starting SpringBootDeleteLogApp on WINDOWS-ESDA5FC with PID 17236 (C:\Users\dotha\IdeaProjects\spring-boot-delete-log\target\classes started by dotha in C:\Users\dotha\IdeaProjects\spring-boot-delete-log)

2019-05-25 09:38:33.760 INFO WINDOWS-ESDA5FC --- [ main] c.e.SpringBootDeleteLogApp : No active profile set, falling back to default profiles: default

2019-05-25 09:38:33.798 INFO WINDOWS-ESDA5FC --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@338fc1d8: startup date [Sat May 25 09:38:33 CDT 2019]; root of context hierarchy

2019-05-25 09:38:33,802 main TRACE DefaultRolloverStrategy.purge() took 3.0 milliseconds

2019-05-25 09:38:33,804 main DEBUG RollingFileManager executing synchronous FileRenameAction[logs\log4j2-demo.log to logs\log4j2-demo-2019-05-25-2.log, renameEmptyFiles=false]

2019-05-25 09:38:33,805 main TRACE Renamed file C:\Users\dotha\IdeaProjects\spring-boot-delete-log\logs\log4j2-demo.log to C:\Users\dotha\IdeaProjects\spring-boot-delete-log\logs\log4j2-demo-2019-05-25-2.log with Files.move

2019-05-25 09:38:33,806 main DEBUG RollingFileManager executing async CompositeAction[DeleteAction[basePath=logs, options=[], maxDepth=1, conditions=[IfFileName(glob:log4j2-demo-*.log), IfLastModified(age=PT1M)]]]

2019-05-25 09:38:33,806 main DEBUG Now writing to logs/log4j2-demo.log at 2019-05-25T09:38:33.806-0500

2019-05-25 09:38:33,807 Log4j2-TF-2-RollingFileManager-3 DEBUG Starting DeleteAction[basePath=logs, options=[], maxDepth=1, conditions=[IfFileName(glob:log4j2-demo-*.log), IfLastModified(age=PT1M)]]

2019-05-25 09:38:33,809 Log4j2-TF-2-RollingFileManager-3 DEBUG DeleteAction complete in 0.001881032 seconds

2019-05-25 09:38:33,810 Log4j2-TF-2-RollingFileManager-3 TRACE Sorted paths:

2019-05-25 09:38:33,810 Log4j2-TF-2-RollingFileManager-3 TRACE logs\log4j2-demo.log (modified: 2019-05-25T14:38:33.807885Z)

2019-05-25 09:38:33,812 Log4j2-TF-2-RollingFileManager-3 TRACE logs\log4j2-demo-2019-05-25-2.log (modified: 2019-05-25T14:38:33.803895Z)

2019-05-25 09:38:33,812 Log4j2-TF-2-RollingFileManager-3 TRACE logs\log4j2-demo-2019-05-25-1.log (modified: 2019-05-25T14:36:13.862034Z)

2019-05-25 09:38:33,812 Log4j2-TF-2-RollingFileManager-3 TRACE IfFileName REJECTED: 'glob:log4j2-demo-*.log' does not match relative path 'log4j2-demo.log'

2019-05-25 09:38:33,812 Log4j2-TF-2-RollingFileManager-3 TRACE Not deleting base=logs, relative=log4j2-demo.log

2019-05-25 09:38:33,812 Log4j2-TF-2-RollingFileManager-3 TRACE IfFileName ACCEPTED: 'glob:log4j2-demo-*.log' matches relative path 'log4j2-demo-2019-05-25-2.log'

2019-05-25 09:38:33,813 Log4j2-TF-2-RollingFileManager-3 TRACE IfLastModified REJECTED: log4j2-demo-2019-05-25-2.log ageMillis '9' < 'PT1M'

2019-05-25 09:38:33,813 Log4j2-TF-2-RollingFileManager-3 TRACE Not deleting base=logs, relative=log4j2-demo-2019-05-25-2.log

2019-05-25 09:38:33,813 Log4j2-TF-2-RollingFileManager-3 TRACE IfFileName ACCEPTED: 'glob:log4j2-demo-*.log' matches relative path 'log4j2-demo-2019-05-25-1.log'

2019-05-25 09:38:33,813 Log4j2-TF-2-RollingFileManager-3 TRACE IfLastModified ACCEPTED: log4j2-demo-2019-05-25-1.log ageMillis '139951' >= 'PT1M'

**2019-05-25 09:38:33,813 Log4j2-TF-2-RollingFileManager-3 TRACE Deleting logs\log4j2-demo-2019-05-25-1.log**

Good luck,

How to configure log4j to only keep log files for the last seven days?

You can perform your housekeeping in a separate script which can be cronned to run daily. Something like this:

find /path/to/logs -type f -mtime +7 -exec rm -f {} \;

---

Related Topics