Why does zookeeper not use my log4j.properties file log directory
I wanted to add how I fixed this problem / customized my environment.
There are 2 logging mechanisms working here:
- bin/zkServer.sh redirects the zookeeper server's stdout and stderr to zookeeper.out
- log4j can append to logs to several places including:
- CONSOLE - which ends up in zookeeper server's stdout and stderr
- ROLLINGFILE - which is sent to zookeeper.log
bin/zkServer.sh uses :
- ZOO_LOG_DIR to set the path for both zookeeper.out and log4j.
- ZOO_LOG4J_PROP to set the log4j logging level and what log appenders are turned on
The "eventual" defaults in the setup conf/log4j.properties are set by a combination of zookeeper bash scripts:
- ZOO_LOG_DIR = . ( the working directory from which zookeeper is started )
- set inside of conf/log4j.properties as zookeeper.log.dir
- ZOO_LOG4J_PROP = INFO, CONSOLE
- set inside of conf/log4j.properties as zookeeper.root.logger
The effect of turning on the log appender CONSOLE is that logs now go to stdout. Because bin/zkServer.sh redirects stdout and stderr to zookeeper.out, log4j logs end up in zookeeper.out. The effect of turning off ROLLINGFILE is that no zookeeper.log file is created.
The zookeeper.out log is not rotated. The zookeeper.log log is set to be rotated and can be set to expire old logs.
I wanted logs to roll and be expired. The first thing I had to do was change conf/log4j.properties to cause the expiration/deletion of old logs. I did that by setting log4j.appender.ROLLINGFILE.MaxBackupIndex inside of conf/log4j.properties. The second thing I had to do was set the log directory, logging level and appenders.
I have a bash script that runs every minute. If it sees that zookeeper isn't running, it runs :
bin/zkServer.sh start
I changed it to specify environmental variables expected by bin/zkServer.sh.
sudo ZOO_LOG_DIR=/opt/zookeeper-3.4.6/logs ZOO_LOG4J_PROP='INFO,ROLLINGFILE' /opt/zookeeper-3.4.6/bin/zkServer.sh start
The effect of turning off the log appender CONSOLE is that log4j logs now no longer end up in zookeeper.out. The effect of turning on ROLLINGFILE is that zookeeper.log file is created, rotated, and expired.
BTW, conf/log4j.properties was apparently already in my classpath. I had to make no changes in that regard.
This chain contributed significantly to my understanding:
https://groups.google.com/forum/#!msg/nosql-databases/aebIvNnT0xY/doky1X9-WfwJ
Cannot change zookeeper log filename
After searching I found this link but I didn't found how to change the variable for the log, I noticed that inside the file zkServer.sh is defined the name of the file, so I needed to change this variable:
_ZOO_DAEMON_OUT="$ZOO_LOG_DIR/zookeeper-$USER-server-$HOSTNAME.out"
I don't know if it's ok to change that variable but know is working as expected.
zookeeper.log file not created inside logs directory
For Chef :
1) Create zookeeper-env.sh.erb template file and set the classpath.
NAME=zookeeper
ZOODIR="path/to/zookeeper/directory"
ZOOCFGDIR="$ZOODIR/conf"
CLASSPATH="$ZOOCFGDIR:$ZOODIR/build/classes:$ZOODIR/bin/build/lib/*.jar:$ZOODIR/lib/slf4j-api-1.7.5.jar:$ZOODIR/lib/netty-3.2.2.Final.jar:$ZOODIR/lib/log4j-1.2.16.jar:$ZOODIR/lib/jline-0.9.94.jar:$ZOODIR/zookeeper-3.4.5-cdh5.3.1.jar:$ZOODIR/src/java/lib/*.jar"
ZOOCFG="$ZOOCFGDIR/zoo.cfg"
ZOO_LOG_DIR="path/to/zookeeper/log/directory"
PIDDIR="~/zookeeper/data/$NAME"
PIDFILE="$PIDDIR/$NAME.pid"
SCRIPTNAME="/etc/init.d/$NAME"
JAVA="/usr/java/jdk1.8.0_25"
ZOOMAIN="org.apache.zookeeper.server.quorum.QuorumPeerMain"
ZOO_LOG4J_PROP="INFO,ROLLINGFILE"
JMXLOCALONLY=false
JAVA_OPTS="-Xms128M -Xmx512M"
- Place this file under zookeeper/conf directory
I assume this works because zoocfg dir is being accessed through zookeeper-env.sh as seen in zkEnv.sh file under zookeeper/bin folder:
if [ -f "${ZOOCFGDIR}/zookeeper-env.sh" ]; then
. "${ZOOCFGDIR}/zookeeper-env.sh"
fi
change the path of the log4j.properties for zookeeper
Here I assume you use the Zookeeper (and it's scripts) provided by apache-kafka
You need to export a variable before starting zookeeper
Here is the default
if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then
export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
fi
So, you need to do
export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:/path/to/your.properties"
zookeeper-server-start zoo.cfg &
It's best if you refactor these together into a systemctl
service, though
log4j ERROR found in kafka connecting to zookeeper
These exceptions are not related to ZooKeeper. They are thrown by log4j as it's not allowed to write to the specified files. These should not prevent Kafka from running but obviously you won't get log4j logs.
When starting Kafka with bin/kafka-server-start.sh
, the default log4j configuration file, log4j.properties
, is used. This attempts to write logs to ../logs/
, see https://github.com/apache/kafka/blob/trunk/bin/kafka-run-class.sh#L194-L197
In your case, this path is /usr/local/kafka/bin/../logs
and Kafka is not allowed to write there.
You can change the default path by setting the LOG_DIR
environment variable to a path where Kafka will be allowed to write logs, for example:
export LOG_DIR=/var/log/kafka/
bin/kafka-server-start.sh config/server.properties &
Related Topics
Timed Out While Waiting for the MAChine to Boot When Vagrant Up
How to Clear the Line Number in Vim When Copying
How to Extract Characters Between the Delimiters Using Sed
Read Line by Line in Bash Script
Tcp_Tw_Reuse VS Tcp_Tw_Recycle:Which to Use (Or Both)
Fastest Way to Tell If Two Files Have the Same Contents in Unix/Linux
One Core Exclusively for My Process
Copy Folder Structure (Without Files) from One Location to Another
Sed Command with -I Option (In-Place Editing) Works Fine on Ubuntu But Not MAC
Can't Su to User Jenkins After Installing Jenkins
Linux Cross-Compilation for Arm Architecture
How to Connect to a Terminal to a Serial-To-Usb Device on Ubuntu 10.10 (Maverick Meerkat)
Download Images from Google with Command Line