How to Create Tar for Files Older Than 7 Days Using Linux Shell Scripting

How to create tar for files older than 7 days using linux shell scripting

This will work:

#!/bin/bash
files=()
while IFS=  read -r -d $'\0'; do
    files+=("$REPLY")
done < <(find /var/log/ -mtime +7 -print0)
tar cvfz backup.tar.gz "${files[@]}"

Note the use of "${files[@]}" as opposed to ${files[*]}. "${files[@]}" will expand to provide tar with one argument per file name and will work even if the file names contain spaces, tabs, or newlines. By contrast, after the shell expands ${files[*]}, it will perform word splitting, potentially mangling your file names.

For a detailed explanation of the loop used to create the files array, see: How can I store find command result as arrays in Bash

All files and directories produced by the command find /var/log/ -mtime +7 will be included in the tar file. To include only files, not directories, see Skynet's answer.

To archive logs from the seven most recent days

Only one character need change:

#!/bin/bash
files=()
while IFS=  read -r -d $'\0'; do
    files+=("$REPLY")
done < <(find /var/log/ -mtime -7 -print0)
tar cvfz backup.tar.gz "${files[@]}"

This works because find interprets numeric arguments as follows:

Numeric arguments can be specified as

+n for greater than n,

-n for less than n,

n for exactly n.

Thus, -mtime +7 means greater than 7 days old while -mtime -7 means less than 7. Note that find will ignore fractional parts. Thus +7 will include 8 days old but not 7.5 days old. See man find for details.

Tar files older than n days

tar's option --files-from handles whitespace in filenames, filename - means stdin.

Hence a simpler version:

name=$(date +%Y-%m-%d)
find /mnt/main/var/www/m2allcrm/ -maxdepth 1 -mtime +180 -type f \( -name "*.log" -o -name "*.csv" \) | tar cvzf backup_$name.tar.gz --files-from -

How to create zip/gz/tar files for if the files are older than particular days in UNIX or Linux

Thanks a lot for your reply.
I got it.

files=($(find /tmp/mallik3/ -mtime +"$days"))
for files in ${files[*]}
do
     echo $files
     zip $files-$(date --date="- "$days"days" +%F)_.zip $files
      #       tar cvfz $(files)_$(date --date='-6months' +%F).tar.gz $files
#       rm $files
done

How to delete files older than 7 days in Linux using the shell

It's correct. However, I've just tested that and it looks like it doesn't only rely on the date info, but also on the hours and minutes. +7 will then remove the files older than 168 hours. I have some similar set up, please have a look on that:

root@it-pbx01:/var/lib/asterisk/backups/BACKUP# date
Fri Mar 15 12:51:03 CET 2013
root@it-pbx01:/var/lib/asterisk/backups/BACKUP# ls -l
total 872780
-rw-rw-r-- 1 asterisk asterisk 128513903 Mar  8 18:01 20130308.18.00.02.tar.gz
-rw-rw-r-- 1 asterisk asterisk 128517514 Mar  9 18:01 20130309.18.00.01.tar.gz
-rw-rw-r-- 1 asterisk asterisk 128517659 Mar 10 18:01 20130310.18.00.01.tar.gz
-rw-rw-r-- 1 asterisk asterisk 126791825 Mar 11 18:01 20130311.18.00.01.tar.gz
-rw-rw-r-- 1 asterisk asterisk 126791573 Mar 12 18:01 20130312.18.00.01.tar.gz
-rw-r--r-- 1 asterisk asterisk 126791404 Mar 13 18:01 20130313.18.00.02.tar.gz
-rw-r--r-- 1 asterisk asterisk 126871966 Mar 14 18:01 20130314.18.00.01.tar.gz
root@it-pbx01:/var/lib/asterisk/backups/BACKUP# find . -type f -mtime +7
root@it-pbx01:/var/lib/asterisk/backups/BACKUP# find . -type f -mtime +6
root@it-pbx01:/var/lib/asterisk/backups/BACKUP# find . -type f -mtime +5
./20130308.18.00.02.tar.gz
root@it-pbx01:/var/lib/asterisk/backups/BACKUP# find . -type f -mtime +4
./20130309.18.00.01.tar.gz
./20130308.18.00.02.tar.gz
root@it-pbx01:/var/lib/asterisk/backups/BACKUP# find . -type f -mtime +3
./20130309.18.00.01.tar.gz
./20130310.18.00.01.tar.gz
./20130308.18.00.02.tar.gz
root@it-pbx01:/var/lib/asterisk/backups/BACKUP# find . -type f -mtime +2
./20130309.18.00.01.tar.gz
./20130311.18.00.01.tar.gz
./20130310.18.00.01.tar.gz
./20130308.18.00.02.tar.gz
root@it-pbx01:/var/lib/asterisk/backups/BACKUP# find . -type f -mtime +1
./20130309.18.00.01.tar.gz
./20130311.18.00.01.tar.gz
./20130312.18.00.01.tar.gz
./20130310.18.00.01.tar.gz
./20130308.18.00.02.tar.gz
root@it-pbx01:/var/lib/asterisk/backups/BACKUP# find . -type f -mtime +0
./20130309.18.00.01.tar.gz
./20130311.18.00.01.tar.gz
./20130312.18.00.01.tar.gz
./20130313.18.00.02.tar.gz
./20130310.18.00.01.tar.gz
./20130308.18.00.02.tar.gz
root@it-pbx01:/var/lib/asterisk/backups/BACKUP# find . -type f -mtime 0
./20130314.18.00.01.tar.gz

Removing files older than X days using a date format in the filename

With bash and a regex:

for i in app-*.log*; do
  [[ "$i" =~ -([0-9]{4}-[0-9]{2}-[0-9]{2}) ]] \
    && [[ "${BASH_REMATCH[1]}" < "2020-12-20" ]] \
    && echo rm -v "$i"
done

${BASH_REMATCH[1]} contains 2020-12-17, e.g.

As one line:

for i in app-*.log*; do [[ "$i" =~ -([0-9]{4}-[0-9]{2}-[0-9]{2}) ]] && [[ "${BASH_REMATCH[1]}" < "2020-12-20" ]] && echo rm -v "$i"; done

Output:

rm -v app-2020-12-17.log.2
rm -v app-2020-12-18.log.1
rm -v app-2020-12-18.log.2
rm -v app-2020-12-18.log.31
rm -v app-2020-12-18.log.32
rm -v app-2020-12-18.log.33
rm -v app-2020-12-18.log.3.gz

Linux shell script to tar.gzip log files older than 1 month grouped by month

Here's the third solution for your updated question:

#!/usr/bin/env bash

LOGTYPES=$( ls *log* | sed -rn "s/([0-9]{6})[0-9]{2}.*$/\1/p" | sort -u )

# the sed command, item by item:
#
# s/ search and replace
# ([0-9]{6}) block of 6 digits, and store it
# [0-9]{2} followed by 2 more digits
# .*$ followed by any and all characters until the end of the input
# / replace all of that with
# \1 the first stored block (the 6 digits)
# /p print the output
#
# So this turns FailedAudit_20150101_000000.log into FailedAudit_201501

THIS_MONTH=$(date +%Y%m)
for LOG in $LOGTYPES; do
    MONTH=${LOG: -6} # Last 6 characters of the LOGTYPE are YYYYMM

    if [[ "$MONTH" -lt "$THIS_MONTH" ]]; then
        LOG_FILES=$(ls ${LOG}*)
        tar -czf ${LOG}.tar.gz ${LOG_FILES}
        RC=$? # Check whether an error occured
        if [[ "$RC" == "0" ]]; then
            rm ${LOG_FILES}
        fi
    fi
done

Note: This assumes that the first block of 8 digits is the datestamp, and everything after that is not relevant for which archive it is to go to.

Update:
The sed script no longer outputs files that do not contain a timestamp.

---

Related Topics