How to Gzip All Files in All Sub-Directories into One Compressed File in Bash

How to gzip all files in all sub-directories into one compressed file in bash

tar -zcvf compressFileName.tar.gz folderToCompress

everything in folderToCompress will go to compressFileName

Edit: After review and comments I realized that people may get confused with compressFileName without an extension. If you want you can use .tar.gz extension(as suggested) with the compressFileName

How to gzip all files in all sub-directories in bash

No need for loops or anything more than find and gzip:

find . -type f ! -name '*.gz' -exec gzip "{}" \;

This finds all regular files in and below the current directory whose names don't end with the .gz extension (that is, all files that are not already compressed). It invokes gzip on each file individually.

---

Edit, based on comment from user unknown:

The curly braces ({}) are replaced with the filename, which is passed directly, as a single word, to the command following -exec as you can see here:

$ touch foo
$ touch "bar baz"
$ touch xyzzy
$ find . -exec echo {} \;

./foo
./bar baz
./xyzzy

gzipping up a set of directories and creating a tar compressed file

You can create a gzipped tar on the commandline as follows:

tar czvf mytar.tar.gz dir1 dir2 .. dirN

If you wanted to do that in a bash script and pass the directories as arguments to the script, those arguments would end up in $@. So then you have:

tar czvf mytar.tar.gz "$@"

If that is in a script (lets say myscript.sh), you would call that as:

./myscript.sh dir1 dir2 .. dirN

If you want to read from a list (your option 1) you could do that like so (this does not work if there is whitespace in directory names):

tar czvf mytar.tar.gz $(<config.txt)

Bash - How can I archive and compress files in subdirectories but only with a certain filename

This overcame this issue described in the other answer.

find main_directory/ -name "myfile.txt" | tar -czvf mytar.tar.gz -T -

GZip every file separately

You can use gzip *

---

Note:

• This will zip each file individually and DELETE the original.
• Use -k (--keep) option to keep the original files.
• This may not work if you have a huge number of files due to limits of the shell
• To run gzip in parallel see @MarkSetchell's answer below.

In linux, how can I zip specific sub directories into their own zip files named parent directory name and output them all into a single directory?

I think you've mostly described the steps. You just need to put them into a script. Something like:

#!/bin/bash

cd /srv/www
for domain in *; do
(
cd $domain/html &&
zip -r /srv/web_backup/$domain.zip . .htaccess
)
done

Breaking that down a bit:

# change into the /srv/www directory
cd /srv/www

# for each domain in this directory (we're largely assuming here that
# this directory contains only directories representing domains you
# want to backup).
for domain in *; do

# a (...) expression runs the included commands in a subshell, so
# that when we exit we're back in /srv/www.
(
# change into the html directory and, if that was successful, update
# your zip file.
cd $domain/html &&
zip -r /srv/web_backup/$domain.zip .
)

done

This is only one way of tackling this particular problem. You could also write something like:

#!/bin/sh

find /srv/www/* -maxdepth 0 -type d -print0 |
xargs -0 -iDOMAIN sh -c 'cd "DOMAIN/html" && zip -r "/srv/www/DOMAIN.zip" .

That does roughly the same thing, but makes clever use of find and xargs.

combine multiple gzip compressed files in a tar file

With GNU tar, you can do:

tar -Oxf Big.tar --wildcards 'Big/*.gz' > /tmp/your_file.gz

With OS X tar, you have to list the files individually:

tar -Oxf Big.tar Big/FileA.gz Big/FileB.gz > /tmp/your_file.gz

The salient feature in both is -O, which writes the files to stdout.

Here's an example transcript on a GNU system:

$ pwd
/home/me

$ tar tf Big.tar
Big/
Big/foo.txt.gz
Big/bar.txt.gz

$ tar -Oxf Big.tar --wildcards 'Big/*.gz' > /tmp/your_file.gz

$ zcat /tmp/your_file.gz
This is the contents of foo.txt
This is the contents of bar.txt

Unzip all gz files in all subdirectories in the terminal

If you want, for each of those, to launch "gzip -d" on them:

cd theparentdir && gzip -d $(find ./ -type f -name '*.gz')

and then, to gzip them back:

cd theparentdir && gzip $(find ./ -type f -name '*.csv')

This will however choke in many cases

• if filenames have some special characters (spaces, tabs, newline, etc) in them
• other similar cases
• or if there are TOO MANY files to be put after the gzip command!

A solution would be instead, if you have GNU find, to do :

find ... -print0 | xarsg -0 gzip -d # for the gunzip one, but still choke on files with "newline" in them

Another (arguably better?) solution, if you have GNU find at your disposal:

cd theparentdir && find ./ -type f -name '*.gz' -exec gzip -d '{}' '+'

and to re-zip all csv in that parentdir & all subdirs:

cd theparentdir && find ./ -type f -name '*.csv' -exec gzip '{}' '+'

"+" tells GNU find to try to put as many found files as it can on each gzip invocation (instead of doing 1 gzip incocation per file, very very ressource intensive and very innefficient and slow), similar to xargs, but with some benefits (1 command only, no pipe needed)

---

Related Topics