Tar: Cowardly Refusing to Create an Empty Archive

tar command failing with Cowardly refusing to create an empty archive

Put a . in the end of the command.

The -C tell tar to change its directory before running but without specifying what to archive tar has no idea what to do.

You can read tar cf foo.tar -C bar zar1 zar2 as: create an archive named foo.tar by going to bar folder and archiving zar1 and zar2 files.

Dockerfile tar error: tar: Cowardly refusing to create an empty archive

That clearly means there are no *.csproj files in your Docker
image. The problem is that RUN commands are run inside inside your
image, not on your host so you have to add *.csproj to your image
first.

Issues with tar command

When you do:

tar -c --use-compress-program=pigz -f /home/jhonst/data_lake/1m/UX.tar /home/jhonst/data_lake/1m/*.UX.csv

You'll get:

tar -t -f /home/jhonst/data_lake/1m/UX.tar
home/jhonst/data_lake/1m/file1.UX.csv
home/jhonst/data_lake/1m/file2.UX.csv
...

Which is not the best. There are two possibilities for getting rid of the "path" inside the tar archive:

Go inside the directory with cd (in a subshell or with pushd/popd if you want to return to the original directory after the tar command):

cd /home/jhonst/data_lake/1m && tar -c --use-compress-program=pigz -f UX.tar *.csv

# returning to the same place after the tar:
(cd /home/jhonst/data_lake/1m && tar -c --use-compress-program=pigz -f UX.tar *.csv)
# or:
pushd /home/jhonst/data_lake/1m && {
    tar -c --use-compress-program=pigz -f UX.tar *.csv
    popd
}

Use the -C option of GNU tar, which is not that easy to handle:

dirpath=/home/jhonst/data_lake/1m
files=("$dirpath"/*.csv)
tar -c --use-compress-program=pigz -f "$dirpath"/UX.tar -C "$dirpath" "${files[@]#$dirpath/}"

Why does tar fail to create an archive when called via Python's subprocess.call while using a * wildcard?

Wildcards aren't processed by tar, they need to be processed by the program that calls it. Usually, that program is a shell.

However, it doesn't have to be; you can get much safer operation (if any of your parameters are user-configurable) by doing the work in native Python instead of using shell=True:

subprocess.call(['tar', '-cvf', 'SAP_2017_04.tar'] + glob.glob('SAP_1704*'),
                stdout=open('SAP_2017_04.filelist', 'w'))

Instead of 1>somefile (an instruction to your shell to redirect stdout, FD 1, to write to somefile), we use stdout=open('somefile', 'w') to tell Python the same thing.
Instead of just putting SAP_1704* directly in the command line, we call glob.glob('SAP_1704*') in Python, and add the list it returns to the argument list.

tar and recursive archiving

Try the next:

arch="archive.tar.gz"
while read -r -d $'\0' dir
do
    (cd "$dir" && find . -maxdepth 1 -iregex '.*\.docx?' -print0 | tar --null -czf "$arch" -T - --remove-files)
    #alternatively
    #(cd "$dir" && shopt -s nocaseglob nullglob && tar --no-recursion -czf "$arch" *.doc *.docx  --remove-files)
done < <(find . \( -ipath '*/test/*' -o -ipath '*/notest/*' \) -iregex '.*\.docx?' -printf '%h\0' | sort -zu)

some comments:

alternative -ipath with the construction \( -ipath '*/test/*' -o -ipath '*/notest/*' \)
the regex .*\.docx? - must match the whole filename and the x? mean zero or one x
tar can read the list of files from stdin with -T -
using null terminated filenames (helps if paths contain spaces)
the --null instructs tar to use such null terminated filenames
(cd ... &&) run in the subshell, so not need cd back

Shell Script - tar comand mistaking archive name for directory

Filenames can't have / in them, since those are directory separators. Change the format of your date to use different delimiters.

DATETIME=$(date +'%y-%m-%d-%H_%M_%S')

Tar command in Python scripting fails after adding two variables

first add the forward slash..

path = '/var/lib/libvirt/images/centos.qcow2'

and strip the \n in your date, it will work...

>>> vmdate
'20220622\n'

>>> vmdate.rstrip()
'20220622'