How to Make Find and Printf Works in Bash Script

How to make find and printf works in bash script

Since you don't specify in what way your attempt failed [See note 1, below], I've just provided some thoughts about how to solve the underlying question of producing lists of checksums for each directory.

Your use of find is both unnecessary and unsafe (if there are any directories whose names include whitespace or shell metacharacters.

For example,

$( find . -maxdepth 1 -mindepth 1 -type d )

is roughly the same as the "glob"

./*/

except that the glob does not perform pathname expansion or word-splitting on the individual directories. Furthermore, you could use the glob */, which will avoid the ./ prefix, which you evidently don't want.

Also,

b=$(echo $i | awk '{print substr($1,3); }' )

could be written much more simply as

b=${i:3}   # Bash substring notation

b=${i#./}  # Posix standard prefix deletion

But, as noted above, you could use a glob and thereby avoid any having to remove the prefix.

So you could have simply written:

#!/bin/bash
for b in */; do
    find "$b" -type f > "$b/$b.txt"
done

Note the quotes around the variable expansions. If you really need the %b format to print filenames, then you definitely need to quote the expansions to avoid problems with whitespace and shell metacharacters, but anyway you should develop the a habit of quoting variable expansions.

That will produce a slightly different output than your script since it doesn't place the ./ at the beginning of each line in the output file, but if you really wanted that you could use find "./$b" ....

I don't understand why you feel the need to use printf to output the filename for the SHA-1 checksums, because sha1sum already prints the filename. It's true that it does so after the checksum instead of before. There is actually a good reason for that behaviour: since there is no way of knowing what characters the filename might include, putting the checksum first makes it easier to parse the output.

Note that in the output of sha1sum, the output for each file whose name includes a backslash or newline, the checksum line will start with a backslash and the backslash and newline characters in the filename will be escaped. This produces an effect somewhat similar to that of using the %b format to printf.

If you really want to turn the order around, you can do so easily enough with awk:

#!/bin/bash
for b in */; do
    find "$b" -type f '!' -iname '*thumbs.db*' -exec sha1sum {} + |
    awk '{print "$2:$1"}' > "$b/$b.txt"
done

Using + instead of ; to terminate the -exec option lets find execute the command with a list of filenames instead of just a single filename, which is much more efficient.

Notes

See the Stack Overflow help page, from which I quote:
Questions seeking debugging help ("why isn't this code working?") must include the desired behavior, a specific problem or error and the shortest code necessary to reproduce it in the question itself. Questions without a clear problem statement are not useful to other readers.
"It doesn't work" is not a "specific problem or error". You should always include the literal text of the error message or a clear statement of how the result of the program failed to meet your expectations.

find with $(printf ... array) conditions

set -x has the answer to the question "why doesn't it work"?:

$ set -x
$ find . $(printf -- "-not -path '*/%s' " "${exclude[@]}")
++ printf -- '-not -path '\''*/%s'\'' ' A C
+ find . -not -path ''\''*/A'\''' -not -path ''\''*/C'\'''

Well, it is painful to work out what is going on with all of those quotes but it's enough to see that the command isn't what you want.

The important thing is that the single quotes in the string produced by printf aren't syntactic, i.e. they do not indicate the start and end of an argument. Instead, they are treated as literal quotes, so the find command is excluding directories which start and end with quotes.

One solution would be to use a loop to build your command:

args=()
for e in "${exclude[@]}"; do 
  args+=( -not -path "*/$e" )
done

find . "${args[@]}"

Is there a program that functions like printf with an entire file as input?

You could use printf with the whole file as the formatting string:

printf "$(< file.txt)\n" one two three

Notice that command substitution removes trailing newlines, so a separate \n is added back; see How to avoid bash command substitution to remove the newline character? for details.

formatting the output using printf in bash script

Change line 34 to

printf "%-*s%s\n" $((${mid}-${#title})) " " "$i"

Result:

                                 Installater Options 
                                ---------------------

                                ---------------------
                                1. Install
                                2. Uninstall
                                ---------------------

How to use a bash variable to hold the format of find's printf statement

You can store a single argument in a regular variable:

print_format="File: %p has modification time [%TY-%Tm-%Td %TH:%TM:%TS %TZ]\n"
find ./ -type f -printf "$print_format"

or you can store one or more arguments in an array:

find_options=( -printf "File: %p has modification time [%TY-%Tm-%Td %TH:%TM:%TS %TZ]\n" )
find ./ -type f "${find_options[@]}"

How to get color and width formatting with printf

It would appear that the printf function is counting the escape sequence characters that set the colours as part of the output width. Adding the number of characters required for each colour format to the width specifier should fix the issue.

Minimal Working Example:

$ GREEN=$(tput setaf 2) && printf "[%-15s][%-10s][%-5s][${GREEN}%-10s]\n" "Hello" "World" "Eat" "Cake"
[Hello          ][World     ][Eat  ][Cake      ]

find command with filename coming from bash printf builtin not working

Don't use for loops. First, it is slower. Your find has to complete before the rest of your program can run. Second, it is possible to overload the command line. The enter for command must fit in the command line buffer.

Most importantly of all, for sucks at handling funky file names. You're running conniptions trying to get around this. However:

find $1 -type f -print0 | while read -r -d $'\0' FILE

will work much better. It handles file names -- even file names that contain \n characters. The -print0 tells find to separate file names with the NUL character. The while read -r -d $'\0 FILE will read each file name (separate by the NUL character) into $FILE.

If you put quotes around the file name in the find command, you don't have to worry about special characters in the file names.

Your script is running find once for each file found. If you have 100 files in your first directory, you're running find 100 times.

Do you know about associative (hash) arrays in BASH? You are probably better off using associative arrays. Run find on the first directory, and store those files names in an associative array.

Then, run find (again using the find | while read syntax) for your second directory. For each file you find in the second directory, see if you have a matching entry in your associative array. If you do, you know that file is in both arrays.

Addendum

I've been looking at the find command. It appears there's no real way to prevent it from using pattern matching except through a lot of work (like you were doing with printf. I've tried using the -regex matching and using \Q and \E to remove the special meaning of pattern characters. I haven't been successful.

There comes a time that you need something a bit more powerful and flexible than shell to implement your script, and I believe this is the time.

Perl, Python, and Ruby are three fairly ubiquitous scripting languages found on almost all Unix systems and are available on other non-POSIX platforms (cough! ...Windows!... cough!).

Below is a Perl script that takes two directories, and searches them for matching files. It uses the find command once and uses associative arrays (called hashes in Perl). I key the hash to the name of my file. In the value portion of the hash, I store an array of the directories where I found this file.

I only need to run the find command once per directory. Once that is done, I can print out all the entries in the hash that contain more than one directory.

I know it's not shell, but this is one of the cases where you can spend a lot more time trying to figure out how to get shell to do what you want than its worth.

#! /usr/bin/env perl

use strict;
use warnings;
use feature qw(say);

use File::Find;
use constant DIRECTORIES => qw( dir1 dir2 );

my %files;
#
# Perl version of the find command. You give it a list of
# directories and a subroutine for filtering what you find.
# I am basically rejecting all non-file entires, then pushing
# them into my %files hash as an array.
#
find (
    sub {
        return unless -f;
        $files{$_} = [] if not exists $files{$_};
        push @{ $files{$_} }, $File::Find::dir;
    },  DIRECTORIES
);

#
# All files are found and in %files hash. I can then go
# through all the entries in my hash, and look for ones
# with more than one directory in the array reference.
# IF there is more than one, the file is located in multiple
# directories, and I print them.
#

for my $file ( sort keys %files ) {
    if ( @{ $files{$file} } > 1 ) { 
        say  "File: $file: " . join ", ", @{ $files{$file} };
    }
}

echo, printf not work in script when run it as command

test is a bash built-in. POSIX systems will also have a test executable.

When you enter a command without specifying a path to the executable, bash will first check if the command is one of its built-in commands before searching for the executable in the PATH. If the command matches the name of one of the bash built-ins, it will run the built-in.

If you still want to run your script without specifying its path, there are two ways to do it:

Recommended: Rename your file, and then run it with its new name (your script file needs to have its executable permission bit(s) set).
Make sure your script has its file permissions set so that it is executable, make sure your PATH is set up so that your test will be found before the system's test, and then run env test to run your script. env will search your PATH to find your test executable, and then it will execute it.

Ultimately, option 2 is not recommended, because it can be brittle to reorder your PATH, and it can be confusing (for you and for others) to have a second test binary on your system.

bash printf: How do I output \x and \f within double quotes?

First, let's understand how quoting works.

The simplest form of quoting is the backslash: it prevents the following character from being interpreted in any special way, allowing it to be used as a literal character. For example:

# Two arguments to printf
$ printf '%s\n' a b
a
b
# One three-character argument to printf
printf '%s\n' a\ b
a b

Double quotes are equivalent to escaping every character contained therein: "a b c" is equivalent to \a\ \b\ \c, which is the same as a\ b\ c because a, b, and c have no special meaning to begin with. You can think of every character inside double quotes as being treated literally, with the following exceptions:

$ can start a parameter expansion or a command substitution. Escape it with a backslash to treat it literally.
```
 $ foo=3
 $ echo "$foo"
 3
 $ echo "\$foo"
 $foo
```
A backquote starts a command substitution. Escape it with a backslash to treat it literally.
```
$ echo "`pwd`"
/home/me
$ echo "\`pwd\`"
`pwd`
```
A double quote ends a quoted string. Escape it with a backslash to treat it literally.
```
$ echo "\""
"
```
Because a backslash might be part of one of the three preceding sequences, escape it with a backslash to treat one literally.
```
$ echo "\\"
\
```

Inside single quotes, everything is treated literally, including any use of a backslash. (One consequence of this is that it is impossible to put a single quote in a single-quoted string, because it will always terminate such a string.)

$ echo '\'
\

Once you understand quoting, you next need to realize that printf itself can process whatever backslashes it sees in its first argument.

$ printf 'a\x20b\n'  # a, space (ASCII 0x20), b, newline
a b

In order to ensure a string is printed literally no matter what characters are present, use the %s format specifier and pass your string as a second argument:

$ printf '\x41\n'
A
$ printf '%s\n' '\x41'
\x41

eval printf works from command line but not in script

As @that other guy says, bash's printf builtin barfs on printf "-L ...". It thinks you're passing it a -L option. You can fix it by adding --:

printf -- "-L $port:127.0.0.1:$port "

Let's make that:

printf -- '-L %s:127.0.0.1:%s ' "$port" "$port"

But since we're here, we can do a lot better. First, let's not process JSON with basic shell tools. We don't want to rely on it being formatting a certain way. We can use jq, a lightweight and flexible command-line JSON processor.

$ jq -r 'to_entries | map(select(.key | test(".*_port"))) | .[].value' bar.json
35932
37145
42704
39329
39253

Here we use to_entries to convert each field to a key-value pair. Then we select entries where the .key matches the regex .*_port. Finally we extract the corresponding .values.

We can get rid of eval by constructing the ssh command in an array. It's always good to avoid eval when possible.

#!/bin/bash

readarray -t ports < <(jq -r 'to_entries | map(select(.key | test(".*_port"))) | .[].value' bar.json)

ssh=(ssh foo -f -N)
for port in "${ports[@]}"; do ssh+=(-L "$port:127.0.0.1:$port"); done
"${ssh[@]}"

How to Make Find and Printf Works in Bash Script