How do I rename multiple files beginning with a Unix timestamp - imapsync issue
Try doing this with rename
:
$ rename -n 's/^\d+/(stat($_))[9]/e' [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]*
from the shell
prompt. It's very useful, you can put some perl code like I does in a substitution for stat
with the e
modifier.
You can remove the -n
(dry-run mode switch) when your tests become valids.
There are other tools with the same name which may or may not be able to do this, so be careful.
If you run the following command (linux
)
$ file $(readlink -f $(type -p rename))
and you have a result like
.../rename: Perl script, ASCII text executable
and not containing:
ELF
then this seems to be the right tool =)
If not, to make it the default (usually already the case) on Debian
and derivative like Ubuntu
:
$ sudo update-alternatives --set rename /path/to/rename
(replace /path/to/rename
to the path of your perl's rename
command.
If you don't have this command, search your package manager to install it or do it manually
Last but not least, this tool was originally written by Larry Wall, the Perl's dad.
Edit
As stated here, if you have the following error :
Argument list too long
Then use find
like this :
find -type f -name '[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]*' -print0|
xargs -0 -n1 rename -n 's/^\d+/(stat($_))[9]/e'
(try it without -n1
, that should works too)
Auto file names using awk
Try doing this :
rename -n '$c++; s/^(.{4}).*/sprintf("%s%05d", $1, $c)/e' [A-Z][A-Z][A-Z]_*
You need the perl's rename
, see this
Remove the -n
switch when your tests are OK (dry run mode).
Ex. :
$ ls -1
DEF_FAI_4O58A
FAI_4O57A
FTH_4O59A
box.py
index.html
robots.txt
test.html
$ rename -n '$c++; s/^(.{4}).*/sprintf("%s%04d", $1, $c)/e' [A-Z][A-Z][A-Z]_*
DEF_FAI_4O58A -> DEF_0001
FAI_4O57A -> FAI_0002
FTH_4O59A -> FTH_0003
If you really want awk to do it :
printf '%s\n' [A-Z][A-Z][A-Z]_* |
awk '{c++; printf("%s%04d\n", substr($1, 0, 4), c)}'
Edit :
if you need to treat columns in YOUR_OWN_FILE
:
rename -n '$c++; s/^(.{4}).*/sprintf("%s%04d", $1, $c)/e' $(awk '{print $2}' YOUR_OWN_FILE)
Issue with timestamp format when trying to create my K8S cluster in AWS using a script
As the error message says, the given date-time string does not match the given format.
This is because the former has a trailing "Z", which is not accounted for in the latter.
For example, using jq, we see:
$ jq -ncM '"2021-09-07T08:29:29Z"|strptime("%Y-%m-%dT%H:%M:%SZ")'
[2021,8,7,8,29,29,2,249]
So you could delete the "Z" in the date-time string (e.g. by sub("Z$";"")), or add it in the format string, as shown in the example.
GREP date from email header and make it the files creation date
What follows assumes you are using the default macOS utilities (touch
, date
...) As they are completely outdated some adjustments will be needed if you use more recent versions (e.g. macports
or brew
). It also assumes that you are using bash
.
If you have sub-folders ls
is not the right tool. And anyway, the output of ls
is not for computers, it is for humans. So, the first thing to do is find all email files. Guess what? The utility that does this is named find
:
$ find . -type f -name '*.emlx'
foo/bar.emlx
baz.emlx
...
searches for true files (-type f
) starting from the current directory (.
) and which name is anything.emlx
(-name '*.emlx'
). Adapt to your situation. If all files are email files you can skip the -name ...
part.
Next we need to loop over all these files and process each of them. This is a bit more complex than for f in ...
for several reasons (large number of files, file names with spaces...) A robust way to do this is to redirect the output of a find
command to a while loop:
while IFS= read -r -d '' f; do
<process file "$f">
done < <(find . -type f -name '*.emlx' -print0)
The -print0
option of find
is used to separate the file names with a null character instead of the default newline character. The < <(find...)
part is a way to redirect the output of find
to the input of the while loop. The while IFS= read -r -d '' f; do
reads each file name produced by find
, stores it in shell variable f
, preserving the leading and trailing spaces if any (IFS=
), the backslashes (-r
) and using the null character as separator (-d ''
).
Now we must code the processing of each file. Let's first retrieve the delivery time, assuming it is always the second word of the last line starting with X-Delivery-Time:
:
awk '/^X-Delivery-Time:/ {t = $2} END {print t}' "$f"
does that. If you don't know awk already it's time to learn a bit of it. It's one of the very useful Swiss knives of text processing (sed is another). But let's improve it a bit such that it returns the first encountered delivery time instead of the last, stops as soon as it encountered it, and also checks that the timestamp is a real timestamp (digits):
awk '/^X-Delivery-Time:[[:space:]]+[[:digit:]]+$/ {print $2; exit}' "$f"
The [[:space:]]+
part of the regular expression matches 1 or more spaces, tabs,... and the [[:digit:]]+
matches 1 or more digits. ^
and $
match the beginning and the end of the line, respectively. The result can be assigned to a shell variable:
t="$(awk '/^X-Delivery-Time:[[:space:]]+[[:digit:]]+$/ {print $2; exit}' "$f")"
Note that if there was no match the t
variable will store the empty string. We will use this later to skip such files.
Once we have this delivery time, which looks like a UNIX timestamp (seconds since 1970/01/01) in your example, we must use it to change the last modification time of the email file. The command that does this is touch
:
$ man touch
...
touch [-A [-][[hh]mm]SS] [-acfhm] [-r file] [-t [[CC]YY]MMDDhhmm[.SS]] file ...
...
Unfortunately touch
wants a time in the CCYYMMDDhhmm.SS
format. No worry, the date
utility can be used to convert a UNIX timestamp in any format we like. For instance, with your example timestamp (1535436541
):
$ date -r 1535436541 +%Y%m%d%H%M.%S
201808280809.01
We are almost done:
while IFS= read -r -d '' f; do
# uncomment for debugging
# echo "processing $f"
t="$(awk '/^X-Delivery-Time:[[:space:]]+[[:digit:]]+$/ {print $2; exit}' "$f")"
if [ -z "$t" ]; then
echo "no delivery time found in $f"
continue
fi
# uncomment for debugging
# echo touch -t "$(date -r "$t" +%Y%m%d%H%M.%S)" "$f"
touch -t "$(date -r "$t" +%Y%m%d%H%M.%S)" "$f"
done < <(find . -type f -name '*.emlx' -print0)
Note how we test if t
is the empty string (if [ -z "$t" ]
). If it is, we print a message and jump to the next file (continue
). Just put all this in a file with a shebang line and run...
If, instead of the X-Delivery-Time
field, you must use a Date
field with a more complex and variable format (e.g. Date: Mon, 11 Jun 2018 10:36:14 +0200
), the best would be to install a decently recent version of touch
with the coreutils package of Mac Ports or Homebrew. Then:
while IFS= read -r -d '' f; do
t="$(awk '/^Date:/ {print gensub(/^Date:[[:space:]+](.*)$/,"\\1","1"); exit}' "$f")"
if [ -z "$t" ]; then
echo "no delivery time found in $f"
continue
fi
touch -d "$t" "$f"
done < <(find . -type f -name '*.emlx' -print0)
The awk command is slightly more complex. It prints the matching line without the Date:
prefix. The following sed command would do the same in a more compact form but would not really be more readable:
t="$(sed -rn 's/^Date:\s*(.*)/\1/p;Ta;q;:a' "$f")"
sed removing spaces before and after something
Try doing this :
rename 's/\s*-\s*/-/g' *.doc
You need the Perl's rename
, see this post
IMAP allows creation of multiple mail boxes or folders in the mail box?
IMAP uses the word mailbox like everyone else uses the word folder. Just treat those last two lines as if they read:
- Create, rename or delete folders on mail server
- Create hierarchy of folders in a folder for storage
Related Topics
Aws Lambda Permission Denied When Trying to Use Ffmpeg
What Is the Explanation of This X86 Hello World Using 32-Bit Int 0X80 Linux System Calls from _Start
Shell Script to Kill the Process Listening on Port 3000
Expression After Last Specific Character
What's the Meaning of a ! Before a Command in the Shell
Hadoop: «Error:Java_Home Is Not Set»
Handling Input Confirmations in Linux Shell Scripting
Multiple Ble Connections Using Linux and Bluez 5.0
Shell Shift Procedure - What Is This
How to Debug a Futex Contention Shown in Strace
Platform Independent Resource Management
How to Copy an Array in Nasm X86 Assembly for Linux, Porting 16-Bit Dos Code
How to Figure Out What Is Using a Linux Kernel Module
Parameter for Shell Scripts That Is Started with Qsub
How to Find the Tomcat 7 Installation Folder on Linux Ami in Elastic Beanstalk
Linux Configure/Make, --Prefix