Pipe String to Gnu Date for Conversion - How to Make It Read from Stdin

Pipe string to GNU Date for conversion - how to make it read from stdin?

Yes.

 echo "yesterday" | xargs date +"%d %m %Y" -d

How to format a string date (with text and milliseconds) using AWK

You can call an external command like this:

awk '{
         cmd="date -d \""$0"\" +%Y%m%d%H%M%S%N"
         cmd | getline ts
         print $0, ts
         # awk opened a pipe for the communication with 
         # the command. close that pipe to avoid running
         # out of file descriptors
         close(cmd)
     }' <<< '16-FEB-2008 14:17:59.994669'

Output:

16-FEB-2008 14:17:59.994669 20080216141759994669000

---

Thanks to dave_thompson_085's comment you can significantly improve the performance if you have date from GNU coreutils and gawk. GNU's date supports reading dates from stdin and gawk supports co-processes which allows to start a single instance of date in the background, write into it's stdin and read from stdout:

{
    cmd = "stdbuf -oL date -f /dev/stdin +%Y%m%d%H%M%S%N"
    print $0 |& cmd 
    cmd |& getline ts
    print $0, ts
}

Note that you need to use the stdbuf command in addition to force date to output the results line by line.

How do I extract log entries between two times without date?

sed can be used to extract lines expressed by regexp adresses

/^11:.*$/,/^13:26:28.849031 .*$/p

First address could be further refined by getting the minutes digits and adding to the expression as
/^11:(2[6-9]|[3-5][0-9]).*$/,/^13:26:28.849031 .*$/p

last_line=$(tail -n1 test.txt)
end_time=$(cut -d ' ' -f1 <<<"$last_line")
end_hour="${end_time:0:2}"
min_msb="${end_time:3:1}"
min_next=$(($min_msb+1))
min_lsb="${end_time:4:1}"
start_hour=$(($end_hour-2))

if [ "$min_msb" -lt 5 ];then
  min_next=$(($min_msb+1))
else
  min_next=5
fi

sed -rn "/^$start_hour:($min_msb[$min_lsb-9]|[$min_next-5][0-9]).*$/,/^$end_time .*$/p" test.txt

If times span more than 24 h

22:57:46.709883 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9136:9287, ack 13044, win 420, length 151
23:26:28.709883 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9136:9287, ack 13044, win 420, length 151
...
00:36:28.849031 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9743:10269, ack 14286, win 420, length 526

Then

UPDATE: fixed regexp for sed first address when time is across midnight.

hour_range=2
last_line=$(tail -n1 test.txt)
#end_time=$(cut -d ' ' -f1 <<<"$last_line")
end_time="${last_line:0:8}"

start_time="$(date -d "$(date -d "$end_time" --iso=seconds) -$hour_range hour" '+%T')"
echo "Time range: $start_time - $end_time"

end_hour="$(printf "%d" ${end_time:0:2})"
min_msb="$(printf "%d" ${end_time:3:1})"
min_lsb="$(printf "%d" ${end_time:4:1})"
start_hour="$(printf '%d' ${start_time:0:2})"

if [ "$min_msb" -lt 5 ];then
  min_next=$(($min_msb+1))
else
  min_next=5
fi
# Crossed midnight
start_hour_expr="$start_hour:($min_msb[$min_lsb-9]|[$min_next-5][0-9])"
if [ "$start_hour" -gt "$end_hour" ];then
  start_hour_lsb_next=$((${start_hour:1:1} + 1))
  start_hour_next="${start_hour:0:1}${start_hour_lsb_next}"
  if [ "$start_hour_next" -eq 24 ]; then
     start_hour_next="00"
  fi
  start_hour_expr="($start_hour_expr|$start_hour_next:[0-5][0-9])"
fi

echo "sed expression:"
echo -e "/^$start_hour_expr.*$/,/^$end_time.*$/p \n"

sed -rn "/^$start_hour_expr.*$/,/^$end_time.*$/p" test.txt

Given

21:32:28.709883 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9136:9287, ack 13044, win 420, length 151
21:57:46.709883 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9136:9287, ack 13044, win 420, length 151
22:10:46.709883 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9136:9287, ack 13044, win 420, length 151
23:07:46.709883 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9136:9287, ack 13044, win 420, length 151
23:26:28.709883 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9136:9287, ack 13044, win 420, length 151
00:26:28.849023 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9599:9743, ack 14286, win 420, length 144
00:36:28.849031 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9743:10269, ack 14286, win 420, length 526

Returns

sed expression:
/^(22:(3[6-9]|[4-5][0-9])|23:[0-5][0-9]).*$/,/^00:36:28.*$/p 

23:07:46.709883 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9136:9287, ack 13044, win 420, length 151
23:26:28.709883 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9136:9287, ack 13044, win 420, length 151
00:26:28.849023 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9599:9743, ack 14286, win 420, length 144
00:36:28.849031 IP unn-37-19-198-173.datapacket.com.https > term-IdeaPad-Flex.46364: Flags [P.], seq 9743:10269, ack 14286, win 420, length 526

How to convert timestamps to dates in Bash?

On systems with GNU Coreutils >= 5.3.0, e.g. Linux you can use:

date -d @1267619929

Is there a Unix utility to prepend timestamps to stdin?

Could try using awk:

<command> | awk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0; fflush(); }'

You may need to make sure that <command> produces line buffered output, i.e. it flushes its output stream after each line; the timestamp awk adds will be the time that the end of the line appeared on its input pipe.

If awk shows errors, then try gawk instead.

Pipe libvips CLI output to stdout in Windows

It looks like you've hit a bug --- Windows creates stdin and stdout in text mode, so when libvips writes to stdout, any newline characters will be automatically expanded to newline + carriage return.

When running on win, libvips probably needs to call _setmode(1, O_BINARY); to force binary mode before writing image data. I've pushed a patch to libvips 8.10 and credited you. This fix will be in 8.10.3, due in a week or two.

How to read input from STDIN in x86_64 assembly?

First of all : there are no variables in assembly. There are just labels for some kind of data. The data is, by design, untyped - at least in real assemblers, not HLA (e.g. MASM).

Reading from the standard input is achieved by using the system call read. I assume you've already read the post you mentioned and you know how to call system calls in x64 Linux. Assuming that you're using NASM (or something that resembles its syntax), and that you want to store the input from stdin at the address buffer, where you have reserved BUFSIZE bytes of memory, executing the system call would look like this :

xor eax, eax      ; rax <- 0 (syscall number for 'read')
xor edi, edi      ; edi <- 0 (stdin file descriptor)
mov rsi, buffer   ; rsi <- address of the buffer.  lea rsi, [rel buffer]
mov edx, BUFSIZE  ; rdx <- size of the buffer
syscall           ; execute  read(0, buffer, BUFSIZE)

Upon returning, rax will contain the result of the syscall. If you want to know more about how it works, please consult man 2 read. Note that the syscall for read on mac is 0x2000003 instead of 0, so that first line would instead be mov rax, 0x2000003.

Parsing an integer in assembly language is not that simple, though. Since read only gives you plain binary data that appears on the standard input, you need to convert the integer value yourself. Keep in mind that what you type on the keyboard is sent to the application as ASCII codes (or any other encoding you might be using - I'm assuming ASCII here). Therefore, you need to convert the data from an ASCII-encoded decimal to binary.

A function in C for converting such a structure to a normal unsigned int could look something like this:

unsigned int parse_ascii_decimal(char *str,unsigned int strlen)
{
    unsigned int ret = 0, mul = 1;
    int i = strlen-1;
    while(i >= 0)
    {
        ret += (str[i] & 0xf) * mul;
        mul *= 10;
        --i;
    }
    return ret;
}

Converting this to assembly (and extending to support signed numbers) is left as an exercise for the reader. :) (Or see NASM Assembly convert input to integer? - a simpler algorithm only has 1 multiply per iteration, with total = total*10 + digit. And you can check for the first non-digit character as you iterate instead of doing strlen separately, if the length isn't already known.)

---

Last but not least - the write syscall requires you to always pass a pointer to a buffer with the data that's supposed to be written to a given file descriptor. Therefore, if you want to output a newline, there is no other way but to create a buffer containing the newline sequence.

How to know when there is input through the terminal pipe line on C++ 11?

C++ POSIX Standard

As far as I know, there is no such thing. There is a C POSIX library, which is part of POSIX standard.

So there is an alternative to them on the C++ 11?

There is no alternative in standard C++ (not before C++11, so far not after either).

You will need to depend on POSIX to get the functionality that you need.

Even in POSIX, it is the unistd.h which defines isatty. You've neglected to include it in your program.

YYYY-MM-DD format date in shell script

In bash (>=4.2) it is preferable to use printf's built-in date formatter (part of bash) rather than the external date (usually GNU date).

As such:

# put current date as yyyy-mm-dd in $date
# -1 -> explicit current date, bash >=4.3 defaults to current time if not provided
# -2 -> start time for shell
printf -v date '%(%Y-%m-%d)T\n' -1 

# put current date as yyyy-mm-dd HH:MM:SS in $date
printf -v date '%(%Y-%m-%d %H:%M:%S)T\n' -1 

# to print directly remove -v flag, as such:
printf '%(%Y-%m-%d)T\n' -1
# -> current date printed to terminal

In bash (<4.2):

# put current date as yyyy-mm-dd in $date
date=$(date '+%Y-%m-%d')

# put current date as yyyy-mm-dd HH:MM:SS in $date
date=$(date '+%Y-%m-%d %H:%M:%S')

# print current date directly
echo $(date '+%Y-%m-%d')

Other available date formats can be viewed from the date man pages (for external non-bash specific command):

man date

---

Related Topics