Bash Read/Write File Descriptors -- Seek to Start of File

Bash read/write file descriptors -- seek to start of file

No. bash does not have any concept of "seeking" with its redirection. It reads/writes (mostly) from beginning to end in one long stream.

How do file descriptors work?

File descriptors 0, 1 and 2 are for stdin, stdout and stderr respectively.

File descriptors 3, 4, .. 9 are for additional files. In order to use them, you need to open them first. For example:

exec 3<> /tmp/foo  #open fd 3.
echo "test" >&3
exec 3>&- #close fd 3.

For more information take a look at Advanced Bash-Scripting Guide: Chapter 20. I/O Redirection.

In bash, how do I open a writable file descriptor that's externally redirectable?

Simple enough: If the parent shell is not redirecting fd 3, then test.sh will be redirecting fd 3 to /dev/tty.

if ! { exec 0>&3; } 1>/dev/null 2>&1; then
   exec 3>/dev/tty
fi
echo foo1
echo foo2 >&2
echo foo3 >&3

Bash - Reading from file descriptor simultaneously

Try

sx -vv -k "./firmware_update.bin" >&5 < <(__processing__)

that uses Bash process substitution (see ProcessSubstitution - Greg's Wiki) to cause sx to take its input from the output of the __processing__ function. You'll also need to modify the function to copy its input (from FD5) to its standard output. Something along the lines of:

function __processing__
{
    local line
    while IFS= read -r line<&5; do 
        # ... some processing ...
        printf '%s\n' "$line" > "log.txt"
        printf '%s\n' "$line"    # Copy input to standard output
   done
}

• That code is Shellcheck-clean.
• See function BashFAQ/001 (How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?) for an explanation of the changes that I made on the while ... line.
• See the accepted, and excellent, answer to Why is printf better than echo? for an explanation of why I replaced echo with printf.
• Note that, even with the changes that I made, the __processing__ function won't work reliably if the data coming from FD5 is binary. ASCII NUL characters will be lost (Bash strings can't contain them), and it depends on chunks of input being terminated by newline characters. I don't know what kind of data you are dealing with in this case. It may not be something that can be handled by Bash code.

Using flock to read and write to a file in bash

You don't need to reuse the same descriptor for both read and write processes. In other languages, this would be a code smell, but bash doesn't have native support for seeking within an existing FD, making it a matter of unfortunate necessity.

myfunc () {
  {
    flock -x 3 || return
    count=$(<countfile)
    echo "$((count + 1 ))" >countfile
  } 3<>countfile
}

That said -- not all solutions are native! @telotortium has written an excellent C helper which can be used to seek within a pre-opened FD; if you were to use their code to seek back to the beginning of the file (or something similar to truncate it to 0 and move the FD there), you could reuse a single file descriptor for both the read and the write.

What are file descriptors, explained in simple terms?

In simple words, when you open a file, the operating system creates an entry to represent that file and store the information about that opened file. So if there are 100 files opened in your OS then there will be 100 entries in OS (somewhere in kernel). These entries are represented by integers like (...100, 101, 102....). This entry number is the file descriptor.
So it is just an integer number that uniquely represents an opened file for the process.
If your process opens 10 files then your Process table will have 10 entries for file descriptors.

Similarly, when you open a network socket, it is also represented by an integer and it is called Socket Descriptor.
I hope you understand.

---

Related Topics