Linux Head/Tail with Offset

Linux head/tail with offset

From man tail:

   -n, --lines=K
        output the last K lines, instead of the last 10; 
        or use -n +K to output lines starting with the Kth

You can therefore use ... | tail -n +2 | head -n 3 to get 3 lines starting from line 2.

Non-head/tail methods include sed -n "2,4p" and awk "NR >= 2 && NR <= 4".

unix - head AND tail of file

You can simply:

(head; tail) < file.txt

And if you need to uses pipes for some reason then like this:

cat file.txt | (head; tail)

Note: will print duplicated lines if number of lines in file.txt is smaller than default lines of head + default lines of tail.

Using head and tail to return the nth item from a file in Linux

head -n 50000 FILENAME | tail -1

That should work. However this may not be the most efficient way of accessing the 50000th line

Tailing a file from where I left off before?

If you save the size of the file when you close tail, you can do tail -c +[previous size]. This starts at a specific byte offset of the file instead of a number of lines from the end. This may be subject to a race condition if the size changes between when you close the first tail process and when you measure the size.

Why are you closing and reopening the tail command, instead of just using the -f option and continuing reading?

Is there a linux command (along the lines of cat or dd) that will allow me to specify the offset of the read syscall?

I figured out the issue I was having. For posterity, I will record the answer rather than just delete my question, because the answer wasn't necessarily easy to find.

Essentially, the issue occurred within FUSE. FUSE defaults to not using direct I/O (which is definitely the correct default to have, don't get me wrong), which is what resulted in the reads in size chunks of 4096 (these are the result of FUSE using a page cache of file contents [AKA a file content cache] in the kernel). For what I wanted to test (as explained in the question), I needed to enable direct I/O. There are a few ways of doing this, but the simplest way for me to do this was to pass -o direct_io as a command line argument. This worked for me because I was using the fuse_main call in the main function of my program.

So my main function looked like this:

int main(int argc, char *argv[])
{
    return fuse_main(argc, argv, &my_operations_structure, NULL);
}

and I was able to call my program like this (I used the -d option in addtion to the -o direct_io option in order to display the syscalls that FUSE was processing and the output/debug info from my program):

./prog_name -d -o direct_io test_directory

Then, I tested my program with the following simple test program (I know I don't do very much error checking, but this program is only for some quick and dirty tests):

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>

int main(int argc, char *argv[])
{
    FILE * file;
    char buf[4096];

    int fd;

    memset(&buf[0], 0, sizeof(buf));

    if (argc != 4)
    {
        printf("usage: ./readTest [size] [offset] [filename]\n");
        return 0;
    }

    file = fopen(argv[3], "r");
    if (file == NULL)
    {
        printf("Couldn't open file\n");
        return -1;
    }   

    fd = fileno(file);

    pread(fd, (void *) buf, atoi(argv[1]), (off_t) atoi(argv[2]));

    printf("%s\n", buf);

    return 0;
 }

Linux command (like cat) to read a specified quantity of characters

head works too:

head -c 100 file  # returns the first 100 bytes in the file

..will extract the first 100 bytes and return them.

What's nice about using head for this is that the syntax for tail matches:

tail -c 100 file  # returns the last 100 bytes in the file

You can combine these to get ranges of bytes. For example, to get the second 100 bytes from a file, read the first 200 with head and use tail to get the last 100:

head -c 200 file | tail -c 100

How do I extract a single chunk of bytes from within a file?

Try dd:

dd skip=102567 count=253 if=input.binary of=output.binary bs=1

The option bs=1 sets the block size, making dd read and write one byte at a time. The default block size is 512 bytes.

The value of bs also affects the behavior of skip and count since the numbers in skip and count are the numbers of blocks that dd will skip and read/write, respectively.

---

Related Topics