Simpler Way to Repeatedly Read Lines and Invoke a Program

Simpler way to repeatedly read lines and invoke a program

Try this:

xargs -d '\n' -n 1 cowsay

Python: How to keep repeating a program until a specific input is obtained?

There are two ways to do this. First is like this:

while True:             # Loop continuously
    inp = raw_input()   # Get the input
    if inp == "":       # If it is a blank line...
        break           # ...break the loop

The second is like this:

inp = raw_input()       # Get the input
while inp != "":        # Loop until it is a blank line
    inp = raw_input()   # Get the input again

Note that if you are on Python 3.x, you will need to replace raw_input with input.

how to read lines in sequence continuously in python

This seems like a pretty straightforward situation to use a loop in. Files in Python are iterable, so you can just do:

with open('tweets.txt') as file: # a with statement ensures the file will get closed properly
    for line in file:
        ... # do your stuff here for each line

Since you want to have a running count of the number of lines you've used, you may want to add a call to enumerate, which will pair each value you iterate over with a number (starting at zero by default, but you can tell it to start at 1 instead):

with open('tweets.txt') as file:
    for num, line in enumerate(file, start=1):
        ...

Idiomatic way to write Clojure code for repeatedly reading lines from the console?

If your goal is to end up with a sequence of exactly x dates entered by user then:

(for [line (repeatedly x read-line)] (DateFormat/parse line))

or using map:

(map DateFormat/parse (repeatedly x read-line))

Beware of lazy sequences in Clojure: user will be asked to enter more dates as they are needed. If your goal is for user to enter all dates at once (say at startup) then use doall:

(map DateFormat/parse (doall (repeatedly x read-line)))

This will read all dates at once, but will parse them lazily still, so date format validation could fail much later in your program. You can move doall one level up to parse promptly as well:

(doall (map DateFormat/parse (repeatedly x read-line)))

And you can use a helper function to read line with prompt:

(defn read-line-with-prompt [prompt]
  (print prompt)
  (read-line))

Then replace read-line with either:

#(read-line-with-prompt "Enter date: ")

or

(partial read-line-with-prompt "Enter date: ")

getline() is repeatedly reading the file, when fork() is used

It appears that closing a FILE in some cases seeks the underlying file descriptor back to the position where the application actually read to, effectively undoing the effect of the read buffering. This matters, since the OS level file descriptors of the parent and the child point to the same file description, and the same file offset in particular.

The POSIX description of fclose() has this phrase:

[CX] [Option Start] If the file is not already at EOF, and the file is one capable of seeking, the file offset of the underlying open file description shall be set to the file position of the stream if the stream is the active handle to the underlying file description.

(Where CX means an extension to the ISO C standard, and exit() of course runs fclose() on all streams.)

I can reproduce the odd behavior with this program (on Debian 9.8):

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#include <sys/types.h>
#include <sys/wait.h>

int main(int argc, char *argv[]){
    FILE *f;
    if ((f = fopen("testfile", "r")) == NULL) {
        perror("fopen");
        exit(1);
    }

    int right = 0;
    if (argc > 1)
        right = 1;

    char *line = NULL;
    size_t len = 0;
    // first line 
    getline(&line, &len, f);
    printf("%s", line);

    pid_t p = fork();
    if (p == -1) {
        perror("fork");
    } else if (p == 0) {
        if (right)
            _exit(0);  // exit the child 
        else
            exit(0);   // wrong way to exit
    } else {
        wait(NULL);  // parent
    }

    // rest of the lines
    while (getline(&line, &len, f) > 0) {
        printf("%s", line);
    }

    fclose(f);
}

Then:

$ printf 'a\nb\nc\n' > testfile
$ gcc -Wall -o getline getline.c
$ ./get
getline   getline2  
$ ./getline
a
b
c
b
c

Running it with strace -f ./getline clearly shows the child seeking the file descriptor back:

clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f63794e0710) = 25117
strace: Process 25117 attached
[pid 25116] wait4(-1,  <unfinished ...>
[pid 25117] lseek(3, -4, SEEK_CUR)      = 2
[pid 25117] exit_group(1)               = ?

(I didn't see the seek back with a code that didn't involve forking, but I don't know why.)

So, what happens is that the C library on the main program reads a block of data from the file, and the application prints the first line. After the fork, the child exits, and seeks the fd back to where the application level file pointer is. Then the parent continues, processes the rest of the read buffer, and when it's finished, it continues reading from the file. Because the file descriptor was seeked back, the lines starting from the second are again available.

In your case, the repeated fork() on every iteration seems to result in an infinite loop.

Using _exit() instead of exit() in the child fixes the problem in this case, since _exit() only exits the process, it doesn't do any housekeeping with the stdio buffers.

With _exit(), any output buffers are also not flushed, so you'll need to call fflush() manually on stdout and any other files you're writing to.

However, if you did this the other way around, with the child reading and buffering more than it processes, then it would be useful for the child to seek back the fd so that the parent could continue from where the child actually left.

Another solution would be not to mix stdio with fork().

Reading file continuously and appending new lines to list (python)

Store the file position between iterations. This allows to efficiently fast-forward the file when it is opened again:

data = []
file_position = 0

while True:
    with open('test.txt', 'r') as f:
        f.seek(file_position)  # fast forward beyond content read previously
        for line in f:
            data.append(int(line))
        file_position = f.tell()  # store position at which to resume

How to repeatedly execute a function every x seconds?

If your program doesn't have a event loop already, use the sched module, which implements a general purpose event scheduler.

import sched, time

def do_something(scheduler): 
    # schedule the next call first
    scheduler.enter(60, 1, do_something, (scheduler,))
    print("Doing stuff...")
    # then do your stuff

my_scheduler = sched.scheduler(time.time, time.sleep)
my_scheduler.enter(60, 1, do_something, (my_scheduler,))
my_scheduler.run()

If you're already using an event loop library like asyncio, trio, tkinter, PyQt5, gobject, kivy, and many others - just schedule the task using your existing event loop library's methods, instead.

---

Related Topics