Is This the Most Efficient Way to Get and Remove First Line in File

Is this the most efficient way to get and remove first line in file?

There is no more efficient way to do this other than rewriting the file.

How to delete the first line of a text file?

Assuming you have enough memory to hold everything in memory:

with open('file.txt', 'r') as fin:
    data = fin.read().splitlines(True)
with open('file.txt', 'w') as fout:
    fout.writelines(data[1:])

We could get fancier, opening the file, reading and then seeking back to the beginning eliminating the second open, but really, this is probably good enough.

How to efficiently remove the first line of a large file?

So, this approach is very hacky. It will work well if your line-sizes are about the same size with a small standard deviation. The idea is to read some portion of your file into a buffer that is small enough to be memory efficient but large enough that writing form both ends will not mess things up (since the lines are roughly the same size with little variance, we can cross our fingers and pray that it will work). We basically keep track of where we are in the file and jump back and forth. I use a collections.deque as a buffer because it has favorable append performance from both ends, and we can take advantage of the FIFO nature of a queue:

from collections import deque
def efficient_dropfirst(f, dropfirst=1, buffersize=3):
    f.seek(0)
    buffer = deque()
    tail_pos = 0
    # these next two loops assume the file has many thousands of
    # lines so we can safely drop and buffer the first few...
    for _ in range(dropfirst):
        f.readline()
    for _ in range(buffersize):
        buffer.append(f.readline())
    line = f.readline()
    while line:
        buffer.append(line)
        head_pos = f.tell()
        f.seek(tail_pos)
        tail_pos += f.write(buffer.popleft())
        f.seek(head_pos)
        line = f.readline()
    f.seek(tail_pos)
    # finally, clear out the buffer:
    while buffer:
        f.write(buffer.popleft())
    f.truncate()

Now, let's try this out with a pretend file that behaves nicely:

>>> s = """1. the quick
... 2. brown fox
... 3. jumped over
... 4. the lazy
... 5. black dog.
... 6. Old McDonald's
... 7. Had a farm
... 8. Eeyi Eeeyi Oh
... 9. And on this farm they had a
... 10. duck
... 11. eeeieeeiOH
... """

And finally:

>>> import io
>>> with io.StringIO(s) as f: # we mock a file
...     efficient_dropfirst(f)
...     final = f.getvalue()
...
>>> print(final)
2. brown fox
3. jumped over
4. the lazy
5. black dog.
6. Old McDonald's
7. Had a farm
8. Eeyi Eeeyi Oh
9. And on this farm they had a
10. duck
11. eeeieeeiOH

This should work out OK if dropfirst < buffersize by a good bit of "slack". Since you only want to drop the first line, just keep dropfirst=1, and you can maybe make buffersize=100 or something just to be safe. It will be much more memory efficient than reading "many thousands of lines", and if no single line is bigger than the previous lines, you should be safe. But be warned, this is very rough around the edges.

How to delete the first line of a BIG file in python efficiently to not waste space?

This would work if you need line number 1 to be removed... not if you need to find first occurence of a certain number.

with open('initial_file.txt', 'r') as file:
    lines = file.readlines()
    file.close()
line_that_you_need = lines[0]
with open('initial_file.txt', 'a') as file:
    for i in range(1, len(lines):
        file.write(lines[i])
    file.close

How can I remove the first line of a text file using bash/sed script?

Try tail:

tail -n +2 "$FILE"

-n x: Just print the last x lines. tail -n 5 would give you the last 5 lines of the input. The + sign kind of inverts the argument and make tail print anything but the first x-1 lines. tail -n +1 would print the whole file, tail -n +2 everything but the first line, etc.

GNU tail is much faster than sed. tail is also available on BSD and the -n +2 flag is consistent across both tools. Check the FreeBSD or OS X man pages for more.

The BSD version can be much slower than sed, though. I wonder how they managed that; tail should just read a file line by line while sed does pretty complex operations involving interpreting a script, applying regular expressions and the like.

Note: You may be tempted to use

# THIS WILL GIVE YOU AN EMPTY FILE!
tail -n +2 "$FILE" > "$FILE"

but this will give you an empty file. The reason is that the redirection (>) happens before tail is invoked by the shell:

1. Shell truncates file $FILE
2. Shell creates a new process for tail
3. Shell redirects stdout of the tail process to $FILE
4. tail reads from the now empty $FILE

If you want to remove the first line inside the file, you should use:

tail -n +2 "$FILE" > "$FILE.tmp" && mv "$FILE.tmp" "$FILE"

The && will make sure that the file doesn't get overwritten when there is a problem.

Fastest Java way to remove the first/top line of a file (like a stack)

I'm away from my compiler at the moment, but I think this will work. Edit: works fine.

I urge you to profile it and see. I bet the constructor calls are going to be nothing compared to the file I/O and your comparison operations.

public class FileStack {
  private File file;
  private long position = 0;
  private String cache = null;

  public FileStack(File file) {
    this.file = file;
  }

  public String peek() throws IOException {
    if (cache != null) {
      return cache;
    }

    BufferedReader r = new BufferedReader(new FileReader(file));
    try {
      r.skip(position);
      cache = r.readLine();
      return cache;
    } finally {
      r.close();
    }
  }

  public String pop() throws IOException {
    String r = peek();
    if (r != null) {
      // if you have \r\n line endings, you may need +2 instead of +1
      // if lines could end either way, you'll need something more complicated
      position += r.length() + 1;
      cache = null;
    }
    return r;
  }
}

read the first line of the txt file then delete that line with php

yes it is /.................................

OK less trolling..

You want fopen and fgets will grab a line. REF : fgets Manual PHP

$file = "file.txt"
$f = fopen($file, 'r');
$line = fgets($f);
fclose($f); // You close because you only want the first one. 

There are so many examples how to do this i feel embarrassed answering. You should show some of what you have tried first!

Now you want to remove it: use file_get_contents REF : PHP file_get_contents

//Get your file contents

$newDoc = file_get_contents($file, true);
$newFileContents = substr( $line, strpos($newDoc, "\n")+1 );
//then you want to save it 
file_put_contents($newFileContents, $file);

I might be wrong but you get the idea!~ ;)

Process :

1. Get Contents of file
2. Get First Line
3. Replace content of all file with your First Line as New Line
4. Save File

Im sure there is a more efficient way to do this, im just winging!

NOTE: You may need to configure your php.ini to work with larger files!

What is the most efficient way to get first and last line of a text file?

docs for io module

with open(fname, 'rb') as fh:
    first = next(fh).decode()

    fh.seek(-1024, 2)
    last = fh.readlines()[-1].decode()

The variable value here is 1024: it represents the average string length. I choose 1024 only for example. If you have an estimate of average line length you could just use that value times 2.

Since you have no idea whatsoever about the possible upper bound for the line length, the obvious solution would be to loop over the file:

for line in fh:
    pass
last = line

You don't need to bother with the binary flag you could just use open(fname).

ETA: Since you have many files to work on, you could create a sample of couple of dozens of files using random.sample and run this code on them to determine length of last line. With an a priori large value of the position shift (let say 1 MB). This will help you to estimate the value for the full run.

Remove first line form file using php

You could do this:

$file = file_get_contents(SOME_FILE);
$arr = explode('\n\r', $file);
if (isset($arr[0])) unset ($arr[0]);
$string = implode('\n\r', $arr);
file_put_contents(SOME_FILE, $string);

---

Related Topics