Read a File Line by Line in Reverse Order

How to read a file in reverse order?

for line in reversed(open("filename").readlines()):
print line.rstrip()

And in Python 3:

for line in reversed(list(open("filename"))):
print(line.rstrip())

read file backwards (last line first)

It goes like this:

  1. Seek to one byte before the end of the file using fseek. There's no guarantee that the last line will have an EOL so the last byte doesn't really matter.
  2. Read one byte using fgetc.
  3. If that byte is an EOL then the last line is a single empty line and you have it.
  4. Use fseek again to go backwards two bytes and check that byte with fgetc.
  5. Repeat the above until you find an EOL. When you have an EOL, the file pointer will be at the beginning of the next (from the end) line.
  6. ...
  7. Profit.

Basically you have to keep doing (4) and (5) while keeping track of where you were when you found the beginning of a line so that you can seek back there before starting your scan for the beginning of the next line.

As long as you open your file in text mode you shouldn't have have to worry about multibyte EOLs on Windows (thanks for the reminder Mr. Lutz).

If you happen to be given a non-seekable input (such as a pipe), then you're out of luck unless you want to dump your input to a temporary file first.

So you can do it but it is rather ugly.

You could do pretty much the same thing using mmap and a pointer if you have mmap available and the "file" you're working with is mappable. The technique would be pretty much the same: start at the end and go backwards to find the end of the previous line.


Re: "I am the one creating this file. So, can I create in a way its in the reverse order? Is that possible?"

You'll run into the same sorts of problems but they'll be worse. Files in C are inherently sequential lists of bytes that start at the beginning and go to the end; you're trying to work against this fundamental property and going against the fundamentals is never fun.

Do you really need your data in a plain text file? Maybe you need text/plain as the final output but all the way through? You could store the data in an indexed binary file (possibly even an SQLite database) and then you'd only have to worry about keeping (or windowing) the index in memory and that's unlikely to be a problem (and if it is, use a "real" database); then, when you have all your lines, just reverse the index and away you go.

How to read lines from a file in python starting from the end

The general approach to this problem, reading a text file in reverse, line-wise, can be solved by at least three methods.

The general problem is that since each line can have a different length, you can't know beforehand where each line starts in the file, nor how many of them there are. This means you need to apply some logic to the problem.

General approach #1: Read the entire file into memory

With this approach, you simply read the entire file into memory, in some data structure that subsequently allows you to process the list of lines in reverse. A stack, a doubly linked list, or even an array can do this.

Pros: Really easy to implement (probably built into Python for all I know)

Cons: Uses a lot of memory, can take a while to read large files

General approach #2: Read the entire file, store position of lines

With this approach, you also read through the entire file once, but instead of storing the entire file (all the text) in memory, you only store the binary positions inside the file where each line started. You can store these positions in a similar data structure as the one storing the lines in the first approach.

Whever you want to read line X, you have to re-read the line from the file, starting at the position you stored for the start of that line.

Pros: Almost as easy to implement as the first approach

Cons: can take a while to read large files

General approach #3: Read the file in reverse, and "figure it out"

With this approach you will read the file block-wise or similar, from the end, and see where the ends are. You basically have a buffer, of say, 4096 bytes, and process the last line of that buffer. When your processing, which has to move one line at a time backward in that buffer, comes to the start of the buffer, you need to read another buffer worth of data, from the area before the first buffer you read, and continue processing.

This approach is generally more complicated, because you need to handle such things as lines being broken over two buffers, and long lines could even cover more than two buffers.

It is, however, the one that would require the least amount of memory, and for really large files, it might also be worth doing this to avoid reading through gigabytes of information first.

Pros: Uses little memory, does not require you to read the entire file first

Cons: Much hard to implement and get right for all corner cases


There are numerous links on the net that shows how to do the third approach:

  • ActiveState Recipe 120686 - Read a text file backwards
  • ActiveState Recipe 439045 - Read a text file backwards (yet another implementation)
  • Top4Download.com Script - Read a text file backwards

Is there a way to reverse the order of lines within a text file using a function in python?

I think the bug you're referring to is in this function:

def reverse_word(line):
data = line.read()
data_1 = data[::-1]
print(data_1)
return data_1

You don't need to call read() on line because it's already a string; read() is called on file objects in order to turn them into strings. Just do:

def reverse_line(line):
return line[::-1]

and it will reverse the entire line.

If you wanted to reverse the individual words in the line, while keeping them in the same order within the line (e.g. turn "the cat sat on a hat" to "eht tac tas no a tah"), that'd be something like:

def reverse_words(line):
return ' '.join(word[::-1] for word in line.split())

If you wanted to reverse the order of the words but not the words themselves (e.g. turn "the cat sat on a hat" to "hat a on sat cat the"), that would be:

def reverse_word_order(line):
return ' '.join(line.split()[::-1])

Read a file line by line in reverse order

[EDIT]

By request, I am prepending this answer with the sentiment of a later comment: If you need this behavior frequently, a "more appropriate" solution is probably to move your logs from text files to database tables with DBAppender (part of log4j 2). Then you could simply query for latest entries.

[/EDIT]

I would probably approach this slightly differently than the answers listed.

(1) Create a subclass of Writer that writes the encoded bytes of each character in reverse order:

public class ReverseOutputStreamWriter extends Writer {
private OutputStream out;
private Charset encoding;
public ReverseOutputStreamWriter(OutputStream out, Charset encoding) {
this.out = out;
this.encoding = encoding;
}
public void write(int ch) throws IOException {
byte[] buffer = this.encoding.encode(String.valueOf(ch)).array();
// write the bytes in reverse order to this.out
}
// other overloaded methods
}

(2) Create a subclass of log4j WriterAppender whose createWriter method would be overridden to create an instance of ReverseOutputStreamWriter.

(3) Create a subclass of log4j Layout whose format method returns the log string in reverse character order:

public class ReversePatternLayout extends PatternLayout {
// constructors
public String format(LoggingEvent event) {
return new StringBuilder(super.format(event)).reverse().toString();
}
}

(4) Modify my logging configuration file to send log messages to both the "normal" log file and a "reverse" log file. The "reverse" log file would contain the same log messages as the "normal" log file, but each message would be written backwards. (Note that the encoding of the "reverse" log file would not necessarily conform to UTF-8, or even any character encoding.)

(5) Create a subclass of InputStream that wraps an instance of RandomAccessFile in order to read the bytes of a file in reverse order:

public class ReverseFileInputStream extends InputStream {
private RandomAccessFile in;
private byte[] buffer;
// The index of the next byte to read.
private int bufferIndex;
public ReverseFileInputStream(File file) {
this.in = new RandomAccessFile(File, "r");
this.buffer = new byte[4096];
this.bufferIndex = this.buffer.length;
this.in.seek(file.length());
}
public void populateBuffer() throws IOException {
// record the old position
// seek to a new, previous position
// read from the new position to the old position into the buffer
// reverse the buffer
}
public int read() throws IOException {
if (this.bufferIndex == this.buffer.length) {
populateBuffer();
if (this.bufferIndex == this.buffer.length) {
return -1;
}
}
return this.buffer[this.bufferIndex++];
}
// other overridden methods
}

Now if I want to read the entries of the "normal" log file in reverse order, I just need to create an instance of ReverseFileInputStream, giving it the "revere" log file.

Read a file backwards line by line using fseek

The question is asking using fseek, so can only assume that performance is an issue and file() is not the solution. Here is a simple approach using fseek:

My file.txt

#file.txt
Line 1
Line 2
Line 3
Line 4
Line 5

And the code:

<?php

$fp = fopen('file.txt', 'r');

$pos = -2; // Skip final new line character (Set to -1 if not present)

$lines = array();
$currentLine = '';

while (-1 !== fseek($fp, $pos, SEEK_END)) {
$char = fgetc($fp);
if (PHP_EOL == $char) {
$lines[] = $currentLine;
$currentLine = '';
} else {
$currentLine = $char . $currentLine;
}
$pos--;
}

$lines[] = $currentLine; // Grab final line

var_dump($lines);

Output:

array(5) {
[0]=>
string(6) "Line 5"
[1]=>
string(6) "Line 4"
[2]=>
string(6) "Line 3"
[3]=>
string(6) "Line 2"
[4]=>
string(6) "Line 1"
}

You don't have to append to the $lines array like I am, you can print the output straight away if that is the purpose of your script. Also it is easy to introduce a counter if you want to limit the number of lines.

$linesToShow = 3;
$counter = 0;
while ($counter <= $linesToShow && -1 !== fseek($fp, $pos, SEEK_END)) {
// Rest of code from example. After $lines[] = $currentLine; add:
$counter++;
}

Read File line by line in reverse order from specific line number using python

Try:

with open("test.txt","r") as f:
print(f.readlines()[:5][::-1])

# ['I am fine\n', 'You\n', 'Are\n', 'How\n', 'Hi\n']

Reading a file line by line backwards

Using a http-Request:

function httpGet(theUrl)
{
var xmlHttp = null;

xmlHttp = new XMLHttpRequest();
xmlHttp.open( "GET", theUrl, false );
xmlHttp.send( null );
return xmlHttp.responseText;
}

var content = httpGet("google.de"); //enter the Url of your text file here
var lines = content.split("\n");
var result = "";

for(var i=0; i<lines.length; i++)
result = lines[i] + "\n" + result;
alert(result);

Demo is here (displaying the source of http://google.de backwards)



Related Topics



Leave a reply



Submit