How to Modify Lines in a File In-Place

Is it possible to modify lines in a file in-place?

Is it possible to parse a file line by line, and edit a line in-place while going through the lines?

It can be simulated using a backup file as stdlib's fileinput module does.

Here's an example script that removes lines that do not satisfy some_condition from files given on the command line or stdin:

#!/usr/bin/env python
# grep_some_condition.py
import fileinput

for line in fileinput.input(inplace=True, backup='.bak'):
    if some_condition(line):
        print line, # this goes to the current file

Example:

$ python grep_some_condition.py first_file.txt second_file.txt

On completion first_file.txt and second_file.txt files will contain only lines that satisfy some_condition() predicate.

sed edit file in place

The -i option streams the edited content into a new file and then renames it behind the scenes, anyway.

Example:

sed -i 's/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g' filename

while on macOS you need:

sed -i '' 's/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g' filename

editing file lines in place python 2.6

but for the files that doesn't have the pattern it deletes their contents completely

Yes because that's how fileinput.input works. You need to print all lines, no matter whether you changed them or not.

for line in fileinput.input(os.path.join(root, fname), inplace=1): 
    if re.search(r"([\w-d])", line):
        x=re.sub(r"([\w-d])", r"(\1).", line)
        print line.replace(line, x)
    else:
        print line

Also consider using sys.stdout.write(line) as print adds a new line. Whereas line is the line read from the file including the new line at the end.

Thus if we have a file called test.txt with the following contents:

a1
b2
c3

Then if we do this:

for line in fileinput.input("test.txt", inplace=1):
    if line.strip() == "b2":
        sys.stdout.write("-> ")
        sys.stdout.write(line)
    else:
        sys.stdout.write(line)

Then after that the file will look like this:

a1
-> b2
c3

So you do need to write unchanged lines as well.

Edit:

The most flexible is probably to do it like that. However you could read the file beforehand to check if the pattern exist and then do the same:

f = open(os.path.join(root, fname), "r")
found = False
for line in f:
    if re.search(r"([\w-d])", line):
        found = True
        break
f.close()

if found:
    for line in fileinput.input(os.path.join(root, fname), inplace=1): 
        if re.search(r"([\w-d])", line):
            x=re.sub(r"([\w-d])", r"(\1).", line)
            print line.replace(line, x)
        else:
            print line

editing a specific line of a file in C

For fopen modes see http://www.cplusplus.com/reference/cstdio/fopen/. I think you need to use the option r+ because you are modifying the file in a random-access way fior both read and write.

"r+" read/update: Open a file for update (both for input and output).
The file must exist.

"w+" write/update: Create an empty file and open
it for update (both for input and output). If a file with the same
name already exists its contents are discarded and the file is treated
as a new empty file.

"a+" append/update: Open a file for update (both
for input and output) with all output operations writing data at the
end of the file. Repositioning operations (fseek, fsetpos, rewind)
affects the next input operations, but output operations move the
position back to the end of file. The file is created if it does not
exist.

I would suggest storing the number of lines in the file as an unsigned integer and not a string. The reason is that as a string 0-9 lines take one byte, but the minute you have 10 lines you need two byes, 100, 3 bytes and so on. In each case when an extra character is required you would have to re-write the entire file. I presume this is why you check that number of scores is less than 10.

A better solution would be to keep the first 4 bytes of the file as an unsigned integer and then start the ascii text after.

int      result;
uint32_t number_of_scores;
size_t   bytesRead;
FILE     *data;

...

/* Open a file for update (both for input and output).
* The file must exist. */
data = fopen("highscore.txt","r+");
if( !data ) 
    exit(SOME_ERROR_CODE);

/* Read a 32-bit unsigned integer from the file. NOTE there is no endianess
 * "protection" here... ignoring this issue for the sake of simplicity and relevance */
bytesRead = fread (&number_of_scores, sizeof(number_of_scores), 1, data);   
if( bytesRead != 1 )
    exit(SOME_ERROR_CODE);

/* Seek to end of file */
result = fseek(data, 0, SEEK_END);    
if( result )
    exit(SOME_ERROR_CODE);

/* Write in the next line */
result = fprintf(data,
                 "%s  %s  %s  %s\n", 
                 user[current_user].name,
                 user[current_user].age, 
                 user[current_user].college,
                 resultVariableRenamedToAvoidNameCollision);

 /* Up the number of scores and write it back to the start of the file */
 number_of_scores++;
 result = fseek(data, 0, SEEK_SET);      
 if( result )
     exit(SOME_ERROR_CODE);

 bytesRead = fwrite (data, sizeof(number_of_scores), 1, data);
 if( bytesRead != 1 )
    exit(SOME_ERROR_CODE);

 fclose(data);

Doh, and I've just realised how late this answer is... never mind :S

Editing specific line in text file in Python

You want to do something like this:

# with is like your try .. finally block in this case
with open('stats.txt', 'r') as file:
    # read a list of lines into data
    data = file.readlines()

print data
print "Your name: " + data[0]

# now change the 2nd line, note that you have to add a newline
data[1] = 'Mage\n'

# and write everything back
with open('stats.txt', 'w') as file:
    file.writelines( data )

The reason for this is that you can't do something like "change line 2" directly in a file. You can only overwrite (not delete) parts of a file - that means that the new content just covers the old content. So, if you wrote 'Mage' over line 2, the resulting line would be 'Mageior'.

How to edit a text file at a specific line or location

I'd use regular expressions to match and replace text inside your file

import re

def search_write_in_file(file_name, string_to_search, description):
    with open(file_name, 'r+') as file_obj:
        text = file_obj.read()
    new_text = re.sub(string_to_search,r"\1 {0}\n".format(description),text)
    with open(file_name, 'w') as file_obj:
        file_obj.write(new_text)
    print(new_text)

if __name__ == '__main__':
    search_write_in_file('text_to_edit.txt',r'(DATATYPE2;\n)',"test2")
    search_write_in_file('text_to_edit.txt',r'(DATATYPE4;\n)',"test4")

This will update the existing file to be

VAR_GROUP
Var1 : DATATYPE1;(Description Var)
Var2 : DATATYPE2; test2
Var3 : DATATYPE3;(Description Var3)
Var4 : DATATYPE4; test4
END_GROUP

In place replacement of text in a file in Python

1) The code adds the replaced text at the end and the text in original place is unchanged.

You can't replace in the body of the file because you're opening it with the + signal. This way it'll append to the end of the file.

file = open('example.php','rb+')

But this only works if you want to append to the end of the document.

To bypass this you may use seek() to navigate to the specific line and replace it. Or create 2 files: an input_file and an output_file.

---

2) Also, instead of just the replaced text, it prints out the whole line.

It's because you're using:

file.write( line.replace('Original', 'Replacement'))

---

Free Code:

I've segregated into 2 files, an inputfile and an outputfile.

First it'll open the ifile and save all lines in a list called lines.

Second, it'll read all these lines, and if 'Original' is present, it'll replace it.

After replacement, it'll save into ofile.

ifile = 'example.php'
ofile = 'example_edited.php'

with open(ifile, 'rb') as f:
    lines = f.readlines()

with open(ofile, 'wb') as g:
    for line in lines:
        if 'Original' in line:
            g.write(line.replace('Original', 'Replacement'))

Then if you want to, you may os.remove() the non-edited file with:

---

More Info: Tutorials Point: Python Files I/O

Modifying a particular line in a text file from the command-line

A simple sed substitute will also work given the line number, e.g.

sed -i '2s/^/# /' file

The above will edit file in place commenting line 2 inserting "# " at the beginning of the line. To pass in the line number as an argument, double-quote the expression, e.g.

#!/bin/bash

sed -i "$1s/^/# /" "$2"

Will take the line number as the first argument and the file to comment as the second and edit the file in-place making the comment. You should validate the first argument is an integer and that the second is a file that exists and is non-empty. If you save the script as cmt and make it executable (e.g. chmod +x cmt), then you would comment the 2nd line in file as:

./cmt 2 file

how to edit or modify or change a single line in a large text file with R

Why don't you edit just the header, and then read the rest in chunks? I don't know how big this file is, but perhaps in blocks of lines (I've guessed 10000). Depending on how much memory you have you can adjust this to be bigger or smaller.

##setup
tf <- tempfile(); tf2 <- tempfile()
write.csv(mtcars,tf)

fr <- file(tf, open="rt") #open file connection to read
fw <- file(tf2, open="wt") #open file connection to write 
header <- readLines(f,n=1) #read in header
header <- gsub( 'disp' , 'newvar' , header) #modify header    
writeLines(header,con=fw) #write header to file
while(length(body <- readLines(fr,n=10000)) > 0) {
  writeLines(body,fw) #pass rest of file in chunks of 10000
}
close(fr);close(fw) #close connections
#unlink(tf);unlink(tf2) #delete temporary files

It should be faster because R will run through the while loop every 10000 lines instead of every single line. Additionally, R will call gsub on just the line you want, instead of every line, saving you R time. R can't edit a file "in-place", so to speak, so there is no way around reading and copying the file. If you have to do it in R, then make your chunks as big as memory allows and then pass your file through.

I saw a 3x performance difference between the two ways:

#test file creation ~3M lines
tf <- tempfile(); tf2 <- tempfile()
fw <- file(tf,open="wt")
sapply(1:1e6,function(x) write.csv(mtcars,fw))
close(fw)

#my way
system.time({
fr <- file(tf, open="rt") #open file connection to read
fw <- file(tf2, open="wt") #open file connection to write 
header <- readLines(f,n=1) #read in header
header <- gsub( 'disp' , 'newvar' , header) #modify header    
writeLines(header,con=fw) #write header to file
while(length(body <- readLines(fr,n=10000)) > 0) {
  writeLines(body,fw) #pass rest of file in chunks of 10000
}
close(fr);close(fw) #close connections
})    
#   user  system elapsed 
#  32.96    1.69   34.85 

#OP's way
system.time({
incon <- file( tf , "r" )
outcon <- file( tf2 , "w" )
while( length( one.line <- readLines( incon , 1 ) ) > 0 ){
    one.line <- gsub( 'disp' , 'newvar' , one.line )
    writeLines( one.line , outcon )
}
close( incon ) ; close( outcon )
})
#   user  system elapsed 
# 104.36    1.92  107.03 

---

Related Topics