Simultaneous Read-Write a File in C#

simultaneous read-write a file in C#

Ok, two edits later...

This should work. The first time I tried it I think I had forgotten to set FileMode.Append on the oStream.

string test = "foo.txt";

var oStream = new FileStream(test, FileMode.Append, FileAccess.Write, FileShare.Read); 
var iStream = new FileStream(test, FileMode.Open, FileAccess.Read, FileShare.ReadWrite); 

var sw = new System.IO.StreamWriter(oStream);
var sr = new System.IO.StreamReader(iStream); 
var res = sr.ReadLine(); 
res = sr.ReadLine();
sw.WriteLine("g"); 
sw.Flush(); 
res = sr.ReadLine();
res = sr.ReadLine();
sw.WriteLine("h"); sw.Flush();
sw.WriteLine("i"); sw.Flush(); 
sw.WriteLine("j"); sw.Flush(); 
sw.WriteLine("k"); sw.Flush(); 
res = sr.ReadLine(); 
res = sr.ReadLine(); 
res = sr.ReadLine();
res = sr.ReadLine();
res = sr.ReadLine();
res = sr.ReadLine();

Handling concurrent reading/writing of a file

You can use a lock to make sure that a read is completed before a write and vice verser. As in:

using System;
using System.Threading;

class Program
{
    static readonly object _fileAccess = new object();

    static void Write()
    {
        // Obtain lock and write
        lock (_fileAccess)
        {
            // Write data to filename
            xmlDoc.Save(filename);
        }
    }

    static void Read()
    {
        // Obtain lock and read
        lock (_fileAccess)
        {
            // Read some data from filename
            xmlDoc.load(filename);
        }
    }

    static void Main()
    {       
        ThreadStart writeT = new ThreadStart(Write);
        new Thread(writeT).Start();

        ThreadStart readT = new ThreadStart(Read);
        new Thread(readT).Start();
    }
}

With the lock, the Read() must wait for the Write() to complete and Write() must wait for Read() to complete.

Read and Write to File at the same time

Simply holding a FileStream open with exclusive (not shared) access will prevent other processes from accessing the file. This is the default when opening a file for read/write access.

You can 'overwrite' a file that you currently hold open by truncating it.

So:

using (var file = File.Open("storage.bin", FileMode.Open))
{
    // read from the file

    file.SetLength(0); // truncate the file

    // write to the file
}

the method should be absolute safe, since the program should run on 2000 devices at the same time

Depending on how often you're writing to the file, this could become a chokepoint. You probably want to test this to see how scalable it is.

In addition, if one of the processes tries to operate on the file at the same time as another one, an IOException will be thrown. There isn't really a way to 'wait' on a file, so you probably want to coordinate file access in a more orderly fashion.

How can I simultaneously read/write form a text file using C#?

The actual problem is that you mix up access to the file.

You open a stream by buffer = new FileStream(this.bufferName, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite); in order to create a StreamReader from the opened stream for later reading by reader = new StreamReader(buffer);.

OTOH, you want get a StreamWriter for writing to the file by StreamWriter sw = File.AppendText(this.bufferName);. This tries to open the file again which fails because of mismatching file sharing mode.

So you need to access the file for writing and reading via the very same "handle" (here FileStream). Furthermore, don't forget to serialize access by some locking mechanism in order to make it thread-safe. Otherwise, you'll get corrupted data. You'll probably need to maintain the read/write pointer (Stream.Position).

Simultaneously read from one file and write to another

I should be able to almost double the speed by simultaneously reading and writing.

Unless you read the whole file in memory at once, doubling the speed is unlikely, because OS optimizes reading latency through readahead.

OS applies the principle of locality of reference: it quickly figures out that your program is reading sequentially, and uses read-ahead to prefetch the next few blocks. By the time your program is done writing and is ready for some reading again, the data is already in the buffer, so OS will serve it to your program. In the time your program spends waiting for the write to finish, the OS will prefectch some more data, so the cycle will continue with virtually no wait on the reading side.

As a matter of experiment you could try making your own program use asynchronous I/O by employing ReadAsync and WriteAsync APIs. The process would go as follows:

1. Before entering the loop, set Task pendingWrite = null
2. Start the read with ReadAsync and await its completion
3. If no additional data is available, exit the loop (step 8)
4. Process the data as needed
5. See if there is a pending write task. If there is, await its completion
6. Initiate the next write with WriteAsync, and assign the task to pendingWrite
7. Go back to step 2
8. await completion of pendingWrite task from the last cycle.

Simultaneously read and write binary file

You're opening the file twice - once for reading and once for writing. That means that one FileStream needs FileAccess.Read and FileShare.Write while the other needs FileAccess.Write and FileShare.Read. This code is tested and verified with a file that had already had an Integer and a String written to it with a BinaryWriter:

Dim filePath As String = Path.Combine(My.Computer.FileSystem.SpecialDirectories.MyDocuments, "Test.bin")

Using source = File.Open(filePath, FileMode.Open, FileAccess.Read, FileShare.Write),
      destination = File.Open(filePath, FileMode.Open, FileAccess.Write, FileShare.Read),
      reader As New BinaryReader(source),
      writer As New BinaryWriter(destination)
    Dim n = reader.ReadInt32()

    writer.Write(98765)
    writer.Write("What's up doc?")

    Dim sz = reader.ReadString()
End Using

Note that you should only specify Read or Write if that's all that's needed. Only specify ReadWrite if you know that you will or might need both. The FileAccess value is for what this FileStream will do or may do to the file while the FileShare value is for what other FileStream objects opened on the same file are allowed to do.

Concurrent file usage in C#

The correct way to do this is to open the file with a write lock (e.g., System.IO.FileAccess.Write, and a read share (e.g., System.IO.FileShare.Read). If one of the processes tries to open the file when the other process already has it open, then the open command will throw an exception, which you need to catch and handle as you see fit (e.g., log and retry). By using a write lock for the file open, you guarantee that the opening and locking are atomic and therefore synchronised between the two processes, and there is no race condition.

So something like this:

try
{
    using (FileStream fileStream = new FileStream(FileName, FileMode.Open, FileAccess.Write, FileShare.Read))
    {
        // Read from or write to file.
    }
}
catch (IOException ex)
{
    // The file is locked by the other process. 
    // Some options here:
    // Log exception.
    // Ignore exception and carry on.
    // Implement a retry mechanism to try opening the file again.
}

You can use FileShare.None if you do not want other processes to be able to access the file at all when your program has it open. I prefer FileShare.Read because it allows me to monitor what is happening in the file (e.g., open it in Notepad).

To cater for deleting the file is a similar principle: first rename/move the file and catch the IOException that occurs if the other process has already renamed it/moved it, then open the renamed/moved file. You rename/move the file to indicate that the file is already being processed and should be ignored by the other process. E.g., rename it with a .pending file extension, or move it to a Pending directory.

try
{
    // This will throw an exception if the other process has already moved the file - 
    // either FileName no longer exists, or it is locked.
    File.Move(FileName, PendingFileName);
    // If we get this far we know we have exclusive access to the pending file.
    using (FileStream fileStream = new FileStream(PendingFileName, FileMode.Open, FileAccess.Write, FileShare.Read))
    {
        // Read from or write to file.
    }
    File.Delete(PendingFileName);
}
catch (IOException ex)
{
    // The file is locked by the other process. 
    // Some options here:
    // Log exception.
    // Ignore exception and carry on.
    // Implement a retry mechanism to try moving the file again.
}

As with opening files, File.Move is atomic and protected by locks, therefore it is guaranteed that if you have multiple concurrent threads/processes attempting to move the file, only one will succeed and the others will throw an exception. See here for a similar question: Atomicity of File.Move.

Simultaneous Reading and Writing of a file in C#

To read and save the file:

StringBuilder newTextFile = new StringBuilder();
string[] lines = File.ReadAllLines(@"1.txt");
foreach (string l in lines)
{
   // logic to replace last number, saved in string newLine
   // you can find the last        
   newTextFile.Append(newLine + "\r\n");
}
File.WriteAllText(@"1.txt", newTextFile.ToString());

To change the line (off the top of my head, test it!):

int locationAt = line.LastIndexOf(',');
string newLine = line.Substring(0, locationAt) + newValue + line.Substring(locationAt);

OR:

string[] values = line.Split(',');
values[values.Length - 1] = 'somethingelse';
string newLine = string.Join(",", values);

---

Related Topics