Write to Text File from Multiple Threads

Write data into text file with multiple threads (simultaneously, in different lines of file)

When you write to file, you need to move the pointer to the correct position. You need to call "seek" method in RandomAccessFile and then move the pointer by number of bytes. For example first thread will seek to 0, second will seek to 21 and so on.

The way your program is right now, every thread will overwrite every other thread.

There may also be a problem with parallelization.

I didn't want to give a ready made solution, but I got curious. So here's something you could learn from

import java.io.IOException;
import java.io.RandomAccessFile;

public class Blah {
    // creates string before writing to the file
    public static String createString(int integer){
        StringBuilder result = new StringBuilder("");

        for(int i = 0; i < 20; i++){
            result.append(integer);
        }

        result.append("\n");
        return result.toString();
    }

    public static void main(final String[] args) throws IOException {
        RandomAccessFile file = new RandomAccessFile("part5.txt", "rw");
        Blah blah = new Blah();
        for(int i = 0; i <= 9; i++){
            Thread thread = new Thread(blah.new NumberWriter(i, file));
            thread.start();
        }

        file.close();
    }

    private class NumberWriter implements Runnable {
        int number;
        RandomAccessFile file;

        public NumberWriter(int number, RandomAccessFile file) {
            this.number = number;
            this.file = file;
        }


        public void run() {
            try {
                int seek = this.number * 20 + number;
                System.out.println("number is  : " + number + " : seeking to : " + seek);

                file.seek(seek);
                file.write((createString(this.number)).getBytes());
            } catch (IOException ioex) {
                ioex.printStackTrace();
            }
        }
    }
}

Write to text file from multiple threads?

Access the file through a class that contains a synchronized method to write to the file. Only one thread at a time will be able to execute the method.

I think that Singleton pattern would fit for your problem:

package com.test.singleton;

public class Singleton {
    private static final Singleton inst= new Singleton();
    
    private Singleton() {
        super();
    }
    
    public synchronized void writeToFile(String str) {
        // Do whatever
    }
    
    public static Singleton getInstance() {
        return inst;
    }
    
}

Every time you need to write to your file, you only would have to call:

Singleton.getInstance().writeToFile("Hello!!");

Writing to a file from multiple threads

Move the join inside the file.open block:

threads = []
File.open("test.txt", "a") do |fp|
  500.times do |time|
    threads << Thread.new do
      fp.puts("#{time}: 1")
      sleep(rand(100) / 100.0)
      fp.puts("#{time}: 2")
    end
  end

  threads.each{ |thread| thread.join }
end

Why? Thread.new launches the thread, but it runs in parallel, and the life of the thread in your version isn't guaranteed to be shorter than the life of the file. File.open closes the file after you exit the attached block. By waiting to close the file until after all the threads are done, everything will work as expected.

However, please note that this IS NOT thread safe on JRuby (or any other implementation without a GIL) and may have output intermixed:

6: 1
5: 17: 1
8: 1

3: 10: 110: 1
4: 11: 1
2: 19: 1

11: 1



12: 1
13: 1
14: 1

Note: this question appears to be from Ruby MRI 1.8.7 - File writing thread safety

How do I output from multiple threads to a .txt file?

Writing into file from multiple threads is a bad idea. I suggest you create a queue (even if just in memory queue) and have all your threads writing the info that they want to write into your file into this queue. In other words your queue will have multiple producers. And than have a single consumer on your queue that will read from the queue and write it into your file. This way you will have only one thread writing into file

Write to a file from multiple threads asynchronously c#

Have a look at Asynchronous I/O. This will free up the cpu to continue with other tasks.

Combine with ReaderWriterLock as @Jack B Nimble mentioned

If by

writing to the file system as
efficient as possible

you mean making the actual file I/O as fast as possible you are going to have a hard time speeding it up much, disk is just physically slower. Maybe SSD's?

c# Writing text to one text file from multiple threads

I am not sure how your locking process works, but if you replace

new FileStream(logFilePath, FileMode.Append)

with

new FileStream(logFilePath, FileMode.Append, FileAccess.Write, FileShare.ReadWrite)

I would imagine that your lock_ is unnecessary.

How to write in a single file with multiple threads?

As mentioned in the comments, this is asking for trouble.

So, you need to have a thread-safe writer class:

public class FileWriter
{
    private ReaderWriterLockSlim lock_ = new ReaderWriterLockSlim();
    public void WriteData(/*....whatever */)
    {
        lock_.EnterWriteLock();
        try
        {
            // write your data here
        }
        finally
        {
            lock_.ExitWriteLock();
        }
    }

} // eo class FileWriter

This is suitable for being called by many threads. BUT, there's a caveat. There may well be lock contention. I used a ReadWriterLockSlim class, because you may want to do read locks as well and hell, that class allows you to upgrade from a read state also.

---

Related Topics