How to Read Files in an Eventmachine-Based App

What is the best way to read files in an EventMachine-based app?

EM.attach/watch cannot be used on files, as select/epoll on a disk-based file descriptor will always return readable.

Ultimately, it depends on what you're trying to do. If it's a small file, just File.read it. If it is larger, you can read small chunks over time. For example, EM::FileStreamer does this to send large file over the network.

Another common use-case is to tail a file and read in new contents when it changes. This can be achieved using EM.watch_file: http://github.com/jordansissel/eventmachine-tail

Read file in EventMachine asynchronously

There is no asynchronous file IO support in EventMachine, the best way to achieve what you're trying to do is to read a couple of lines on each tick and send them off to the database. The most important is to not read too large chunks since that would block the reactor.

EM.run do
  io = File.open('path/to/file')
  read_chunk = proc do
    lines_sent = 10
    10.times do
      if line = io.gets
        send_to_db(line) do
          # when the DB call is done
          lines_sent -= 1
          EM.next_tick(read_chunk) if lines_sent == 0
        end
      else
        EM.stop
      end
    end
  end
  EM.next_tick(read_chunk)
end

See What is the best way to read files in an EventMachine-based app?

Show files on FTP in a Rails App

You can use library links below:

http://www.ruby-doc.org/stdlib-1.9.3/libdoc/net/ftp/rdoc/index.html

http://oreilly.com/catalog/ruby/chapter/ch04.html

and you can use EventMachine https://github.com/schleyfox/em-ftp-client.

Cheers!

How can I communicate to a Sinatra web app over a Unix socket?

I haven't done it personally but I know Event Machine supports working with unix socket. The advantage of event machine is that you can have code listening to the socket without 'interfering' with sinatra web server. So you can for example listen to socket for some statistics, store those statistics in memory and then return them to clients on HTTP requests.

And if you are going for EventMachine I suggest using thin as your webserver which is implemented on top of it so you don't need to start it manually.

How do I log asynchronous thin+sinatra+rack requests?

I eventually found that using rack-async with async_sinatra was causing problems with 404 pages, exception handling, and logging:

!! Unexpected error while processing request: undefined method `bytesize' for nil:NilClass

Instead I used the following wrapper around aroute for logging:

module Sinatra::Async
    alias :oldaroute :aroute
    def aroute verb, path, opts = {}, &block
        # Based on aroute from async_sinatra

        run_method = :"RunA#{verb} #{path} #{opts.hash}"
        define_method run_method, &block

        log_method = :"LogA#{verb} #{path} #{opts.hash}"
        define_method(log_method) { |*a|
            puts "#{request.ip} - #{status} #{verb} #{path}"
        }

        oldaroute verb, path, opts do |*a|
            oldcb = request.env['async.callback']
            request.env['async.callback'] = proc { |*args|
                async_runner(log_method, *a)
                oldcb[*args]
            }
            async_runner(run_method, *a)
        end
    end
end

This is for the same versions of async_sinatra, Thin, and Rack that I was using when I asked this question last year; newer versions may allow the use of common Rack middleware for logging.

---

Related Topics