Where Does Ruby Keep Track of Its Open File Descriptors

Where does Ruby keep track of its open file descriptors?

TL; DR

All File and IO objects are stored in ObjectSpace.

Answer

The ObjectSpace class says:

The ObjectSpace module contains a number of routines that interact with the garbage collection facility and allow you to traverse all living objects with an iterator.

How I Tested This

I tested this at the console on Ruby 1.9.3p194.

The test fixture is really simple. The idea is to have two File objects with different object identities, but only one is directly accessible through a variable. The other is "out there somewhere."

# Don't save a reference to the first object.
filename='/tmp/foo'
File.open(filename)
filehandle = File.open(filename)

I then explored different ways I could interact with the File objects even if I didn't use an explicit object reference. This was surprisingly easy once I knew about ObjectSpace.

# List all open File objects.
ObjectSpace.each_object(File) do |f|
  puts "%s: %d" % [f.path, f.fileno] unless f.closed?
end

# List the "dangling" File object which we didn't store in a variable.
ObjectSpace.each_object(File) do |f|
  unless f.closed?  
    printf "%s: %d\n", f.path, f.fileno unless f === filehandle
  end
end

# Close any dangling File objects. Ignore already-closed files, and leave
# the "accessible" object stored in *filehandle* alone.
ObjectSpace.each_object(File) {|f| f.close unless f === filehandle rescue nil}

Conclusion

There may be other ways to do this, but this is the answer I came up with to scratch my own itch. If you know a better way, please post another answer. The world will be a better place for it.

How to monitor open file descriptors in Ruby on Rails?

One method to consider (probably the simplest) is using a background worker of some sort, such as with Workling, and making it run lsof in intervals, and getting output using syntax:

`lsof | grep something` # shell command example.

Programs like lsof can really hurt performance if run too frequently. Perhaps every 10s to 30s. Perhaps down to maybe 5s, but that's really pushing it. I'm assuming you have a dedicated server or a beasty virtual machine.

In your background worker, you can store these command results into a variable, or grep it down to what you're really looking for (as demonstrated), and access/manipulate the data as you please.

Fork, Ruby, ActiveRecord and File Descriptors on Fork

If you close a file descriptor in one process it stays valid in the other process, this is why your file example works fine.

The mysql case is different because it's a socket with another process at the end. When you call close on the mysql adapter (or when the adapter gets garbage collected when ruby exits) it actually sends a "QUIT" command to the server saying that you're disconnecting, so the server tears down its side of the socket. In general you really don't want to share a mysql connection between two processes - you'll get weird errors depending on whether the two processes are trying to use the socket at the same time.

If closing a redis connection just closes the socket (as opposed to sending a "I'm going away " message to the server) then the child connection should continue to work because the socket won't actually have been closed

Ruby's File.open and the need for f.close

I saw many times in ruby codes unmatched File.open calls

Can you give an example? I only ever see that in code written by newbies who lack the "common knowledge in most programming languages that the flow for working with files is open-use-close".

Experienced Rubyists either explicitly close their files, or, more idiomatically, use the block form of File.open, which automatically closes the file for you. Its implementation basically looks something like like this:

def File.open(*args, &block)
  return open_with_block(*args, &block) if block_given?
  open_without_block(*args)
end

def File.open_without_block(*args)
  # do whatever ...
end

def File.open_with_block(*args)
  yield f = open_without_block(*args)
ensure
  f.close
end

Scripts are a special case. Scripts generally run so short, and use so few file descriptors that it simply doesn't make sense to close them, since the operating system will close them anyway when the script exits.

Do we need to explicitly close?

Yes.

If yes then why does the GC autoclose?

Because after it has collected the object, there is no way for you to close the file anymore, and thus you would leak file descriptors.

Note that it's not the garbage collector that closes the files. The garbage collector simply executes any finalizers for an object before it collects it. It just so happens that the File class defines a finalizer which closes the file.

If not then why the option?

Because wasted memory is cheap, but wasted file descriptors aren't. Therefore, it doesn't make sense to tie the lifetime of a file descriptor to the lifetime of some chunk of memory.

You simply cannot predict when the garbage collector will run. You cannot even predict if it will run at all: if you never run out of memory, the garbage collector will never run, therefore the finalizer will never run, therefore the file will never be closed.

What does it mean to open a file in Ruby by using File.new?

The File.new method executes a call to IO::new (docs here).

The thing being "opened" in this case is an input/output stream which Ruby tracks using file descriptors. These file descriptors can be expensive to keep around which is why its good practice to call the close method on any instances of File or IO.

C get all open file descriptors

I'd use brute force: for (i = 0; i < fd_max; ++i) close (i);. Quick and pretty portable.

Ruby: how to check if a file is still open?

Reasonable (=all) OSs close fds automatically when a program ends, no matter what the code does.

As for ruby code trying to be nice during execution, Ruby has finalizers that autoclose when a variable that used to reference a filedescriptor (or an encapsulated filedescriptor) gets garbage-collected.

On a UNIX system, you can check open files with lsof.

The code below demonstrates the concepts:

rb.rb:

def func()
    fd = IO.sysopen("file.txt", "a")
    myios = IO.new(fd)
    myios.puts "new line"
end
func
sleep 3 #most likely open here unless the GC managed to run
GC.start #should be closed after this point
sleep 3

Now if you invoke it with:

$ ruby rb.rb  & pid=$!; while kill -0 $pid; do if lsof -p $pid | grep -q file.txt; then echo open; else echo closed; fi; sleep 0.3; done

you'll probably get one "closed" (before the ruby code catches up), 3 seconds of open and then 3 seconds of closed.

If you don't want to rely on finalizers (which are will run at indeterministic times, because they rely on the garbage collecter), then the block syntax for opening files is really nice in ruby -- the end of the block will deterministically close the file at the very point where the block ends.

What do file descriptor 3 and 4 stand for in Ruby?

From Standard Input, Output, & Error :

When it is started by the shell, a program inherits three open files, with file descriptors 0, 1, and 2, called the standard input, the standard output, and the standard error. All of these are by default connected to the terminal, so if a program only reads file descriptor 0 and writes file descriptors 1 and 2, it can do I/O without having to open files. If the program opens any other files, they will have file descriptors 3, 4, etc.

Update

$stdin.fileno    # => 0
$stdout.fileno   # => 1
$stderr.fileno # => 2
File.open('test1').fileno # => 7
File.open('test2').fileno # => 8
File.open('test.txt').fileno # => 9

Now lets try to read the filename from the file descriptors using for_fd method.

File.for_fd(7) # => #<File:fd 7> # refers file test1
File.for_fd(8) # => #<File:fd 8> # refers file test2
File.for_fd(9) # => #<File:fd 9> # refers file test.txt

Opps!, these are not possible, as file descriptors 3,4,5,6 are used by RubyVM.

File.for_fd(3) # => 
# The given fd is not accessible because RubyVM reserves it (ArgumentError)
File.for_fd(4) # => 
# The given fd is not accessible because RubyVM reserves it (ArgumentError)
File.for_fd(5) # => 
# The given fd is not accessible because RubyVM reserves it (ArgumentError)
File.for_fd(6) # => 
# The given fd is not accessible because RubyVM reserves it (ArgumentError)

Note : My Ruby version is - 2.0.0-p451 in openSUSE13.1.

How to check if the file is still locked by current thread?

If f.flock(File::LOCK_EX | File::LOCK_NB) returns non false value then f IS locked. It will keep the lock until you close the file or explicitly call f.flock(File::LOCK_UN). You don't have to check whether it is locked again. To explain what really happens there we need to look into a file system internals and related system calls first:

 File Descriptor Table       Open File Table        i-node Table      Directory Index
╒════════════════════╕       ╒═════════════╕       ╒════════════╕     ╒═════════════╕
┃3 ..................┣━━━━━━▷┃ open file1  ┣━━┳━━━▷┃ /tmp/file1 ┃◃━━━━┫ file1       ┃
┃4 ..................┣━━━━━━▷┃ open file1  ┣━━┚ ┏━▷┃ /tmp/file2 ┃◃━━━━┫ file2       ┃
┃5 ..................┣━━━┳━━▷┃ open file2  ┣━━━━┚                   
┃6 ..................┣━━━┚

The key point in this diagram is that there are two different and unrelated entry points into the i-node Table: Open File Table and Directory Index. Different system calls work with different entry points:

open(file_path) => finds i-node number from Directory Index and creates an entry in Open File Table referenced by File Descriptor Table (one table per process), then increments ref_counter in the related i-node Table entry.
close(file_descriptor) => closes (frees) related File Descriptor Table entry and related entry from Open File Table (unless there are other referencing File Descriptors), then decrements ref_counter in related i-node Table entry (unless Open File entry stays open)
unlink(file_path) => there is no Delete system call! Unlinks i-node Table from Directory Index by removing entry from Directory Index. Decrements counter in the related i-node Table entry (unaware of Open File Table!)
flock(file_desriptor) => apply/remove lock on entries in Open File Table (unaware of Directory Index!)
i-node Table entry is removed (practically deleting a file) IFF ref_counter becomes Zero. It can happen after close() or after unlink()

The key point here is that unlink not necessarily deletes a file(the data) immediately! It only unlinks Directory Index and i-node Table. It means that even after unlink the file may still be open with active locks on it!

Keeping that in mind, imagine the following scenario with 2 threads, trying to synchronise on a file using open/flock/close and trying to cleanup using unlink:

   THREAD 1                              THREAD 2
==================================================
       |                                    |
       |                                    |
(1) OPEN (file1, CREATE)                    |
       |                             (1) OPEN (file1, CREATE)
       |                                    |
(2) LOCK-EX (FD1->i-node-1)                 |
  [start work]                       (2) LOCK-EX (FD2->i-node-1) <---
       |                                    .                       |
       |                                    .                       |
(3)  work                                   .                       |
       |                             (3) waiting loop               |
       |                                    .                       |
   [end work]                               .                       |
(4) UNLINK (file1)                          . -----------------------
(5) CLOSE (FD1)--------unlocked------> [start work]
       |                                    |
       |                                    |
(6) OPEN (file1, CREATE)                    |
       |                                    |
       |                             (5)  work
(7) LOCK-EX (FD1->i-node-2)                 |
  [start work] !!! does not wait            |
       |                                    |
(8)  work                                   |
       |                                    |

(1) both threads open(potentially create) the same file. As a result there is a link from Directory Index to i-node Table. Each thread gets its own File Descriptor.
(2) both threads try to get an exclusive lock using File Descriptor they get from an open call
(3) first thread gets a lock and second thread is blocked (or is trying to get a lock in a loop)
(4) first thread finishes a task and deletes (unlink) a file. At this point link from Directory Index to i-node is removed and we won't see it in the directory listing. BUT, the file is still there and is open in two threads with an active lock! It simply lost its name.
(5) first thread closes File Descriptor and as a result releases a lock. Thus second thread gets a lock and starts working on a task
(6) first thread repeats and tries to open a file with the same name. But is it the same file as before? No. Because at this point there is no file with a given name in Directory Index. So it creates a NEW file instead! new i-node Table entry.
(7) first thread gets a lock on a NEW file!
(8) and we get two threads with a lock on two different files and UNsynchronised

The problem in the above scenario is that open/unlink work on Directory Index, while lock/close work on File Descriptors, which are not related to each other.

To solve this issue we need to synchronise these operations through some central entry point. It can be implemented by introducing a singleton service which will provide this synchronisation using a Mutex or primitives from Concurrent Ruby.

Here is one possible PoC implementation:

class FS
  include Singleton

  def initialize
    @mutex = Mutex.new
    @files = {}
  end

  def open(path)
    path = File.absolute_path(path)
    file = nil
    @mutex.synchronize do
      file = File.open(path, File::CREAT | File::RDWR)
      ref_count = @files[path] || 0
      @files[path] = ref_count + 1
    end

    yield file
  ensure
    @mutex.synchronize do
      file.close
      ref_count = @files[path] - 1
      if ref_count.zero?
        FileUtils.rm(path, force: true)
        @files.delete(path)
      else
        @files[path] = ref_count
      end
    end
  end
end

And here is your re-written example from the question:

FS.instance.open('a.txt') do |f|
  if f.flock(File::LOCK_EX | File::LOCK_NB)
    # you can be sure that you have a lock
  end
  # 'a.txt' will finally be deleted
end

Where Does Ruby Keep Track of Its Open File Descriptors