Sigbus While Doing Memcpy from Mmap Ed Buffer Which Is in Ram as Identified by Mincore

Valid read from mmap-ed memory produces SIGBUS under load. Why?

With your current program, it can happen that thread 0 creates /tmp/buf_0, writes to it and closes it. Then thread 1 removes and creates /tmp/buf_0, but before thread 1 writes to it, thread 0 opens, maps, and reads from /tmp/buf_0 - and thus tries to access a file does not yet contain 64 kiB data. You get a SIGBUS.

To avoid that issue, just make unique files / and bufs for each thread, by using omp_get_thread_num() instead of bufIdx.

Bus Error when writing to mmaped data

A documented cause for SIGBUS with mmap is

Attempted access to a portion of the buffer that does not correspond to the file (for example, beyond the end of the file, including the case where another process has truncated the file).

My guess is that the accounts file didn't exist, so open with O_CREAT created it. But it has zero size, so any attempt to read or write through the mapping will fault. You need to fill the file with enough zeroes (or something else) to cover the mapping, for example using ftruncate.

rrdtool gives bus error when an attempt is made to create an rrd database

This problem may be related to some mmap bug of your old Centos distribution.

If upgrading to a newer version is not an option, you could try to compile your rrdtool with the following option:

./configure --disable-mmap

By doing so, the faulty memcpy at line 716 in the rrd_write() function (shown below) should not be executed:

/* Write count bytes from buffer buf to the current position
 * rrd_file->pos of rrd_simple_file->fd.
 * Returns the number of bytes written or <0 on error.  */

ssize_t rrd_write(
    rrd_file_t *rrd_file,
    const void *buf,
    size_t count)
{
    rrd_simple_file_t *rrd_simple_file = (rrd_simple_file_t *)rrd_file->pvt;
#ifdef HAVE_MMAP
    size_t old_size = rrd_file->file_len;
    if (count == 0)
        return 0;
    if (buf == NULL)
        return -1;      /* EINVAL */

    if((rrd_file->pos + count) > old_size)
    {
        rrd_set_error("attempting to write beyond end of file");
        return -1;
    }
    memcpy(rrd_simple_file->file_start + rrd_file->pos, buf, count);
    rrd_file->pos += count;
    return count;       /* mimmic write() semantics */
#else
    ssize_t   _sz = write(rrd_simple_file->fd, buf, count);

    if (_sz > 0)
        rrd_file->pos += _sz;
    return _sz;
#endif
}

How to work around lack of NUL terminator in strings returned from mmap()?

I would suggest undergoing a paradigm shift here.

You're looking at the entire universe consisting of '\0'-delimited strings that define your text. Instead of looking at the world this way, why don't you try looking at the world where text is defined as a sequence defined by a beginning and an ending iterator.

You mmap your file, then initially set the beginning iterator, call it beg_iter to the start of the mmap-ed segment, and the ending iterator, call it end_iter, to the first byte following the last byte in the mmap-ed segment, or beg_iter+number_of_pages*pagesize, then until either

A) end_iter equals beg_iter, or

B) beg_iter[-1] is not a null character, then

C) decrement end_iter, and go back to step A.

When you're done, you have a pair of iterators, the beginning iterator value, and the ending iterator value that define your text string.

Of course, in this case, your iterators are plain char *, but that's really not very important. What is important is that now you find yourself with a rich set of algorithms and templates from the C++ standard library at your disposal, that let you implement many complicated operations, both mutable (like std::transform), and non-mutable, (like std::find).

Null-terminated strings are really a holdover from the days of plain C. With C++, null-terminated strings are somewhat archaic, and mundane. Modern C++ code should use std::string objects, and sequences defined by beginning and ending iterators.

One small footnote: instead of figuring out how much NULL padding you ended up mmap-ing(), you might find it easier to fstat() the file, and get the file's exact length, in bytes, before mmap-ing it. Then you'll now exactly know much got mmaped, and you don't have to reverse-engineer it, by looking at the padding.

What is a bus error? Is it different from a segmentation fault?

Bus errors are rare nowadays on x86 and occur when your processor cannot even attempt the memory access requested, typically:

• using a processor instruction with an address that does not satisfy its alignment requirements.

Segmentation faults occur when accessing memory which does not belong to your process. They are very common and are typically the result of:

• using a pointer to something that was deallocated.
• using an uninitialized hence bogus pointer.
• using a null pointer.
• overflowing a buffer.

PS: To be more precise, it is not manipulating the pointer itself that will cause issues. It's accessing the memory it points to (dereferencing).

using mmap to copy a file

Mapping two files to the same memory region is problematic. What should the contents of this memory be, data from the first file or the second, or a mix? This won't work.

What you can do is map two files and memcpy from one mapped region to the other. Note, however, that it is a good idea to create the file first and set its length, otherwise mmap may return SIGBUS (see docs).

SIGBUS Attempted access to a portion of the buffer that does not
correspond to the file (for example, beyond the end of the file,
including the case where another process has truncated the file).

C Bus Error with mmap, what is the problem

You can't write to a non-existent portion of a file via mmap().

This truncates the file:

int dfd = open(comms->destination, O_CREAT | O_RDWR | O_TRUNC, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);

This mmap()'s the truncated file:

void* destination = mmap(NULL, fileSize, PROT_READ | PROT_WRITE, MAP_PRIVATE,  dfd, 0);

This tries to copy to a non-existent part of the file:

memcpy(destination, content, fileSize);

Per the Linux mmap() man page ERRORS:

ERRORS

...

Use of a mapped region can result in these signals:

...

SIGBUS Attempted access to a portion of the buffer that does not
correspond to the file (for example, beyond the end of the
file, including the case where another process has truncated
the file).

Per the POSIX mmap() specification:

... References within the address range starting at pa and continuing for len bytes to whole pages following the end of an object shall result in delivery of a SIGBUS signal.

An implementation may generate SIGBUS signals when a reference would cause an error in the mapped object, such as out-of-space condition.

The fix is to call ftruncate() to set the file length on the output file:

ftruncate( dfd, fileSize );
void* destination = mmap(NULL, fileSize, PROT_READ | PROT_WRITE, MAP_SHARED,  dfd, 0);

Note that you also have to replace the MAP_PRIVATE flag with MAP_SHARED. Per POSIX again (bolding mine):

MAP_SHARED and MAP_PRIVATE describe the disposition of write references to the memory object. If MAP_SHARED is specified, write references shall change the underlying object. If MAP_PRIVATE is specified, modifications to the mapped data by the calling process shall be visible only to the calling process and shall not change the underlying object. It is unspecified whether modifications to the underlying object done after the MAP_PRIVATE mapping is established are visible through the MAP_PRIVATE mapping. Either MAP_SHARED or MAP_PRIVATE can be specified, but not both. The mapping type is retained across fork().

---

Related Topics