How to Force a Range of Virtual Addresses

Force memory allocation always to the same virtual address

You could try mmap(2).

The instrumented version of your program will use a different memory layout than the original program because pin needs memory for the dynamic translation etc. and will change the memory layout. (if I recall correctly)

The range of virtual memory address in userspace

The possible range of virtual addresses is processor and kernel and ABI specific. But I strongly advise against coding some tricks related to some bits in addresses and pointers (since your code might work on some processors, but not on others).

These days, some ^* x86-64 processors apparently use only 48 bits of the virtual address space, but I don't recommend using that knowledge in your code (it might be wrong within a few years, or on some higher end models). See also x86-64 ABI.

^{If a pointer is outside that 48 bits range, you get a page fault, i.e. a SIGSEGV, the processor is not ignoring the unimplemented bits. So upper bits of pointers or addresses should be all zeros or all ones.}

On Linux, you might play with cat /proc/self/maps and cat /proc/$$/mapsto get more clues.

BTW, on Linux, you could reserve, using mmap(2) with MAP_ANONYMOUS | MAP_NORESERVE, some large address range (and either later call mmap with MAP_FIXED inside it, or never use it) to avoid it being later used by your process (or, as commented by damon, use MAP_32BITS which is x86-64 specific) This is probably more sensible than relying on the address bits to be restricted to 48 bits.

Also, catching SIGSEGV on Linux is tricky (you'll need some processor specific code) and costly. Perhaps you want some external pager mechanism (which exists on GNU Hurd, but not on Linux). Or mmap-ing some pseudo-file on some FUSE filesystem.

NB: most x86-64 processors have only 48 bits of addresses, but I don't recommend using that.

Note 2: Processor makers remembered what IBM/360 did: ignoring the upper address bits (originally 24 bits). When IBM had to extend address to 31 bits it was a nightmare for the software industry. So hardware makers understood the lesson, and disallow today (in hardware) playing naughty tricks on unused address bits.

What happens when the number of possible virtual addresses are exceeded

what happens if create more than 4294967296 variables

I guess that you are very confused. You don't create that much variables. You practically use C dynamic memory allocation (using malloc & free and friends) or C++ dynamic memory allocation (e.g. dynamic memory management, above low level memory management like ::operator new, which, in many implementations is using malloc).

Notice that malloc (in C) and new (in C++) don't create fresh variables. They are allocating fresh memory zones whose address could go into one pointer variable, if you code int *ptr = malloc(100000*sizeof(int); in C, or int* ptr = new int[100000]; in C++ ....

A variable (in C or C++) is a source code thing which has a name (e.g. ptr or x000002 or array in this answer here) and a scope. During execution, only locations matter (and variables do not exist). Read about memory addresses (what practically locations are).

So to have many variables, you'll need to have for example a huge source file with:

int x000001;
int x000002;

and so on. You probably can generate (with some other program) such a huge C or C++ source file, e.g. up to:

////etc
int x999998;
int x999999;

But even if you generate a four billion lines of source C file, you won't have patience to compile it. And if you did, the compilation will surely fail (at least at link time, which I view as part of the overall compilation of your program).

Notice that an array declaration defines only one variable:

/// one single variable, but a huge one
int array[4294967296];

declares one variable, named array. Again, that won't compile & link (and if the variable is a local one inside some function, you'll get at the very least a stack overflow at runtime). Typical call stacks are limited to one or a few megabytes (this is operating system & computer dependent).

Look at the picture in virtual address space wikipage and understand what pointer aliasing means and what virtual memory is.

In practice, on a 32 bits computer, the virtual address space is often limited to e.g. 3Gigabytes for a given process (each process is running some executable and has its own virtual address space). Details are operating system specific. On Linux, use setrlimit(2) - probably thru the ulimit builtin of your bash shell - to lower that limit.

On Linux, the dynamic memory allocation (malloc or new) is based upon system calls modifying the virtual address space, notably mmap(2). Such calls can fail (and then malloc would fail by returning NULL, and new would raise an exception), and on a 32 bits system they will fail before 3Gbytes. You probably want to disable memory overcommitment.

If you happen to use a Linux system, read about proc(5) and try

cat /proc/self/maps
cat /proc/$$/maps

then understand what is their output. You probably should also read Advanced Linux Programming.

I recommend taking several days to read: Operating Systems : Three Easy Pieces (it is freely downloadable).

^{(on Windows or MacOSX or Android, both malloc & new are also using some operating system primitive to increase the virtual address space. I leave you to find which ones)}

Can we change virtual memory address of a block of data without accessing the values in it?

You can't change &a, but you can change the contents of a by playing around with virtual memory. But not just int a; without copying actual memory contents around, you can only change whole page-sized chunks.

You could potentially swap these two arrays with virtual memory trickery, as an alternate way of doing std::swap_range(a, a+1024, b) which may or may not be faster.

alignas(4096) int32_t a[1024];     // assuming 4k page size and
alignas(4096) int32_t b[1024];     // CHAR_BITS=8 so sizeof(int32_t) = 4

Maybe only faster for much larger arrays, since copying is O(N), while manipulating page tables has large fixed cost (system call, TLB shootdown across cores) but only a small cost per page touched, like 8 / 4096 of the amount of data actually manipulated. (8 bytes of page-table-entry per 4096 bytes of data, on x86-64 for example.) Or less with large/hugepages.

---

The page size is (much) larger than 4 bytes on every real-world system, so both those objects are in the same virtual page in your example. 4-byte page size would be completely impractical, taking about as much space for page-tables as for actual data, and needing a TLB larger than the caches. (A ~40-bit physical address for every 48-2 = 46-bit virtual page number, for every 4 bytes of address-space you want to cover. With Accessed, Dirty, and R/W/X permissions.)

Common page sizes range from 4kiB (x86-64) to 16k or 64k, with 4k being uncomfortably small (too many TLB entries needed to cover the large working-sets modern software often uses). Some systems support largepages / hugepages using a page-directory entry (higher up in the radix-tree) as one contiguous large page, e.g. x86-64's 2M / 1G large/hugepages.

---

It is in theory possible to ask an OS to re-map your virtual address space differently onto the same data in physical memory, e.g. to swap the contents of two whole virtual pages by just updating their page-table-entries (PTEs) to swapping the physical addresses. (And invaliding the TLB entries on the current and every other core: TLB shootdown.)

---

Linux does not AFAIK have an API to ask for a mapping-swap of two virtual pages, but it does have mremap(2). (mremap is Linux-specific. Other OSes may have something similar. ISO C++ doesn't require virtual memory, so doesn't have any functions to portably manipulate it).

With three mremap(MREMAP_FIXED) calls and a temporary virtual page (that you weren't using or that you know is unallocated), you can do a tmp=a / a=b / b=tmp swap, where a and b are the contents of whole (ranges) of pages.

#define _GNU_SOURCE
#include <sys/mman.h>

// swap contents of pa[0..size] with pa[0..size]
// effectively mmap(tmp, MAP_FIXED) then munmap(tmp, size)
// size must be a multiple of system page size, and pointers must be page-aligned
void swap_page_contents(void *pa, void *pb, void *tmp, size_t size)
{
    // need to force moving, otherwise kernel will leave it in place because we aren't growing.
    void *ret = mremap(pa, size, size, MREMAP_MAYMOVE|MREMAP_FIXED, tmp);
    assert(ret == tmp);  // t2 != MAP_FAILED
    ret = mremap(pb, size, size, MREMAP_MAYMOVE|MREMAP_FIXED, pa);
    assert(ret != MAP_FAILED);
    ret = mremap(tmp, size, size, MREMAP_MAYMOVE|MREMAP_FIXED, pb);
    assert(ret != MAP_FAILED);
}

You might allocate tmp with mmap(MAP_PRIVATE|MAP_ANONYMOUS). Lazy allocation means a physical page would never get allocated to back that mapping, and Linux will put it somewhere unused in your virtual address space. This swap ends up unmapping it, so maybe I should have put that inside this function. But if you can be sure your process hasn't mapped any new memory since the last swap, you can reuse the same tmp. It doesn't need to be mapped, you just need to know it's not in use for anything else.

This can fail with EINVAL if you pass bad args (not page-aligned or overlaps). So perhaps have it return an error instead of assert, although if b isn't aligned then it will fail after already moving a to tmp.

This is also not atomic or thread-safe: pa is temporarily unmapped, and temporarily we have pa and pb both pointing to the original contents of pb. MREMAP_DONTUNMAP doesn't really help with that; it only works on MAP_PRIVATE|MAP_ANONYMOUS mappings (e.g. like malloc would allocate, but of course you'll probably break malloc's bookkeeping if you round down to the start of a page and swap its metadata.) Also, DONTUNMAP makes the old mapping read as zeros, although the man page says you can install a handler with userfaultfd(2) to do something else (e.g. to assist garbage collection).

Apparently you can pass old_size=0 to get it to make another virtual mapping for the data, but only if the original mapping was a MAP_SHARED mapping. So you can't do this to make the kernel pick an unused page-range for tmp for arbitrary mappings, only shared (probably file-backed) mappings.

---

Linux also has remap_file_pages(2) which can duplicate a page mapping within a tmpfs file-backed mmap, although that syscall is deprecated and apparently always uses a "slower in-kernel emulation" instead of whatever it used to do. Regardless, I think it still can't swap, only create a 2nd mapping for one part of a file, within a larger mapping.

---

Related Topics