Why Does Pages Allocation with Order of 10 or 11 Using _Get_Free_Pages() Usually Fail

Why does pages allocation with order of 10 or 11 using __get_free_pages() usually fail?

If I remember correctly, __get_free_pages uses buddy allocation, which not only does scatter its allocations throughout physical memory, it does so in the worst possible pattern for subsequent attempts to allocate large contiguous blocks. If my calculations are correct, on your system with 24GB of physical RAM, even if no space whatsoever were occupied by anything but buddy allocations, it would take less than 8192 order-0 (4KB) allocations to render allocation of a 4MB chunk with __get_free_pages impossible.

There is a thing called the contiguous memory allocator, which is supposed to address the genuine need for large physically-contiguous allocations by device drivers; as of June 2011 it was not in the official kernel, but that was more than a year ago now. You should look into that.

Initialization of number of free pages in each order in a memory zone

There is a call mem_init(), which marks the free areas in the mem_map and tells us how much memory is free. This is done after various parts of the system have claimed their memory after the kernel image.

which further calls free_unused_memmap() and free_all_bootmem() api's to free the memory (in pages) and will adds the free memory chunks(in pages) into different orders free_lists.

You can print the zones(Normal, Highmem etc) related information using show_free_areas().

The result of the show_free_areas() before/after freeing all bootup and other mem_map related memory.

Before:

Normal free:0kB ... present:778240kB managed:772160kB ... etc

HighMem free:0kB ... present:270336kB managed:270336kB ... etc

Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB

HighMem: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB

After:

Normal free:542868kB ... present:778240kB managed:542868kB ... etc

HighMem free:270336kB ... present:270336kB managed:270336kB ... etc

Normal: 5*4kB (M) 2*8kB (M) 3*16kB (M) 4*32kB (M) 3*64kB (M) 4*128kB (M) 3*256kB (M) 7*512kB (M) 7*1024kB (M) 9*2048kB (M) 125*4096kB (M) = 542868kB

HighMem: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 66*4096kB (M) = 270336kB

What is the purpose of allocating pages in the pagefile with CreateFileMapping?

From the CreateFileMappingFunction:

A single file mapping object can be shared by multiple processes.

Can the Virtual memory be shared across multiple processes?

Redhat 7.1 kernel process stack size from 8K to 16KB

The kernel often needs to allocate a set of one or more pages which are physically contiguous. This may be necessary while allocating buffers (required by drivers for data transfers such as DMA) or when creating a process stack.

Typically, to meet such requirements, kernel attempts to avoid fragmentation by allocating pages which are physically contiguous and additionally freed pages are merged/grouped into larger physically contiguous group of pages (if available). This is handled by the memory management sub-system and the buddy allocator. Now when your stack (8k or 16k in RHEL7) is created when the program has starts executing.

If kernel is unable to obtain or allocate a the requested set of physically contiguous pages (either 2 for 8k stack or 4 for 16k stack assuming 4k page size), then this could potentially lead to page allocation failures, order:2. (i.e 2^2=4 pages * 4K). The order depends on the size of your physically contiguous pages requested. We can observe the /proc/buddyinfo file during the time when page allocation failures occur, it could show signs of physically memory being fragmented.

---

Related Topics