Does malloc lazily create the backing pages for an allocation on Linux (and other platforms)?
Linux does deferred page allocation, aka. 'optimistic memory allocation'. The memory you get back from malloc is not backed by anything and when you touch it you may actually get an OOM condition (if there is no swap space for the page you request), in which case a process is unceremoniously terminated.
See for example http://www.linuxdevcenter.com/pub/a/linux/2006/11/30/linux-out-of-memory.html
Are some allocators lazy?
On Linux, malloc requests memory with sbrk() or mmap() - either way, your address space is expanded immediately, but Linux does not assign actual pages of physical memory until the first write to the page in question. You can see the address space expansion in the VIRT column, while the actual, physical memory usage in RES.
Why is malloc not using up the memory on my computer?
malloc()
does not use memory. It allocates it.
After you allocate the memory, use it by assigning some data.
size_t Size = 256 * 1024 * 1024;
p = malloc(Size);
if (p != NULL) {
memset(p, 123, Size);
}
Some platforms implement malloc()
is such a way that the physical consumption of memory does not occur until that byte (or more likely a byte within a group or "page" of bytes) is accessed.
calloc()
may or may not truly use the memory either. A system could map lots of memory to the same physical zeroed memory, at least until the data gets interesting. See
Why malloc+memset is slower than calloc?
Confusion after counting maximum allocation that can be done by malloc()
This is memory overcommit:
[...]Under the default memory management strategy, malloc() essentially always succeeds, with the kenrel assuming you're not really going to use all of the memory you just asked for. The malloc()'s will continue to succeed, but not until you actually try to use the memory you allocated will the kernel 'really' allocate it. [...]
If we look at a Linux man page for malloc it says (emphasis mine):
By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer.
and:
For more information, see the description of /proc/sys/vm/overcommit_memory and /proc/sys/vm/oom_adj in proc(5), and the Linux kernel source file Documentation/vm/overcommit-accounting.
What is aligned memory allocation?
Alignment requirements specify what address offsets can be assigned to what types. This is completely implementation-dependent, but is generally based on word size. For instance, some 32-bit architectures require all int
variables start on a multiple of four. On some architectures, alignment requirements are absolute. On others (e.g. x86) flouting them only comes with a performance penalty.
malloc
is required to return an address suitable for any alignment requirement. In other words, the returned address can be assigned to a pointer of any type. From C99 §7.20.3 (Memory management functions):
The pointer returned if the allocation
succeeds is suitably aligned so that
it may be assigned to a pointer to any
type of object and then used to access
such an object or an array of such
objects in the space allocated (until
the space is explicitly deallocated).
Can the stack adress be deduced from the heap adress (or vice-versa) with ASLR?
No, it does not, for security reasons. If they were related in any way it would be a severe security flaw.
On the other hand, libc is the one responsible of allocating this two areas using mmap(), not kernel... or at least not directly. Each area has its own call to mmap() (including libraries), and the kernel gives partially random addresses for each call.
You can see in /proc/$pid/maps
the different areas allocated for a specific program with $pid
as process id.
Related Topics
How to Instruct Cron to Execute a Job Every Second Week
How to Run a Program With a Different Working Directory from Current, from Linux Shell
How to Use the Lines of a File as Arguments of a Command
How to Open a New Tab in Gnome Terminal from Command Line
Bash Function to Find Newest File Matching Pattern
Bash Script Process Substitution Syntax Error: "(" Unexpected
The Difference Between Fork(), Vfork(), Exec() and Clone()
Shell Script: Run Function from Script Over Ssh
Syntax Error in Shell Script With Process Substitution
How to Use Grep to Show Just Filenames on Linux
How to Quickly Sum All Numbers in a File
How to Get Docker Linux Container Information from Within the Container Itself
Get Most Recent File in a Directory on Linux
Difference Between ${} and $() in Bash
Why Is a Tilde in a Path Not Expanded in a Shell Script
Format and Then Convert Txt to CSV Using Shell Script and Awk
Fastest Way to Find Lines of a File from Another Larger File in Bash