How to Limit the Heap Size

How to limit the heap size?

Check out resource.setrlimit(). It only works on Unix systems but it seems like it might be what you're looking for, as you can choose a maximum heap size for your process and your process's children with the resource.RLIMIT_DATA parameter.

EDIT: Adding an example:

import resource

rsrc = resource.RLIMIT_DATA
soft, hard = resource.getrlimit(rsrc)
print 'Soft limit starts as  :', soft

resource.setrlimit(rsrc, (1024, hard)) #limit to one kilobyte

soft, hard = resource.getrlimit(rsrc)
print 'Soft limit changed to :', soft

I'm not sure what your use case is exactly but it's possible you need to place a limit on the size of the stack instead with resouce.RLIMIT_STACK. Going past this limit will send a SIGSEGV signal to your process, and to handle it you will need to employ an alternate signal stack as described in the setrlimit Linux manpage. I'm not sure if sigaltstack is implemented in python, though, so that could prove difficult if you want to recover from going over this boundary.

How to limit heap consumption without setting -Xmx flag

The garbage collector did not get around to immediately freeing memory in your first run. Processing XLSX files will end up generating a lot of transient objects, so the first run lets them build up to around 250MB before cleaning up; while the second, memory-constrained run, is forced to clean up the objects sooner.

The garbage collector has many, many options and strategies that can be configured. Off the top of my head, the only way I know to limit consumption for only that specific code is to run that code only in its own JVM process, with appropriate GC parameters.

How to limit heap size for a c code in linux

You could use (inside your program) setrlimit(2), probably with RLIMIT_AS (as cited by Ouah's answer).

Better yet, make your shell do it. With bash it is the ulimit builtin.

Be sure that your program is indeed correctly and completely handling malloc failure everywhere (testing every return of malloc against NULL indicating its failure).

If you don't test malloc's result, when it is failing, it gives NULL and it is very likely that the next instructions would dereference a null pointer (or some address very close to it), which is undefined behavior and on Linux giving a segmentation violation.

You probably should consider using valgrind during the debugging phase.

BTW, 70Kilobytes of memory is tiny today (at least on Linux laptops, desktops and even tablets). Notice that the C standard library may call malloc under the hoods (for example, fopen gives a FILE handle, which has some buffer, which might be internally obtained thru malloc)

And memory overcommit can be disabled on Linux with the following command

echo 0 > /proc/sys/vm/overcommit_memory

to be run as root.

Increasing the JVM maximum heap size for memory intensive applications

Get yourself a 64-bit JVM from Oracle.

How I can limit the heap's size so when I allocate a lot it won't get the machine stuck?

Yes, use the Debug Heap hooks in the CRT.

You can hook malloc to breakpoint when you allocate a large block, using _CrtSetAllocHook and _CrtDbgBreak. Or if your problem is lots of small blocks, you can set a breakpoint on the 10,000th allocation (for example) using _CrtSetBreakAlloc.

• CRT Debug Heap: http://msdn.microsoft.com/en-us/library/974tc9t1%28v=VS.100%29.aspx
• _CrtSetAllocHook: http://msdn.microsoft.com/en-us/library/820k4tb8(v=vs.100).aspx
• _CrtDbgBreak: http://msdn.microsoft.com/en-us/library/k4wx2tde(v=vs.100).aspx
• _CrtSetBreakAlloc: http://msdn.microsoft.com/en-us/library/4wth1ha5(v=vs.100).aspx

Visual C++: possible to limit heap size?

You can set memory allocation and deallocation hooks, using _CrtSetAllocHook.

How to set the maximum memory usage for JVM?

use the arguments -Xms<memory> -Xmx<memory>. Use M or G after the numbers for indicating Megs and Gigs of bytes respectively. -Xms indicates the minimum and -Xmx the maximum.

How do you limit non-heap size on Open JDK 11

You cannot constraint off-heap memory, otherwise the application would crash. When direct memory is allocated, it is always needed, so denying it is the same as OutOfMemoryError.

You can limit metaspace size with -XX:MaxMetaspaceSize=.

You can reduce the amount of memory used by the JIT compiler to store compiled methods, therefore potentially reducing compilation, with -XX:ReservedCodeCacheSize=.

---

Related Topics