How to deal with Java Heap Space error while running rjava
You need to modify the java parameters with the following function at the start of your script.
options(java.parameters = "-Xmx8000m")
You may also need restart your environment first.
RStudio: Error in .jarray(m) : java.lang.OutOfMemoryError: Java heap space
The Xmx
flag controls the size of the Java runtime heap. You can increase this to a larger value which may allow your R code to run without hitting the ceiling:
> options(java.parameters = "-Xmx4g") # or 8g, or larger than this, ...
Note that this should increase the heap only for the Java process called by your R script. Outside R, whatever heap size was being used by your Java should remain the same. You could also change it externally if you wanted to.
I know about Xmx
but did not know how to do this from within the R console. For that, I found this useful blog post:
http://www.bramschoenmakers.nl/en/node/726
How to deal with java.lang.OutOfMemoryError: Java heap space error?
Ultimately you always have a finite max of heap to use no matter what platform you are running on. In Windows 32 bit this is around 2GB
(not specifically heap but total amount of memory per process). It just happens that Java chooses to make the default smaller (presumably so that the programmer can't create programs that have runaway memory allocation without running into this problem and having to examine exactly what they are doing).
So this given there are several approaches you could take to either determine what amount of memory you need or to reduce the amount of memory you are using. One common mistake with garbage collected languages such as Java or C# is to keep around references to objects that you no longer are using, or allocating many objects when you could reuse them instead. As long as objects have a reference to them they will continue to use heap space as the garbage collector will not delete them.
In this case you can use a Java memory profiler to determine what methods in your program are allocating large number of objects and then determine if there is a way to make sure they are no longer referenced, or to not allocate them in the first place. One option which I have used in the past is "JMP" http://www.khelekore.org/jmp/.
If you determine that you are allocating these objects for a reason and you need to keep around references (depending on what you are doing this might be the case), you will just need to increase the max heap size when you start the program. However, once you do the memory profiling and understand how your objects are getting allocated you should have a better idea about how much memory you need.
In general if you can't guarantee that your program will run in some finite amount of memory (perhaps depending on input size) you will always run into this problem. Only after exhausting all of this will you need to look into caching objects out to disk etc. At this point you should have a very good reason to say "I need Xgb of memory" for something and you can't work around it by improving your algorithms or memory allocation patterns. Generally this will only usually be the case for algorithms operating on large datasets (like a database or some scientific analysis program) and then techniques like caching and memory mapped IO become useful.
H2O anomaly per_feature = TRUE java.lang.OutOfMemoryError: Java heap space
[cutting and pasting my response to the h2ostream mailing list here too...]
i suspect the large number of categorical levels is causing the memory to blow up.
try removing that variable and seeing if it at least completes.
if it does, try re-binning into a smaller number of levels somehow.
Related Topics
What Does "Error: '.Class' Expected" Mean and How to Fix It
Java Cannot Be Applied to Given Types String and Integer
Why Does Java Implicitly (Without Cast) Convert a 'Long' to a 'Float'
Converting JSON to Xml in Java
How to Avoid Using Scriptlets in My Jsp Page
How to Generate Random Number in Specific Range in Android
How to Make a Copy of a File in Android
What Do I Use Now That Handler() Is Deprecated
Return a Value from Asynctask in Android
Transitive Dependencies Not Resolved for Aar Library Using Gradle
How to Connect via Https Using Jsoup
Sending a JSON Http Post Request from Android
How to Resolve Host "<Url Here>" No Address Associated with Host Name