How to select R installation when using rpy2 on conda?
Recompile rpy2 locally, with a linker flag to explicate which R library to link to.
Inspired by the blog post https://thomas-cokelaer.info/blog/2012/01/installing-rpy2-with-different-r-version-already-installed/ I decided to recompile rpy2 from source, and supply the R path as an environment variable to the compilation.
(rnvp) ludhu126@hyperion:~/real-nvp/real-nvp$ export LDFLAGS="-Wl,-rpath,/home/ludhu126/miniconda3/envs/rnvp/lib/R/lib"
(rnvp) ludhu126@hyperion:~/real-nvp/real-nvp$ pip install rpy2 --force-reinstall --compile --no-binary rpy2
Collecting rpy2
Using cached rpy2-3.4.5.tar.gz (194 kB)
Collecting cffi>=1.10.0
Using cached cffi-1.14.6-cp39-cp39-manylinux1_x86_64.whl (405 kB)
Collecting jinja2
Using cached Jinja2-3.0.1-py3-none-any.whl (133 kB)
Collecting pytz
Using cached pytz-2021.1-py2.py3-none-any.whl (510 kB)
Collecting tzlocal
Using cached tzlocal-3.0-py3-none-any.whl (16 kB)
Collecting pycparser
Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting MarkupSafe>=2.0
Using cached MarkupSafe-2.0.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (30 kB)
Skipping wheel build for rpy2, due to binaries being disabled for it.
Installing collected packages: pycparser, MarkupSafe, tzlocal, pytz, jinja2, cffi, rpy2
Attempting uninstall: pycparser
Found existing installation: pycparser 2.20
Uninstalling pycparser-2.20:
Successfully uninstalled pycparser-2.20
Attempting uninstall: MarkupSafe
Found existing installation: MarkupSafe 2.0.1
Uninstalling MarkupSafe-2.0.1:
Successfully uninstalled MarkupSafe-2.0.1
Attempting uninstall: tzlocal
Found existing installation: tzlocal 3.0
Uninstalling tzlocal-3.0:
Successfully uninstalled tzlocal-3.0
Attempting uninstall: pytz
Found existing installation: pytz 2021.1
Uninstalling pytz-2021.1:
Successfully uninstalled pytz-2021.1
Attempting uninstall: jinja2
Found existing installation: Jinja2 3.0.1
Uninstalling Jinja2-3.0.1:
Successfully uninstalled Jinja2-3.0.1
Attempting uninstall: cffi
Found existing installation: cffi 1.14.6
Uninstalling cffi-1.14.6:
Successfully uninstalled cffi-1.14.6
Attempting uninstall: rpy2
Found existing installation: rpy2 3.4.5
Uninstalling rpy2-3.4.5:
Successfully uninstalled rpy2-3.4.5
Running setup.py install for rpy2 ... done
Successfully installed MarkupSafe-2.0.1 cffi-1.14.6 jinja2-3.0.1 pycparser-2.20 pytz-2021.1 rpy2-3.4.5 tzlocal-3.0
(rnvp) ludhu126@hyperion:~/real-nvp/real-nvp$ python
Python 3.9.6 (default, Aug 18 2021, 19:38:01)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpy2.robjects as ro; print(ro.r('version'))
_
platform x86_64-conda-linux-gnu
arch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 4
minor 0.5
year 2021
month 03
day 31
svn rev 80133
language R
version.string R version 4.0.5 (2021-03-31)
nickname Shake and Throw
>>>
How to set a custom R installation for using rpy2 in Jupyter?
There are two approaches to solve this, a local (for individual Jupyter notebooks) and a global one (for the kernel itself). Both are related to setting the R_HOME environment variable.
Local (source):
Before calling %load_ext rpy2.ipython in your Jupyter notebook, run:
import os
os.environ['R_HOME'] = '/home/your/anaconda3/envs/myenv/lib/R' #path to your R installation
Global:
Find your kernel directory via: jupyter kernelspec list and edit the file kernel.json. Update the JSON by adding:"env": {"R_HOME":"/home/your/anaconda3/envs/my-env-name/lib/R"}, then restart your kernel (you might have to restart Jupyter as well).
Update (messed up LD_LIBRARY_PATH)
Recently, I tried running rpy2 in jupyter again after setting up a new environment using conda:
conda config --add channels conda-forge
conda config --set channel_priority strict
conda create -n myenv python=3.7
conda activate myenv
conda install r-essentials pandas rpy2
And this time I ran into the following issue when trying to either %load_ext rpy2.ipython (Jupyter) or simply import rpy2.robjects (any script):
>>> import rpy2.robjects
Warning message:
package ‘methods’ was built under R version 3.6.3
Error: package or namespace load failed for ‘stats’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/your/anaconda3/envs/myenv/lib/R/library/stats/libs/stats.so':
/home/your/anaconda3/envs/myenv/lib/R/library/stats/libs/stats.so: undefined symbol: MARK_NOT_MUTABLE
During startup - Warning messages:
1: package ‘datasets’ was built under R version 3.6.3
2: package ‘utils’ was built under R version 3.6.3
3: package ‘grDevices’ was built under R version 3.6.3
4: package ‘graphics’ was built under R version 3.6.3
5: package ‘stats’ was built under R version 3.6.3
6: package ‘stats’ in options("defaultPackages") was not found
R[write to console]: Error: package or namespace load failed for ‘tools’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/your/anaconda3/envs/myenv/lib/R/library/tools/libs/tools.so':
/home/your/anaconda3/envs/myenv/lib/R/library/tools/libs/tools.so: undefined symbol: R_NewPreciousMSet
R[write to console]: Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/home/your/anaconda3/envs/myenv/lib/R/library/tools/libs/tools.so':
/home/your/anaconda3/envs/myenv/lib/R/library/tools/libs/tools.so: undefined symbol: R_NewPreciousMSet
R[write to console]: In addition:
R[write to console]: Warning message:
R[write to console]: package ‘tools’ was built under R version 3.6.3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/your/anaconda3/envs/myenv/lib/python3.7/site-packages/rpy2/robjects/__init__.py", line 20, in <module>
import rpy2.robjects.functions
File "/home/your/anaconda3/envs/myenv/lib/python3.7/site-packages/rpy2/robjects/functions.py", line 12, in <module>
from rpy2.robjects import help
File "/home/your/anaconda3/envs/myenv/lib/python3.7/site-packages/rpy2/robjects/help.py", line 43, in <module>
tools_ns = _get_namespace(StrSexpVector(('tools',)))
File "/home/your/anaconda3/envs/myenv/lib/python3.7/site-packages/rpy2/rinterface_lib/conversion.py", line 44, in _
cdata = function(*args, **kwargs)
File "/home/your/anaconda3/envs/myenv/lib/python3.7/site-packages/rpy2/rinterface.py", line 621, in __call__
raise embedded.RRuntimeError(_rinterface._geterrmessage())
rpy2.rinterface_lib.embedded.RRuntimeError: Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/home/your/anaconda3/envs/myenv/lib/R/library/tools/libs/tools.so':
/home/your/anaconda3/envs/myenv/lib/R/library/tools/libs/tools.so: undefined symbol: R_NewPreciousMSet
The issue seemed to have been a screwed up R "situation" (check via %run -m rpy2.situation in Jupyter or simply python -m rpy2.situation on the command line), which had R's additions to LD_LIBRARY_PATH: pointing to and old, globally installed R version.
I had to manually unset the LD_LIBRARY_PATH to solve this issue. This path can be set / unset analogously to R_HOME.
PS: I found R_HOME and LD_LIBRARY_PATH set in my .bashrc to custom (from source) R installation. Which confused the Jupyter kernel obviously. Not smart ;)
PPS: rpy2.situation still tells me that there is a Warning: The environment variable R_HOME differs from the default R in the PATH.:
Looking for R's HOME:
Environment variable R_HOME: /home/your/anaconda3/envs/myenv/lib/R
Calling `R RHOME`: /home/your/anaconda3/envs/jupyter-env/lib/R
Environment variable R_LIBS_USER: None
Warning: The environment variable R_HOME differs from the default R in the PATH.
Which worries me that R actually defaults to the R installed for the Jupyter installation. So if anybody has comments about this, I would be grateful.
Related Topics
Script to Get User That Has Process with Most Memory Usage
How to Sort Comma Separated Values in Bash
How Come _Exit(0) (Exiting by Syscall) Prevents Me from Receiving Any Stdout Content
How to Start a Process That Won't End When My Ssh Session Ends
Random Alphanumeric String Linux Swift 3
Executable Object Files and Virtual Memory
Nginx Location Deny by File Extension Syntax
Docker-Compose Stop Working After Docker Desktop Installation on Debian 11
Why Can't I Access This Folder
How to Enable Keep-Alive in Haproxy
Simple Way to Get Filesize in X86 Assembly Language
One-Liner: Print All Lines Except the Last 3
Bash - Calculate the Average of Numbers Inputted
Usb Modem Is Echoing Back Wrong Characters
How to Make .Gitignore Configurable Based on Environment Variables
How to Increase Openfabrics Memory Limit for Torque Jobs
Gnuplot Doesn't Work Through Ssh Command
How to Install Flash Builder 4 or a Flash Builder Plugin on Eclipse in Ubuntu 10.04