How can I make jenkins run pip install ?
The fact that you have to run use sudo
to run pip
is a big warning that your virtual environment isn't working. The build output shows that pip
is installing the requirements in your system site-packages directory, which is not the way virtualenv works.
Your build script doesn't actually preserve the activated virtual environment. The environment variables set by the activate script are set in a child bash process and are not propagated up to the build script. You should source the activate
script instead of running a separate shell:
virtualenv venv --distribute
. venv/bin/activate
pip install -r requirements.txt
python tests.py
If you're running this as one build step, that should work (and install your packages in venv). If you want to add more steps, you'll need to set the PATH environment variable in your other steps. You're probably better off providing full paths to pip
and python
to ensure you're not dependent on system package installations.
pip not found in Jenkins build
You are probably running Jenkins in a container. Jenkins container does not have python
installed because it's not needed in order to run Jenkins, and containers by design only include the minimal necessary things.
Moreover, you are running your job on Jenkins master, so it runs in the same place as Jenkins (Jenkins container), so no python
and no pip
and no easy_install
.
You should run your job somewhere where python
and pip
are installed. You may want to consider running the job in the python
container, e.g. like this:
pipeline {
agent {
docker { image 'python:3' }
}
stages {
stage('Test') {
steps {
sh 'pip --version'
}
}
}
}
This will run a new container (official python3 release container) and execute the steps inside that container.
Unable to use pip inside jenkins
The pip command cannot be found within the user's path. the solution is either call it directly from /usr/local/bin/pip or add /usr/local/bin to the user's path
for bash: PATH=${PATH}:/usr/local/bin
for (t)csh: setenv PATH "${PATH}:/usr/local/bin"
Unable to pip install in Docker image as agent through Jenkins declarative pipeline
I have found what I myself would think is the prettier solution:
stage("Python Test") {
agent {
docker {
label "docker && linux"
image "python:3.7"
}
}
steps {
withEnv(["HOME=${env.WORKSPACE}"]) {
sh "pip install -r requirements.txt --user"
# python stuff
}
}
post {
cleanup {
cleanWs()
}
}
}
This workaround steers completely around the issue itself, installing the packages at user level. The issue here was that the HOME-directory was not initially writeable either, thus overwriting the HOME directory.
How to pip install in a docker image with a jenkins pipline step?
As I mentioned in this comment, the solution should be adding a proper user inside the container. Jenkins uses 984:984
for uid/gid on my machine (but may be different on yours - login to the host Jenkins is running on and execute sudo -u jenkins id -a
to detect them), so you need to replicate it in the container that should be run by Jenkins:
FROM python:3.7
RUN mkdir /home/jenkins
RUN groupadd -g 984 jenkins
RUN useradd -r -u 984 -g jenkins -d /home/jenkins jenkins
RUN chown jenkins:jenkins /home/jenkins
USER jenkins
WORKDIR /home/jenkins
CMD ["/bin/bash"]
Of course, since you aren't the root
user in the container anymore, either create a virtual environment:
$ docker run --rm -it jenkins/python /bin/bash
jenkins@d0dc87c39810:~$ python -m venv myenv
jenkins@d0dc87c39810:~$ source myenv/bin/activate
jenkins@d0dc87c39810:~$ pip install numpy
or use the --user
argument:
$ docker run --rm -it jenkins/python /bin/bash
jenkins@d0dc87c39810:~$ pip install --user --upgrade pip
jenkins@d0dc87c39810:~$ pip install --user numpy
etc.
Alternatively, you can (but in most cases shouldn't) enter the container as root
, but with jenkins
group:
$ docker run --user 0:984 ...
This way, although the modified files will still change the owner, their group ownership will still be intact, so Jenkins will be able to clean up the files (or you can do it yourself, via
sh 'rm -f modified_file'
in the Jenkinsfile
.
How to install packages using pip in jenkins
By default, sudo
prompts for a password. Basically, you have two options of solving that:
- running Jenkins as root, which is obviously not what you'd like to do, or
allowing the user that Jenkins runs under to execute
pip
specifically without a password. Add the following line to the end of/etc/sudoers
file (make sure to usesudo visudo
for editing that):jenkins_user ALL=NOPASSWD:/usr/bin/pip
Related Topics
Sum of Square Differences (Ssd) in Numpy/Scipy
Pip Install Not Working With Jenkins
Stuck With Loops in Python - Only Returning First Value
How to Tell If Tensorflow Is Using Gpu Acceleration from Inside Python Shell
Auto Reloading Python Flask App Upon Code Changes
Windowserror: [Error 193] %1 Is Not a Valid Win32 Application in Python
How to Get a Random Value from Dictionary
Pythonic Way to Find Maximum Value and Its Index in a List
Django: Get Current User in Model Save
Get Business Days Between Start and End Date Using Pandas
Split String At Nth Occurrence of a Given Character
How to Extract Table Names and Column Names from SQL Query
Remove the Selected Elements from the Image in Opencv
How to Clean \Xc2\Xa0 \Xc2\Xa0..... in Text Data
Spark Add New Column With Value Form Previous Some Columns
How to Manage Division of Huge Numbers in Python