Combine a Shell Script and a Zip File into a Single Executable for Deployment

Combine a shell script and a zip file into a single executable for deployment

If it's ok to assume that the user who will run the script has the unzip utility, then you can create a script like this:

#!/usr/bin/env bash

# commands that you need to do ...
# ...

unzip <(tail -n +$((LINENO + 2)) "$0")
exit

Make sure the script has a newline \n character at the end of the line of exit. And, it's important that the last line of the script is the exit command, and that the unzip command with tail is right in front of it.

Then, you can append to this file the zipped content, for example with:

cat file.zip >> installer.sh

Users will be able to run installer.sh, which will unzip the zipped content at the end of the file.

How to deploy a shell script to private linux vm

I think quoting the whole script and its deployment script for use with --scripts is a lot of work and error prone too. Luckily, there are easier alternatives for both quoting steps. The documentation of az vm run-command invoke --scripts states

Use @{file} to load script from a file.

Therefore, you can do

deploymentScript() {
  echo   '#! /usr/bin/env bash'
  printf 'tail -n+4 "${BASH_SOURCE[0]}" > %q\n' "$2"
  echo   'exit'
  cat "$1"
}
deploymentScript local.sh remote.sh > tmpDeploy.sh
az vm run-command invoke ... --scripts '@{tmpDeploy.sh}'
rm tmpDeploy.sh

Replace local.sh with the path of the local script you want to deploy and replace remote.sh with the remote path where the script should be deployed.

If you are lucky, then you might not even need tmpDeploy.sh. Try

az vm ... --scripts "@{<(deploymentScript local.sh remote.sh)}"

Some notes on the implementation:

• The deployed script is an exact copy of your script file. Even embedded binary data is kept. The trick with tail $BASH_SOURCE is inspired by this answer.

• The script can be arbitrarily long. Even huge scripts won't run into the error Argument list too long imposed by getconf ARG_MAX.

How can you bundle all your python code into a single zip file?

You can automate most of the work with regular python tools. Let's start with clean virtualenv.

[zart@feena ~]$ mkdir ziplib-demo
[zart@feena ~]$ cd ziplib-demo
[zart@feena ziplib-demo]$ virtualenv .
New python executable in ./bin/python
Installing setuptools.............done.
Installing pip...............done.

Now let's install set of packages that will go into zipped library. The trick is to force installing them into specific directory.

(Note: don't use --egg option either on command-line or in pip.conf/pip.ini because it will break file layout making it non-importable in zip)

[zart@feena ziplib-demo]$ bin/pip install --install-option --install-lib=$PWD/unpacked waitress
Downloading/unpacking waitress
  Downloading waitress-0.8.5.tar.gz (112kB): 112kB downloaded
  Running setup.py egg_info for package waitress

Requirement already satisfied (use --upgrade to upgrade): setuptools in ./lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg (from waitress)
Installing collected packages: waitress
  Running setup.py install for waitress

    Installing waitress-serve script to /home/zart/ziplib-demo/bin
Successfully installed waitress
Cleaning up...

Update: pip now has -t <path> switch, that does the same thing as --install-option --install-lib=.

Now let's pack all of them into one zip

[zart@feena ziplib-demo]$ cd unpacked
[zart@feena unpacked]$ ls
waitress  waitress-0.8.5-py2.7.egg-info
[zart@feena unpacked]$ zip -r9 ../library.zip *
  adding: waitress/ (stored 0%)
  adding: waitress/receiver.py (deflated 71%)
  adding: waitress/server.pyc (deflated 64%)
  adding: waitress/utilities.py (deflated 62%)
  adding: waitress/trigger.pyc (deflated 63%)
  adding: waitress/trigger.py (deflated 61%)
  adding: waitress/receiver.pyc (deflated 60%)
  adding: waitress/adjustments.pyc (deflated 51%)
  adding: waitress/compat.pyc (deflated 56%)
  adding: waitress/adjustments.py (deflated 60%)
  adding: waitress/server.py (deflated 68%)
  adding: waitress/channel.py (deflated 72%)
  adding: waitress/task.pyc (deflated 57%)
  adding: waitress/tests/ (stored 0%)
  adding: waitress/tests/test_regression.py (deflated 63%)
  adding: waitress/tests/test_functional.py (deflated 88%)
  adding: waitress/tests/test_parser.pyc (deflated 76%)
  adding: waitress/tests/test_trigger.pyc (deflated 73%)
  adding: waitress/tests/test_init.py (deflated 72%)
  adding: waitress/tests/test_utilities.pyc (deflated 78%)
  adding: waitress/tests/test_buffers.pyc (deflated 79%)
  adding: waitress/tests/test_trigger.py (deflated 82%)
  adding: waitress/tests/test_buffers.py (deflated 86%)
  adding: waitress/tests/test_runner.py (deflated 75%)
  adding: waitress/tests/test_init.pyc (deflated 69%)
  adding: waitress/tests/__init__.pyc (deflated 21%)
  adding: waitress/tests/support.pyc (deflated 48%)
  adding: waitress/tests/test_utilities.py (deflated 73%)
  adding: waitress/tests/test_channel.py (deflated 87%)
  adding: waitress/tests/test_task.py (deflated 87%)
  adding: waitress/tests/test_functional.pyc (deflated 82%)
  adding: waitress/tests/__init__.py (deflated 5%)
  adding: waitress/tests/test_compat.pyc (deflated 53%)
  adding: waitress/tests/test_receiver.pyc (deflated 79%)
  adding: waitress/tests/test_adjustments.py (deflated 78%)
  adding: waitress/tests/test_adjustments.pyc (deflated 74%)
  adding: waitress/tests/test_server.pyc (deflated 73%)
  adding: waitress/tests/fixtureapps/ (stored 0%)
  adding: waitress/tests/fixtureapps/filewrapper.pyc (deflated 59%)
  adding: waitress/tests/fixtureapps/getline.py (deflated 37%)
  adding: waitress/tests/fixtureapps/nocl.py (deflated 47%)
  adding: waitress/tests/fixtureapps/sleepy.pyc (deflated 44%)
  adding: waitress/tests/fixtureapps/echo.py (deflated 40%)
  adding: waitress/tests/fixtureapps/error.py (deflated 52%)
  adding: waitress/tests/fixtureapps/nocl.pyc (deflated 48%)
  adding: waitress/tests/fixtureapps/getline.pyc (deflated 32%)
  adding: waitress/tests/fixtureapps/writecb.pyc (deflated 42%)
  adding: waitress/tests/fixtureapps/toolarge.py (deflated 37%)
  adding: waitress/tests/fixtureapps/__init__.pyc (deflated 20%)
  adding: waitress/tests/fixtureapps/writecb.py (deflated 50%)
  adding: waitress/tests/fixtureapps/badcl.pyc (deflated 44%)
  adding: waitress/tests/fixtureapps/runner.pyc (deflated 58%)
  adding: waitress/tests/fixtureapps/__init__.py (stored 0%)
  adding: waitress/tests/fixtureapps/filewrapper.py (deflated 74%)
  adding: waitress/tests/fixtureapps/runner.py (deflated 41%)
  adding: waitress/tests/fixtureapps/echo.pyc (deflated 42%)
  adding: waitress/tests/fixtureapps/groundhog1.jpg (deflated 24%)
  adding: waitress/tests/fixtureapps/error.pyc (deflated 48%)
  adding: waitress/tests/fixtureapps/sleepy.py (deflated 42%)
  adding: waitress/tests/fixtureapps/toolarge.pyc (deflated 43%)
  adding: waitress/tests/fixtureapps/badcl.py (deflated 45%)
  adding: waitress/tests/support.py (deflated 52%)
  adding: waitress/tests/test_task.pyc (deflated 78%)
  adding: waitress/tests/test_channel.pyc (deflated 78%)
  adding: waitress/tests/test_regression.pyc (deflated 68%)
  adding: waitress/tests/test_parser.py (deflated 80%)
  adding: waitress/tests/test_server.py (deflated 78%)
  adding: waitress/tests/test_receiver.py (deflated 87%)
  adding: waitress/tests/test_compat.py (deflated 51%)
  adding: waitress/tests/test_runner.pyc (deflated 72%)
  adding: waitress/__init__.pyc (deflated 50%)
  adding: waitress/channel.pyc (deflated 58%)
  adding: waitress/runner.pyc (deflated 54%)
  adding: waitress/buffers.py (deflated 74%)
  adding: waitress/__init__.py (deflated 61%)
  adding: waitress/runner.py (deflated 58%)
  adding: waitress/parser.py (deflated 69%)
  adding: waitress/compat.py (deflated 69%)
  adding: waitress/buffers.pyc (deflated 69%)
  adding: waitress/utilities.pyc (deflated 60%)
  adding: waitress/parser.pyc (deflated 53%)
  adding: waitress/task.py (deflated 72%)
  adding: waitress-0.8.5-py2.7.egg-info/ (stored 0%)
  adding: waitress-0.8.5-py2.7.egg-info/dependency_links.txt (stored 0%)
  adding: waitress-0.8.5-py2.7.egg-info/installed-files.txt (deflated 83%)
  adding: waitress-0.8.5-py2.7.egg-info/top_level.txt (stored 0%)
  adding: waitress-0.8.5-py2.7.egg-info/PKG-INFO (deflated 65%)
  adding: waitress-0.8.5-py2.7.egg-info/not-zip-safe (stored 0%)
  adding: waitress-0.8.5-py2.7.egg-info/SOURCES.txt (deflated 71%)
  adding: waitress-0.8.5-py2.7.egg-info/entry_points.txt (deflated 33%)
  adding: waitress-0.8.5-py2.7.egg-info/requires.txt (deflated 5%)
[zart@feena unpacked]$ cd ..

Note that those files should be at top of zip, you can't just zip -r9 library.zip unpacked

Checking the result:

[zart@feena ziplib-demo]$ PYTHONPATH=library.zip python
Python 2.7.1 (r271:86832, Apr 12 2011, 16:15:16)
[GCC 4.6.0 20110331 (Red Hat 4.6.0-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import waitress
>>> waitress
<module 'waitress' from '/home/zart/ziplib-demo/library.zip/waitress/__init__.pyc'>
>>>
>>> from wsgiref.simple_server import demo_app
>>> waitress.serve(demo_app)
serving on http://0.0.0.0:8080
^C>>>

Update: since python 3.5 there is also zipapp module which can help with bundling the whole package into .pyz file. For more complex needs pyinstaller, py2exe or py2app might better fit the bill.

Pack setup files to single executable setup

IExpress.exe is ideal for your job. Google for samples. It is included in your Windows installation. Just open a Command Prompt and type iexpress.exe - this starts a wizard that helps you getting started.

Trying to create a deployment file for AWS lambda

NVM i figured it out. The code I copied was using cp as the copy command for the subprocess. I looked up the command for my OS (windows 10) and it was COPY not cp. here is the new call to subprocess cmd = "COPY {0} {1}".format(deployment_file, os.path.abspath(deployment_dir)).split()

---

Related Topics