Stdin into Zip Command, How to Specify a File Name

How do you specify filenames within a zip when creating it on the command line from a pipe?

From what i can gather you cannot do both with the zip command, i mean you cannot both specify the filenames and pipe the content. You can either pipe the contents and the resulting file is - or you can pipe the filenames with -@.

That does not mean that doing so is impossible using other techniques. I outline one of those below. It does mean that you have to have PHP installed and the zip extension loaded.

There could be a whole bunch of other ways to do it. But this is the easiest that I know of. Oh and it's a one-liner too.

This is a working example using PHP

echo contents | php -r '$z = new ZipArchive();$z->open($argv[1],ZipArchive::CREATE);$z->addFromString($argv[2],file_get_contents("php://stdin"));$z->close();' test.zip testfile

To run on windows just swap single and double quotes. Or just place the script in a file.

"test.zip" is the resulting Zip file, "testfile" is the name of the contents that are being piped into the command.

Result from unzip -l test.zip

Archive:  test.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        6  01-07-2010 12:56   testfile
---------                     -------
        6                     1 file

And here is a working example using python

echo contents | python -c "import sys
import zipfile
z = zipfile.ZipFile(sys.argv[1],'w')
z.writestr(sys.argv[2],sys.stdin.read())
z.close()
" test5.zip testfile2

Result of unzip -l

Archive:  test5.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        9  01-07-10 13:43   testfile2
 --------                   -------
        9                   1 file

Result of unzip -p

contents

How to redirect input in cmd

use the -@ option in zip.

type zip -@ 1.zip < file.txt

from the zip help found by typing zip -h

Copyright (C) 1990-1996 Mark Adler, Richard B. Wales, Jean-loup Gailly
Onno van der Linden and Kai Uwe Rommel. Type 'zip -L' for the software License.
Zip 2.1 (April 27th 1996). Usage:
zip [-options] [-b path] [-t mmddyy] [-n suffixes] [zipfile list] [-xi list]
  The default action is to add or replace zipfile entries from list, which
  can include the special name - to compress standard input.
  If zipfile and list are omitted, zip compresses stdin to stdout.
  -f   freshen: only changed files  -u   update: only changed or new files
  -d   delete entries in zipfile    -m   move into zipfile (delete files)
  -k   force MSDOS (8+3) file names -g   allow growing existing zipfile
  -r   recurse into directories     -j   junk (don't record) directory names
  -0   store only                   -l   convert LF to CR LF (-ll CR LF to LF)
  -1   compress faster              -9   compress better
  -q   quiet operation              -v   verbose operation/print version info
  -c   add one-line comments        -z   add zipfile comment
  -b   use "path" for temp file     -t   only do files after "mmddyy"
  -@   read names from stdin        -o   make zipfile as old as latest entry
  -x   exclude the following names  -i   include only the following names
  -F   fix zipfile (-FF try harder) -D   do not add directory entries
  -A   adjust self-extracting exe   -J   junk zip file prefix (unzipsfx)
  -T   test zipfile integrity       -X   eXclude eXtra file attributes
  -$   include volume label         -S   include system and hidden files
  -h   show this help               -n   don't compress these suffixes

bash zip file with version read from file and used in zip file name

Replace build.zip with "build_$(cat manifest.json | jq -r .version).zip".

Update:

Replace build.zip with "build_$(jq -r .version manifest.json).zip".

why can't python execute a zip archive passed via stdin?

The reason that you can 'python file.zip', but not 'cat file.zip | python' is that Python has the 'zipimport' built in so that when you run python against files (or try to import them), zipimport takes a crack at them as part of the import process. (See the import module for details).

But with stdin, python does not make any attempt to search the streaming data - because the streaming data could be anything - could be user input that is handled by code, could be code. There's no way to know and Python makes no real effort to know for that reason.

edit

Occasionally, when you're answering questions - you think 'I really shouldn't tell someone the answer', not because you wish to be secretive or hold some amount of power over them. Simply because the path they're going down isn't the right path and you want to help them out of the hole they're digging. This is one of those situations. However, against my better judgement, here's an extremely hacky way of accomplishing something similar to what you want. It's not the best way, it's probably in fact the worst way to do it.

I just played around with the zipimporter for a while and tried all the tricks I could think of. I looked at 'imp', 'compile' as well.. Nothing can import a zipped module (or egg) from memory so far that I can see. So, an interim step is needed.

I'll say this up front, I'm embarrassed to even be posting this. Don't show this to people you work with or people that you respect because they laugh at this terrible solution.

Here's what I did:

mkdir foo
echo "print 'this is foo!'" >>foo/__init__.py
zip foo.zip -r foo
rm -rf foo                   # to ensure it doesn't get loaded from the filesystem
mv foo.zip somethingelse.zip # To ensure it doesn't get zipimported from the filesystem

And then, I ran this program using

cat somethingelse.zip | python script.py

#!/usr/bin/python 

import sys
import os
import zipfile
import StringIO
import zipimport
import time

sys.path.append('/tmp')

class SinEater(object):
    def __init__(self):
        tmp = str(int(time.time()*100)) + '.zip'
        f = open(tmp, 'w')
        f.write(sys.stdin.read(1024*64)) # 64kb limit
        f.close()
        try:
            z = zipimport.zipimporter(tmp)
            z.load_module('foo')

        except:
            pass

if __name__ == '__main__':
    print 'herp derp'
    s = SinEater()

Produces:

herp derp
this is new

A solution that would be about a million times better than this would be to have a filesystem notification (inotify, kevent, whatever windows uses) that watches a directory for new zip files. When a new zip file is dropped in that directory, you could automatically zipimport it.
But, I cannot stress enough even that solution is terrible. I don't know much about Ansible (anything really), but I cannot imagine any engineer thinking that it would be a good solution for how to handle code updates or remote control.

how can i search for files and zip them in one zip file

The command you use will run zip on each file separately, try this:

find . -name <name> -print | zip newZipFile.zip -@

The -@ tells zip to read files from the input. From man zip(1),

-@ file lists. If a file list is specified as -@ [Not on MacOS], zip takes the list of input files from standard input instead of from the command line.

---

Related Topics