Zlib.Error: Error -3 While Decompressing: Incorrect Header Check

Node Zlib incorrect header check

The file is gzipped, so you need to use zlib.Gunzip instead of zlib.Inflate.

Also, streams are very efficient in terms of memory usage, so if you want to perform the retrieval without storing the .gz file locally first, you can use something like this:

request('https://wiki.mozilla.org/images/f/ff/Example.json.gz')
  .pipe(zlib.createGunzip())
  .pipe(fs.createWriteStream('example.json'));

Otherwise, you can modify your existing code:

var gunzip = zlib.createGunzip();
readstream.pipe(gunzip).pipe(writestream);

ZlibDecompressor throws incorrect header check exception

The problem reported is that there is no valid zlib header in the first two bytes. The problem with the data is that it does not appear to have any deflate-compressed data anywhere in it, regardless of whether such data could be zlib wrapped, gzip wrapped, or raw.

Reconstructing zlib stream

To assure that you can start decompression at some point with no history, you must either use Z_FULL_FLUSH or simply end the stream and start a new one.

For the former you could do a Z_SYNC_FLUSH followed by a Z_FULL_FLUSH in order to insert two markers resulting in the nine bytes 00 00 ff ff 00 00 00 ff ff, which would be unlikely to be seen randomly in the compressed data. You can do the same for the latter, simply inserting a large-enough marker between the end of the previous zlib stream and the start of the next zlib stream.

Handling Haskell zlib decompression errors

The function from zlib you need to use here is called decompressWithErrors. Its value is the recursive DecompressStream data structure that you can fold to a ByteString using v:fromDecompressStream

Here's a full example of how to write the function you asked for:

import Data.Either
import Codec.Compression.Zlib.Internal
import qualified Data.ByteString.Lazy.Char8 as LB

-- | Convert & unfold the custom DecompressStream
--   error format from zlib to a Either
decompressStreamToEither :: DecompressStream -> Either String LB.ByteString
decompressStreamToEither (StreamError _ errmsg) = Left errmsg
decompressStreamToEither stream@(StreamChunk _ _) = Right $ fromDecompressStream stream
decompressStreamToEither StreamEnd = Right $ ""

-- | Decompress with explicit error handling
safeDecompress :: LB.ByteString -> Either String LB.ByteString
safeDecompress bstr = decompressStreamToEither $ decompressWithErrors gzipOrZlibFormat defaultDecompressParams bstr

-- | Decompress gzip, if it fails, return uncompressed String
decompressIfPossible :: LB.ByteString -> LB.ByteString
decompressIfPossible bstr =
    let conv (Left a) = bstr
        conv (Right a) = a
    in (conv . safeDecompress) bstr

Note that this example uses gzipOrZlibFormat which automatically detects if the header is a zlib or a gzip header.

Zlib is unable to extract a compress String in java, compression is done in python

That is a gzip stream, not a zlib stream. Use GZIPInputStream.

---

Related Topics