Pickled File Won't Load on MAC/Linux

Pickled file won't load on Mac/Linux

Probably you didn't open the file in binary mode when writing and/or reading the pickled data. In this case newline format conversion will occur, which can break the binary data.

To open a file in binary mode you have to provide "b" as part of the mode string:

char_file = open('pickle.char', 'rb')

Python Pickle File from Linux Incompatible with Mac?

Try explicitly setting the protocol version, to make sure both machines are using the same version:

pickle.dump(network, g, protocol=4)

pickle.load Not Working

You should open the pickled file in binary mode, especially if you are using pickle on different platforms. See this and this questions for an explanation.

pickle dependency failure

I've reproduced your problem by saving the pickle with CRLF (Windows) line endings on my Mac OS X machine.

The pickle machinery is quite particular about newlines. If the pickle is saved or copied using a non-binary transfer mode (e.g. resaved with a text editor on Windows, copied using ASCII FTP transfer, saved as a text file from a website, etc.), the pickle will be corrupted with the addition of CR characters.

Now, the problem is this line in pickle.py:

module = self.readline()[:-1]

If the file you give the Unpickler is opened 'rb' but contains CRLF, then these lines will read module = "photo_data\r", which isn't a valid module name. Upon importing, the error will appear as

ImportError: No module named photo_data

with an unprinted carriage return after the photo_data (dreadfully sneaky!).

The solution is to ensure you transfer files in a binary fashion, and don't run unix2dos or any similar utilities on the pickle. Alternately, if using protocol 0 (text) pickles, it is safe to use 'rU' (universal newline mode) to open the pickled files instead.

See also Pickled file won't load on Mac/Linux.

Is pickle file of python cross-platform?

Python's pickle is perfectly cross-platform.

This is likely due to EOL (End-Of-Line) differences between Windows and Linux. Make sure to open your pickle files in binary mode both when writing them and when reading them, using open()'s "wb" and "rb" modes respectively.

Note: Passing pickles between different versions of Python can cause trouble, so try to have the same version on both platforms.

Python 3.7 Error: Unsupported Pickle Protocol 5

Use pickle5 or load it into python 3.8+ and then serialize it to a lower version of it using the protocol parameter.

UnicodeDecodeError in pickle.load

The solution was found in this answer. The pickle file was probably encoded with Python 2, and providing pickle.load with the optional argument encoding='latin1' solved the problem.

The code that works looks like this:

import pickle

# Load model weights and metadata
weightFile = open('vgg16.pkl', 'rb')
d = pickle.load(weightFile, encoding='latin1')

---

Related Topics