How to Read a (Static) File from Inside a Python Package

Accessing static files included in a Python module

dir() won't tell you anything about static files. The correct way (or one of them, at least) to get access to this data is with the resource_* functions in pkg_resources (part of setuptools), e.g.:

import pkg_resources

pkg_resource.resource_listdir('glm_plotter', 'templates')
# Returns a list of files in glm_plotter/templates

pkg_resource.resource_string('glm_plotter', 'templates/index.html')
# Returns the contents of glm_plotter/templates/index.html as a byte string

Python read static file from within a pip package that I've deployed

The solution was

1. Import the library .... import mylib

with pkg_resources.open_text(mylib, 'to_read.json') as file:
    return json.load(file)

So basically everything that I thought shouldn't have applied did apply. RIP

How include static files to setuptools - python package

As pointed out in the comments, there are 2 ways to add the static files:

1 - include_package_data=True + MANIFEST.in

A MANIFEST.in file in the same directory of setup.py that looks like this:

include src/static/*
include src/Potato/*.txt

With include_package_data = True in setup.py.

2 - package_data in setup.py

package_data = {
    'static': ['*'],
    'Potato': ['*.txt']
}

Specify the files inside the setup.py.

---

Do not use both include_package_data and package_data in setup.py.

include_package_data will nullify the package_data information.

Official docs:

https://setuptools.readthedocs.io/en/latest/userguide/datafiles.html

loading data from folder inside package

.pkl data are probably serialized data using pickle python module. It can't be imported. You have to deserialize data.

import pickle
data = pickle.load(open("data.pkl", "rb"))

As say in other answer, you can wraps this in a python module.

# filename: data.py
import pickle

def load_data(filename):
    return pickle.load(open(filename, "rb"))

If your .pkl files are in a python package, you can retreive its using pkg_resources.

import pickle
import pkg_resources

def load_data(resource_name):
    return pickle.load(
        pkg_resources.resource_stream("my_package", resource_name))

In python >= 3.7, data can be retreived using importlib.resources to prevent use of thrird-party package.

data = pickle.load(
    importlib.resources.open_binary("my_package.data_folder", "data.pkl"))

Access data in package subdirectory

You can use __file__ to get the path to the package, like this:

import os
this_dir, this_filename = os.path.split(__file__)
DATA_PATH = os.path.join(this_dir, "data", "data.txt")
print open(DATA_PATH).read()

---

Related Topics