What Is the Best Project Structure for a Python Application

What is the best project structure for a Python application?

Doesn't too much matter. Whatever makes you happy will work. There aren't a lot of silly rules because Python projects can be simple.

• /scripts or /bin for that kind of command-line interface stuff
• /tests for your tests
• /lib for your C-language libraries
• /doc for most documentation
• /apidoc for the Epydoc-generated API docs.

And the top-level directory can contain README's, Config's and whatnot.

The hard choice is whether or not to use a /src tree. Python doesn't have a distinction between /src, /lib, and /bin like Java or C has.

Since a top-level /src directory is seen by some as meaningless, your top-level directory can be the top-level architecture of your application.

• /foo
• /bar
• /baz

I recommend putting all of this under the "name-of-my-product" directory. So, if you're writing an application named quux, the directory that contains all this stuff is named /quux.

Another project's PYTHONPATH, then, can include /path/to/quux/foo to reuse the QUUX.foo module.

In my case, since I use Komodo Edit, my IDE cuft is a single .KPF file. I actually put that in the top-level /quux directory, and omit adding it to SVN.

What is the correct folder structure to use for a Python project using pytest?

Python uses the 'environment variable' PYTHONPATH to look for sources to import code from. By default, the directory you execute a python program is automatically included, but you want to include something like this when you test:

PYTHONPATH=$PYTHONPATH,../src python test_main.py

This is if you're executing a test from the source directory. Tools like IntelliJ (PyCharm) will let you add this as a value in your test invocation. Alternatively you can use export PYTHONPATH=.... (Note this is for a *nix environment, your mileage on windows may vary.)

The upshot is that every directory in PYTHONPATH will be loaded and Python will attempt to use it as a 'root' for modules you try to import. Your basic directory structure is the most idiomatic.

• See this answer for more on configuring PYTHONPATH correctly.
• See this doc for more about how the PYTHONPATH is modified and used 'under the hood'.
• See this answer for options to include the src directory when running pytest tests.
• See this blog post about using autoenv (a Python library) to enable the usage of .env files to manage this for you (at least within a virtualenv setup - a good idea generally).
• setup.py is also idiomatic for including many modules, and may provide a more convenient path for the situation you're handling.

Python git project-structure convention?

This is likely dependent on what you are building. The article I linked in my comment breaks out different layouts like so:

CLI Layouts:

• One-Off Script
• Installable Single Package
• Application with Internal Packages

Web App Layouts:

• Django
• Flask

While link only answers are discouraged here, I think its counter productive to rehash the whole article but the "Installable Single Package" layout has worked for me and looks like this (borrowed from the same article):

helloworld/
│
├── helloworld/
│   ├── __init__.py
│   ├── helloworld.py
│   └── helpers.py
│
├── tests/
│   ├── helloworld_tests.py
│   └── helpers_tests.py
│
├── .gitignore
├── LICENSE
├── README.md
├── requirements.txt
└── setup.py

NOTE: If you have a data directory, I would put it at the same level as tests

What are the best practices for structuring a FastAPI project?

Harsha already mentioned my project generator but I think it can be helpful for future readers to explain the ideas behind of it.

If you are going to serve your frontend something like yarn or npm. You should not worry about the structure between them. With something like axios or the Javascript's fetch you can easily talk with your backend from anywhere.

When it comes to structuring the backend, if you want to render templates with Jinja, you can have something that is close to MVC Pattern.

your_project
├── __init__.py
├── main.py
├── core
│   ├── models
│   │   ├── database.py
│   │   └── __init__.py
│   ├── schemas
│   │   ├── __init__.py
│   │   └── schema.py
│   └── settings.py
├── tests
│   ├── __init__.py
│   └── v1
│       ├── __init__.py
│       └── test_v1.py
└── v1
    ├── api.py
    ├── endpoints
    │   ├── endpoint.py
    │   └── __init__.py
    └── __init__.py 

By using __init__ everywhere, we can access the variables from the all over the app, just like Django.

Let's the folders into parts:

• Core
  
  • models
    
    • database.py
  • schemas
    
    • users.py
    • something.py
  • settings.py
  • views (Add this if you are going to render templates)
    
    • v1_views.py
    • v2_views.py
• tests
• v1
• v2

Models

It is for your database models, by doing this you can import the same database session or object from v1 and v2.

Schemas

Schemas are your Pydantic models, we call it schemas because it is actually used for creating OpenAPI schemas since FastAPI is based on OpenAPI specification we use schemas everywhere, from Swagger generation to endpoint's expected request body.

settings.py

It is for Pydantic's Settings Management which is extremely useful, you can use the same variables without redeclaring it, to see how it could be useful for you check out our documentation for Settings and Environment Variables

Views

This is optional if you are going to render your frontend with Jinja, you can have something close to MVC pattern

• Core
  
  • views
    
    • v1_views.py
    • v2_views.py

It would look something like this if you want to add views.

Tests

It is good to have your tests inside your backend folder.

APIs

Create them independently by APIRouter, instead of gathering all your APIs inside one file.

Notes

You can use absolute import for all your importing since we are using __init__ everywhere, see Python's packaging docs.

So assume you are trying to import v1's endpoint.py from v2, you can simply do

from my_project.v1.endpoints.endpoint import something

Project structure for python projects with maven

It is the good news: you do not need any tool. You can organise your source code in any way you want.

Let recap why we need tools in the java world:

In java you want to generate directories upfront because the namespace system dictates that each class must live in one file in a directory structure that reflects that package hierarchy. As a consequence you have a deep folder structure. Maven enforces an additional set of convention for file location. You want to have tools to automate this.

Secondly, different artefacts require use of different goals and even additional maven projects (e.g. a ear project requires a few jars and war artefacts). There are so many files to create you want to have tools to automate this.

The complexity makes tools like mvn archetype:generate not just helpful. It is almost indispensable.

In python land, we just do not have these complexity in the language.

If my project is small, I can put all my classes and functions in a single file (if it makes sense)

If my project is of a bigger size (LOC or team size), it makes sense to group .py files into modules in whatever way makes sense to you and your peers.

At the end of the days, it is about striking a balance between ease of maintenance and readability.

How to structure a python project with three applications that use common module

I have a similar project that is set up like this

project_root/
    App1/
        __init__.py
    FlaskControlPanel/
        app.py
        static/
        templates/
    models/
        __init__.py
        mymodels.py

Then, I run everything from project_root. I have a small script (either batch or shell depending on my environment) that sets PYTHONPATH=. so that imports work correctly. This is done because I usually develop using PyCharm, where the imports "just work", but when I deploy the final product the path doesn't match what it did in my IDE.

Once the PYTHONPATH is set to include everything from your project root, you can do standard imports.

For example, from my FlaskControlPanel app.py, I have this line:

from models.mymodels import Model1, Model2, Model3

From the App1 __init__.py I have the exact same import statement:

from models.mymodels import Model1, Model2, Model3

I can start the Flask application by running this from my command line (in Windows) while I am in the project_root directory:

setlocal
SET PYTHONPATH=.
python FlaskControlPanel\app.py

The setlocal is used to ensure the PYTHONPATH is only modified for this session.

Best practices for structuring Python project with non python files e.g. DLL, .c, .cpp?

Perfect place, see Python Packaging Guide. Don't forget to put the files into setup.py or MANIFEST.in.

How to organize a Python Project?

A Package is basically a folder with __init__.py file under it and usually some Modules, where Module is a *.py file.
It has to do with import mainly. If you add __init__.py to Indicators you can use:

from Indicators.Stochastics import *

or

from Indicators import Stochastics

By the way, I would recommend to keep module/package names lowercase. It does not affect functionality but it's more "pythonic".

---

Related Topics