How to Install Pysqlite for Python3.4.2

Install pysqlite in virtualenv with python3 support

There is no public version of pysqlite for Python 3.x. For Python 3.x, the sqlite3 module in the standard library is the most up-to date version of pysqlite there is.

Django/Sqlite3 Module Load Path in Python Virtual Environment

Okay, the answer provided by Laenka-Oss solved my problem with minor adjustments: django can't find new sqlite version? (SQLite 3.8.3 or later is required (found 3.7.17))

Install Sqlite from source:

cd ~
wget https://www.sqlite.org/2020/sqlite-autoconf-3320300.tar.gz
tar zxvf sqlite-autoconf-3290000.tar.gz
cd sqlite-autoconf-3290000

./configure --prefix=$HOME/opt/sqlite --disable-dynamic-extensions --enable-static --disable-shared
make && make install

Update library paths by adding these lines to your .bash_profile:

export PATH=$HOME/opt/sqlite/bin:$PATH
export LD_LIBRARY_PATH=$HOME/opt/sqlite/lib
export LD_RUN_PATH=$HOME/opt/sqlite/lib

Install Python from source:

cd ~
wget https://www.python.org/ftp/python/3.8.3/Python-3.8.3.tar.xz
tar xJf Python-3.8.3.tar.xz
cd Python-3.8.3
./configure --prefix=$HOME/opt
make && make install

Now update Python path by adding this to the end of your .bash_profile:

export PATH=$HOME/opt/Python-3.8.3/bin:$PATH

Check that everything worked:

source .bash_profile
python3 --version
Python 3.8.3

python3
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.32.3'

If you encounter a "SqLite header and version mismatch: ..." error, make sure you run source .bash_profile or restart your connection. If that doesn't work, then double check the sqlite install used the commands shown above.

Your .bash_profile should look like:

export PATH=$HOME/opt/sqlite/bin:$PATH
export LD_LIBRARY_PATH=$HOME/opt/sqlite/lib
export LD_RUN_PATH=$HOME/opt/sqlite/lib
export PATH=$HOME/opt/Python-3.8.3/bin:$PATH

Install Spatialite for python (GeoDjango) on OS X

I was able to get this working now, using the tip here:

https://github.com/ghaering/pysqlite/issues/60#issuecomment-50345210

I'm not sure if it was using the real paths that fixed it, or just the Homebrew kegs or underlying packages have been updated and now install cleanly. Still, it works now.

I reproduce below the steps I took:

brew update
brew install sqlite  # 3.8.5
brew install libspatialite  # 4.2.0
brew install spatialite-tools  # 4.1.1

git clone https://github.com/ghaering/pysqlite.git
cd pysqlite

(where brew reported I had existing versions I unlinked them and installed the latest as commented above)

then edited setup.cfg to comment out #define=SQLITE_OMIT_LOAD_EXTENSION and specify the paths:

include_dirs=/usr/local/opt/sqlite/include
library_dirs=/usr/local/opt/sqlite/lib

activated the virtualenv where I want it installed, then

python setup.py build
python setup.py install

(build_static still fails with clang: error: no such file or directory: 'sqlite3.c')

(maybe I should have done pip install . as suggested in the github issue)

now the spatialite geodjango.db "SELECT InitSpatialMetaData();" succeeds, albeit with an ignorable error:

InitSpatiaMetaData() error:"table spatial_ref_sys already exists"

i.e. it's probably not even necessary to run that command

Cannot download node-sqlite3@4.2.0 - node-pre-gyp ERROR Tried to download(403) Access Denied - node.js

The other answer is not helpful in my case. Also, I found that simply using yarn instead of npm solves the issues mentioned by the other answer.

However, that answer does not address the main issue:

node-pre-gyp WARN Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.2.0/node-*******.tar.gz

node-pre-gyp WARN Pre-built binaries not found

TL;DR

For me, upgrading to latest sqlite3 (e.g. via yarn add "sqlite3@^5" (NOTE: this command does not work properly in powershell)) fixes this issue. The reason for that is: it finds the pre-built binaries for this version just fine.

If you (absolutely) NEED to use an older version, you will have to make sure, the local compilation succeeds, as explained below.

In addition to that, you also want to make sure, you have a somewhat recent version of node installed, because often times, pre-built binaries are commonly not provided for out-of-date Node versions (check node -v; recommendation: use volta to manage your node version).

Warning: Of course, those pre-built binaries might also be removed by the package author in the near fututure. Hopefully, they will at least replace them with a newer version, and not just take them offline entirely.

Explanation

The error message above indicates that gyp was not able to download the system-specific pre-built binaries (which is very different from the npm package). Because it cannot find the pre-built binaries, it would try to compile them locally, which is always messy and can easily go wrong.

Q: Why can it not find the pre-built binaries?
- A: Pre-built binaries are not so easy to manage because you need a lot more different versions of them than just the version of the npm package they are for. Usually, you need one for each major version of the npm package + version of node + operating system + instruction set architecture (e.g. ARM 32bit vs. x86 64bit etc.). It appears that, for some reason, Mapbox decided not to store those pre-built binaries for the long haul. I am sure they were there in the past, but for version 4.2.0 (and possibly many other versions), they are gone now.
- They might only keep all those different binaries for the latest major version of their npm package and wipe old versions to save storage space. I have not found any documentation on that on their repo.
Q: What happens if it cannot download the pre-built binaries?
- A: Things get very messy. Instead of a short download, it tries to compile them on your system, involving a complex labyrinth of system-level dependencies. For example:
  - In case of the OP, that compilation failed due to:
    gyp: No Xcode or CLT version detected!
  - In my case, it failed because (on Windows), I did not have VS 2017 (nor its redistributables) installed.
- In general, local compilation firstly requires node-gyp to play along, and then you also need python and a system-specific C++ compiler. If you Google a little, you will find many stories of people succeeding (and failing) to get such local compilation to work. It is possible to aim for local compilation, but getting the pre-built binaries makes things a lot easier.