set --enable-loadable-sqlite-extensions in configure?

[wsxiaoys]

It'll be nice if heroku could enable sqlite extension by default

[edmorley]

Hi! To pre-empt what I imagine will be Kenneth's question... what's the use case for using sqlite extensions (or sqlite at all) on Heroku dynos? (Since the filesystem is ephemeral)

[wsxiaoys]

Sqlite:
We have a few readonly sqlite database stored in S3, they'll be fetched to dyno to be served as API.

Sqlite-extension:
Many mathematical functions, e.g log, expt are not available in sqlite, we'd like to load extension-functions.c

[kennethreitz]

sqlite is very useful to have, as it’s a very fast database that can be used despite our ephemeral filesystem — it just can’t be used persistiently.

In addition to that, some tools/libraries, e.g. iPython, import sqlite, expecting it to be available as part of the stdlib, and will not function if it is not compiled at build time.

I don’t forsee changing our level of support here, though.

[obfusk]

I just ran into the missing support for loadable extensions myself (I have a read-only sqlite DB and an extension for PCRE support).

I understand that this is not a common use case, but the change is trivial and I can't really think of any downsides (there's no extra dependency or anything). Would you be willing to reconsider?

My workaround for now is using a post_compile hook to re-build python with --enable-loadable-sqlite-extensions:

# rebuild python if it's missing loadable sqlite extensions
if  ! python3 -msysconfig | grep enable-loadable-sqlite-extensions \
    > /dev/null; then
  PYTHON_VERSION="$(< runtime.txt)"
  export WORKSPACE_DIR="$(cd "$BIN_DIR"/.. || exit; pwd)/builds"
  rm -f .heroku/python/bin/python   # prevent failing ln after build
  sed -i 's!figure --pre!figure --enable-loadable-sqlite-extensions --pre!' \
    "$WORKSPACE_DIR"/runtimes/python3
  "$WORKSPACE_DIR/runtimes/$PYTHON_VERSION" /app/.heroku/python/
fi

[edmorley]

@obfusk Hi! I've reopened this issue to remind me to take a look when I get a chance to triage the issues in this repo (have been catching up with urgent fixes/tech debt first; hope to get to triage soon).

[edmorley]

The official Python docker images use --enable-loadable-sqlite-extensions so I'd be inclined to copy them:
docker-library/python#125

[obfusk]

The official Python docker images use --enable-loadable-sqlite-extensions

So do Debian & Ubuntu and AFAIK most other Linux distros. Also Termux.

[edmorley]

Tracking internally at W-8060029.

[buncis]

@obfusk hello I have same problem with you but what I want to add is --enable-shared

can you tell me how and where to add that post_compile hook scripts of yours?

thanks

edit:

Ok seems I just need to add post_compile file to the bin folder

[obfusk]

@buncis https://github.com/obfusk/jiten/blob/master/bin/post_compile