Version 6 fails to find OCI libs where version 5 succeeded

[tlandschoff-scale]

We are preparing to replace cx_Oracle 5 with version 6 in our application. Updating the requirements lead to our tests to fail.

The reason is quite obvious but I think cx_Oracle 5 was more robust: On our build slave the instantclient packages are installed but nothing else of the Oracle database. This always worked fine:

(venv) (python27)[jenkins@build-rhel6-v2 ~]$ pip install cx_Oracle==5.3
Collecting cx_Oracle==5.3
[...]
Successfully installed cx-Oracle-5.3
(venv) (python27)[jenkins@build-rhel6-v2 ~]$ python -c "import cx_Oracle; print cx_Oracle.version"
5.3

After upgrading to 6.0b1 we get this:

(venv) (python27)[jenkins@build-rhel6-v2 ~]$ pip install cx_Oracle==6.0b1
Collecting cx_Oracle==6.0b1
Installing collected packages: cx-Oracle
  Found existing installation: cx-Oracle 5.3
    Uninstalling cx-Oracle-5.3:
      Successfully uninstalled cx-Oracle-5.3
Successfully installed cx-Oracle-6.0b1
(venv) (python27)[jenkins@build-rhel6-v2 ~]$ python -c "import cx_Oracle; print cx_Oracle.version"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
cx_Oracle.DatabaseError: DPI-1047: Oracle Client library cannot be loaded: libclntsh.so: cannot open shared object file: No such file or directory. See https://oracle.github.io/odpi/doc/installation.html for help

The version of instant client installed is a bit old:

(venv) (python27)[jenkins@build-rhel6-v2 ~]$ rpm -qa|grep instant
oracle-instantclient11.2-basic-11.2.0.3.0-1.x86_64
oracle-instantclient11.2-devel-11.2.0.3.0-1.x86_64
(venv) (python27)[jenkins@build-rhel6-v2 ~]$ rpm -ql oracle-instantclient11.2-basic-11.2.0.3.0-1.x86_64|grep libclnt
/usr/lib/oracle/11.2/client64/lib/libclntsh.so.11.1

The obvious way to fix this is to extend LD_LIBRARY_PATH:

$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/oracle/11.2/client64/lib python -c "import cx_Oracle; print cx_Oracle.version"
6.0b1

For easing the upgrade to users of cx_Oracle it would make sense though to keep the behaviour to guess the instant client installations like setup.py did for cx_Oracle 5.3.

Feel free to close this if you disagree.

[anthony-tuininga]

This was noted in our own testing and discussed without coming to a definite conclusion. The new driver is capable of using any Oracle Client without needing to be recompiled. Setting RPATH detracts from that ability -- but it may be a price worth paying. :-) I'll do some further research and get back to you.

[tlandschoff-scale]

Setting RPATH detracts from that ability -- but it may be a price worth paying. :-)

I actually think the current behaviour is more correct, but anyway expect many complaints. We actually fixed our build environment to ensure LD_LIBRARY_PATH is set.

Setting RPATH is probably really a bad idea, but how about doing the same discovery as before during the build if libclntsh is not found and storing the path into the binary. On import this could be used as a fallback together with reporting a deprecation warning that this behaviour will go away in, say, 7.0!?

Actually it would be even more compatible to stick to the path discovered during build whenever available and warn if another version would be selected by standard linker behaviour.

My two cents...

[jmcp]

Setting LD_LIBRARY_PATH rather than using RPATH / RUNPATH is deprecated on Solaris.
Please review https://blogs.oracle.com/rie/entry/tt_ld_library_path_tt and https://blogs.oracle.com/rie/entry/tt_ld_library_path_tt from the Solaris linker architects for why.

When I built and packaged cx_Oracle for Solaris, as well as bundling the Instant Client software, I ensured that we baked in the RPATH so as to avoid needing any sort of environment hack with LD_LIBRARY_PATH. In general, it's much more secure to bake in RPATH.

[anthony-tuininga]

Thanks for the link @jmcp. It appears that RUNPATH is preferred over RPATH since there is no way to override RPATH without using LD_PRELOAD -- something I believe we want to avoid, but your point is well-taken.

I attempted to set RPATH/RUNPATH in the cx_Oracle module and it failed miserably. The reason for this is that libclntsh.so itself depends on a few other libraries (like libmql1.so) and apparently the value of RPATH/RUNPATH in the cx_Oracle module is not consulted for looking up these secondary dependencies. And libclntsh.so itself does not have an RPATH/RUNPATH setting, unfortunately. I did a quick test with a utility called patchelf that can set RPATH/RUNPATH in an existing library without requiring relinking -- and that worked beautifully. I set the value to '$ORIGIN' since the dependent libraries are all found in the same directory. I'll see if this can be corrected in further releases of the Instant Client or whether it is worth mentioning the workaround for existing users.

Setting RPATH is probably really a bad idea, but how about doing the same discovery as before during the build if libclntsh is not found and storing the path into the binary. On import this could be used as a fallback together with reporting a deprecation warning that this behaviour will go away in, say, 7.0!?

Unfortunately, as noted above, even attempting to use the full path name of the known Instant Client library will fail due to the fact that it has no RPATH/RUNPATH setting so cannot find its own dependencies. And setting LD_LIBRARY_PATH within the library itself is useless since that environment variable is only consulted when a process starts.

Actually it would be even more compatible to stick to the path discovered during build whenever available and warn if another version would be selected by standard linker behaviour.

This would be nice but there is no way to determine the actual library that was loaded as far as I know. And this is actually desirable behaviour anyway. I am producing a single binary which doesn't require OCI libraries or header files available and should be usable in all Oracle Client configurations. So this would need to be an opt-in warning if this was even doable.

Assuming the Instant Client libraries can be adjusted, the best solution it would seem to me would be to set the RUNPATH setting to include the known locations for the instant client RPMs (on Linux) and any known location of an Oracle client located during the build process.

If anyone has further thoughts and ideas I would love to hear them!

[jmcp]

Re the Instant Client delivery, I did file a bug requesting that that team update their Makefiles to bake in the rpath, but it doesn't seem to have gone very far :(

[tlandschoff-scale]

I did file a bug requesting that that team update their Makefiles to bake in the rpath, but it doesn't seem to have gone very far :(

Not sure if that would be such a great idea. All in all I am against rpath tweaks, in my eyes setting LD_LIBRARY_PATH is the more flexible approach.

I seldom got bitten by a incorrectly set LD_LIBRARY_PATH and it was easy to debug. And I had a few fights with rpath already. See also https://wiki.debian.org/RpathIssue for more information, suffice to say is that Debian discourages using it.

[anthony-tuininga]

Closing due to lack of activity. Improving installation/configuration of the Oracle Client libraries is an ongoing concern, however.