Remote: OpenSSL version mismatch

[niieani]

I'm using the binaries from the GitHub release page on the repository server:
sha256: 5d8362f545aedf71be1893bd0fab30ecd79d40c89671323c804376b53f8886af
https://github.com/borgbackup/borg/releases/download/0.29.0/borg-linux64
Note: borg-linux64 binary built with openssl 1.0.1e (debian wheezy)

Initially, on the client, the same binary was used.

But when running Borg I get:

Remote: OpenSSL version mismatch. Built against 1000204f, you have 1000105f
Connection closed by remote host. Is borg working on the server?

Does that mean I need to recompile borg? If so, where, on the repository-side or locally?

On the remote repo server, there is openssl 1.0.2d-0ubuntu1.2 installed.

Note: "Remote:" does not always mean that the message to the right of it really came from the remote system. It is just stuff emitted on stderr on our side of the ssh tunnel. It can come from remote stderr, go through the tunnel and appear on our stderr. But it could be also emitted by our ssh client on stderr.

[niieani]

Later, I've compiled the same version of Borg locally (0.29.0) (using pip) and the error went away, the repository-side works fine with the GitHub release binary.

[ThomasWaldmann]

The binary made with pyinstaller is expected to contain all libraries it needs, so it should not emit a openssl version mismatch.

Strange is that you say that a change on the client fixed the error msg from the server, I can't make sense from that.

[ThomasWaldmann]

Please also give the openssl version installed as linux dist package on the client.

[ThomasWaldmann]

ping. see last comment. it's important as the version on the server somehow does not make sense with the error msg.

[ThomasWaldmann]

Ah, I think I can now make sense from the error message:

when doing remote backups, borg forks ssh and establishes a connection to the remote server's ssh and executes "borg serve" there (similar like rsync works via ssh).

The error msg comes from the client's ssh binary (/usr/bin/ssh) and is emitted on stderr.
We can't differentiate that from stderr stuff coming from remote via the tunnel, thus the "Remote:" prefix, which is wrong here.

"built against" is the version /usr/bin/ssh was built against.
"you have" is the openssl library bundled into the pyinstaller binary and unpacked into /tmp/_ME*/ (intended to be used by borg, but seems that ssh accidentally loads it also).

So the question now is: how can we avoid that ssh binary grabs the wrong openssl library?

[ThomasWaldmann]

hmm, do we need to unset LD_LIBRARY_PATH when forking ssh?

$ showenv  # this is a pyinstaller binary that just prints os.environ['LD_LIBRARY_PATH']
/tmp/_MEIaFMKPJ

Without pyinstaller, this is usually not set at all.