Seg fault when sending message to SB without the with/as statement

[mrtnmch]

Hello everyone,
When I tried to send a message to ServiceBus, I experienced segmentation faults. It turns out I tried to send them like this:

topic_client.get_sender().send(Message(msg))

which causes segfault. I tried it on my Fedora PC, as well as Windows PC of a colleague of mine - same results. It seems that calling the sender like this fixes the issue:

with topic_client.get_sender() as sender:
    sender.send(Message(msg))

I can provide a Valgrind log, which is a bit long, so maybe the log from Windows may be enough:

2019-07-11 16:41:07,442 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.END: 13> to <ConnectionState.START: 0>
2019-07-11 16:41:07,469 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0>
2019-07-11 16:41:07,678 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.START: 0> to <ConnectionState.HDR_SENT: 2>
2019-07-11 16:41:07,729 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.HDR_SENT: 2> to <ConnectionState.HDR_EXCH: 3>
2019-07-11 16:41:07,730 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.HDR_EXCH: 3> to <ConnectionState.OPEN_SENT: 7>
2019-07-11 16:41:07,783 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.OPEN_SENT: 7> to <ConnectionState.OPENED: 9>
2019-07-11 16:41:07,885 [INFO ]  CBS for connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' completed opening with status: 0
2019-07-11 16:41:07,936 [INFO ]  Token put complete with result: 0, status: 202, description: b'Accepted', connection: b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e'
2019-07-11 16:41:07,988 [INFO ]  Message sender b'sender-link-d285ab2d-de04-4638-8232-732ce63df535' state changed from <MessageSenderState.Idle: 0> to <MessageSenderState.Opening: 1> on connection: b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e'
2019-07-11 16:41:08,090 [INFO ]  Message sender b'sender-link-d285ab2d-de04-4638-8232-732ce63df535' state changed from <MessageSenderState.Opening: 1> to <MessageSenderState.Open: 2> on connection: b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e'
2019-07-11 16:41:08,295 [INFO ]  b'Failure: sending socket failed 10038.' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\deps\\azure-c-shared-utility\\adapters\\socketio_win32.c':b'socketio_send':422)
2019-07-11 16:41:08,296 [INFO ]  b'xio_send failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\deps\\azure-c-shared-utility\\adapters\\tlsio_schannel.c':b'send_chunk':484)
2019-07-11 16:41:08,297 [INFO ]  b'send_chunk failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\deps\\azure-c-shared-utility\\adapters\\tlsio_schannel.c':b'internal_send':525)
2019-07-11 16:41:08,298 [INFO ]  b'send failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\deps\\azure-c-shared-utility\\adapters\\tlsio_schannel.c':b'tlsio_schannel_send':1263)
2019-07-11 16:41:08,298 [INFO ]  b'xio_send failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_send_async':1190)
2019-07-11 16:41:08,299 [INFO ]  b'Cannot send encoded bytes' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':268)

Steps to reproduce

sbbs.py:

import json

from azure.servicebus import TopicClient, Message
from azure.servicebus.control_client import ServiceBusService

CN = "Endpoint=sb://xxx.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=yyy"
TOPIC = "topic_name"    

if __name__ == "__main__":
    topic_client = TopicClient.from_connection_string(CN, name=TOPIC)
    msg = json.dumps({})

    print("Sending a message 1")
    with topic_client.get_sender() as sender:
        sender.send(Message(msg))

    print("Sending a message 2")
    topic_client.get_sender().send(Message(msg))

Output

~ python3 sbbs.py                                  
Sending a message 1
Sending a message 2
[1]    1603 segmentation fault (core dumped)  python3 sbbs.py

It seems the problem is in the underlying azure-uamqp-c library - thus the segfault. But as a user of the python library, I am posting the issue here - apparently, there's a problem in the way the C lib is being used when not using the with/as statement.

[annatisch]

Thanks for the report @mxmxcz!
This would likely involve a fix in both SB and uAMQP - as while uamqp shouldn't be crashing, it looks like SB is also mishandling the sender object. I will open an issue in the Python SDK repo as well :)

[yunhaoling]

hey @mxmxcz , sorry for not getting to you sooner.
we have fixed the garbage collection issue in uamqp v1.2.5, can you try upgrading uamqp to the latest version to see if the problem gets resolved on your side?

[mrtnmch]

Unfortunately, I have left the project, therefore I don't have access to the codebase anymore. If the example above runs without any error, I'd consider the issue resolved.

Thanks for getting back to me, though.

[yunhaoling]

@mxmxcz , thanks for letting me know.

Closing the issue as it has been addressed in uamqp v1.2.5+.