Node crashes after an issue with multiplex

[zerofeerouting]

Issue and Steps to Reproduce

lightning_connectd: FATAL SIGNAL 6 (version v0.11.2rc2-3-g4142823)
0x55e1f6ef3b20 send_backtrace
	common/daemon.c:33
0x55e1f6ef3baa crashdump
	common/daemon.c:46
0x7fd8c9bc772f ???
	???:0
0x7fd8c99c27bb ???
	???:0
0x7fd8c99ad534 ???
	???:0
0x55e1f6f313d1 call_error
	ccan/ccan/tal/tal.c:93
0x55e1f6f314ad check_bounds
	ccan/ccan/tal/tal.c:165
0x55e1f6f314c4 to_tal_hdr
	ccan/ccan/tal/tal.c:173
0x55e1f6f322f7 tal_resize_
	ccan/ccan/tal/tal.c:694
0x55e1f6eebc51 multiplex_subd_setup
	connectd/multiplex.c:1158
0x55e1f6eebd02 activate_subd
	connectd/multiplex.c:470
0x55e1f6eed1ab read_body_from_peer_done
	connectd/multiplex.c:1035
0x55e1f6f26af2 next_plan
	ccan/ccan/io/io.c:59
0x55e1f6f26fe7 do_plan
	ccan/ccan/io/io.c:407
0x55e1f6f27019 io_ready
	ccan/ccan/io/io.c:417
0x55e1f6f2891e io_loop
	ccan/ccan/io/poll.c:453
0x55e1f6eea167 main
	connectd/connectd.c:2165
0x7fd8c99af09a ???
	???:0
0x55e1f6ee2e99 ???
	???:0
0xffffffffffffffff ???
	???:0

Nothing spectacular in the logs unfortunately (and no crashdump was generated)

2022-07-02T20:32:37.914Z INFO    03ed6942f3a32b143b3479b85ed64453c9b7fbd2057ea15094c0486a39df8b1691-channeld-chan#135: Peer connection lost
2022-07-02T20:32:37.914Z INFO    03ed6942f3a32b143b3479b85ed64453c9b7fbd2057ea15094c0486a39df8b1691-chan#135: Peer transient failure in CHANNELD_NORMAL: channeld: Owning subdaemon channeld died (62208)
2022-07-02T20:32:37.914Z INFO    03e472ddc37b014ce7d5f942b00b28fb1db53d461c4a89173962abd4b331a85179-channeld-chan#11367: Peer connection lost
2022-07-02T20:32:37.914Z INFO    03e472ddc37b014ce7d5f942b00b28fb1db53d461c4a89173962abd4b331a85179-chan#11367: Peer transient failure in CHANNELD_NORMAL: channeld: Owning subdaemon channeld died (62208)
2022-07-02T20:32:37.914Z INFO    021ceeb2fe8d92a81e38f7bb2018b2219d18a31cae6f76f9d979a9195de5fee7a6-channeld-chan#6514: Peer connection lost
2022-07-02T20:32:37.914Z INFO    021ceeb2fe8d92a81e38f7bb2018b2219d18a31cae6f76f9d979a9195de5fee7a6-chan#6514: Peer transient failure in CHANNELD_NORMAL: channeld: Owning subdaemon channeld died (62208)
2022-07-02T20:32:37.914Z INFO    03d328c2836861d59c05eb0e33f03ffaafdeb69a409ccda5e6a3a0ba6607954c10-channeld-chan#9118: Peer connection lost
2022-07-02T20:32:37.914Z INFO    03d328c2836861d59c05eb0e33f03ffaafdeb69a409ccda5e6a3a0ba6607954c10-chan#9118: Peer transient failure in CHANNELD_NORMAL: channeld: Owning subdaemon channeld died (62208)
2022-07-02T20:32:37.914Z INFO    02045f289f0de16b275e925aff584bc6c626dddc0f29e15b367d68a35de445d98b-channeld-chan#8018: Peer connection lost
lightningd: gossipd failed (exit status 2), exiting.
2022-07-02T20:32:37.914Z INFO    02045f289f0de16b275e925aff584bc6c626dddc0f29e15b367d68a35de445d98b-chan#8018: Peer transient failure in CHANNELD_NORMAL: channeld: Owning subdaemon channeld died (62208)
Lost connection to the RPC socket.
Lost connection to the RPC socket.
Lost connection to the RPC socket.
Lost connection to the RPC socket.
Lost connection to the RPC socket.
2022-07-02T20:32:39.553Z INFO    connectd: Static Tor service onion address: "xtdo5qvvfwcjaruj6z4acdcw4azagn6tdgac4ajnekjdn4ghr6qw2nqd.onion:9735,0.0.0.0:9735" bound from extern port 9735 
2022-07-02T20:32:40.098Z INFO    plugin-bcli: bitcoin-cli initialized and connected to bitcoind.
2022-07-02T20:32:54.976Z **BROKEN** lightningd: Local update for bad scid 728635x990x1
2022-07-02T20:32:55.001Z **BROKEN** lightningd: Local update for bad scid 736709x2001x1
2022-07-02T20:32:55.021Z **BROKEN** lightningd: Local update for bad scid 737131x1909x1
2022-07-02T20:32:55.030Z **BROKEN** lightningd: Local update for bad scid 728838x790x1

getinfo output

v0.11.2rc2-3-g4142823

[wtogami]

$ git diff 4142823 v0.11.2
$ 

Note: Commit hash 4142823 appears to be identical to v0.11.2?

[cdecker]

Yep, that's because rc2 became the 0.11.2 final release.

[cdecker]

Looks like the peer->subds array isn't initialized at this point:

lightning/connectd/multiplex.c

Line 1158 in c9587ab

tal_arr_expand(&peer->subds, subd);

Seems strange though since the constructor initializes it:

lightning/connectd/connectd.c

Line 306 in c9587ab

peer->subds = tal_arr(peer, struct subd *, 0);

[wtogami]

Yep, that's because rc2 became the 0.11.2 final release.

The commit history is confusing because 4142823 is rc2 plus two commits.
rc3 which became v0.11.2 became the final release but has a different githash.
They are in fact the same code but there might have been confusion.