Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lnd crash after 30 seconds. #1292

Closed
robtex opened this issue May 28, 2018 · 7 comments · Fixed by #1294
Closed

lnd crash after 30 seconds. #1292

robtex opened this issue May 28, 2018 · 7 comments · Fixed by #1294

Comments

@robtex
Copy link

robtex commented May 28, 2018

just upgraded to version 524291d
since then lnd crashes shortly after start

with bitcoind:

2018-05-28 15:10:05.946 [ERR] CNCT: ChannelArbitrator(15ced24c5eeb2167765a20958470bb18cc062bc96e13a5e62d7bbee7d5979398:1): unable to broadcast close tx: -26: bad-txns-in-belowout, value in (0.002) < value out (0.00202286) (code 16)
2018-05-28 15:10:05.946 [ERR] CNCT: unable to advance state: -26: bad-txns-in-belowout, value in (0.002) < value out (0.00202286) (code 16)
2018-05-28 15:10:05.946 [ERR] SRVR: unable to start server: -26: bad-txns-in-belowout, value in (0.002) < value out (0.00202286) (code 16)

-26: bad-txns-in-belowout, value in (0.002) < value out (0.00202286) (code 16)

with btcd:

2018-05-28 15:11:28.267 [ERR] CNCT: ChannelArbitrator(15ced24c5eeb2167765a20958470bb18cc062bc96e13a5e62d7bbee7d5979398:1): unable to broadcast close tx: -22: TX rejected: total value of all transaction inputs for transaction 37c39e8dcf1f7b4e31a266c27e8bf4914ac228203db0a97ded691b87eeba3cfe is 200000 which is less than the amount spent of 202286
2018-05-28 15:11:28.267 [ERR] CNCT: unable to advance state: -22: TX rejected: total value of all transaction inputs for transaction 37c39e8dcf1f7b4e31a266c27e8bf4914ac228203db0a97ded691b87eeba3cfe is 200000 which is less than the amount spent of 202286
2018-05-28 15:11:28.267 [ERR] SRVR: unable to start server: -22: TX rejected: total value of all transaction inputs for transaction 37c39e8dcf1f7b4e31a266c27e8bf4914ac228203db0a97ded691b87eeba3cfe is 200000 which is less than the amount spent of 202286

-22: TX rejected: total value of all transaction inputs for transaction 37c39e8dcf1f7b4e31a266c27e8bf4914ac228203db0a97ded691b87eeba3cfe is 200000 which is less than the amount spent of 202286

some other things in the log that looks bad but doesn't crash:

2018-05-28 11:35:04.080 [DBG] LNWL: Could not resend transaction 76552c1183731ce514d4908934ed1a07e449ee3f13e41000f6f396f059112db7: -22: TX rejected: transaction output has negative value of -643
2018-05-28 11:35:04.081 [DBG] LNWL: Could not resend transaction fc09569edc501ef75f6ce61d5116ee6319ba8096adeb6d7d6884cb6902c51c39: -22: TX rejected: transaction output has negative value of -4633
2018-05-28 11:35:04.082 [DBG] LNWL: Could not resend transaction 6a6b136b5bfeae3fe06014e74628e38a30b981cd0f139caefc12b315f5fbdc31: -22: TX rejected: transaction 6a6b136b5bfeae3fe06014e74628e38a30b981cd0f139caefc12b315f5fbdc31 is not standard: transaction output 0: payment of 50 is dust
2018-05-28 11:35:04.083 [DBG] LNWL: Could not resend transaction da182019d8853332ebb4d43999df961d0b661631d220043145393c8e7a85f7aa: -22: TX rejected: transaction output has negative value of -45
2018-05-28 11:35:04.084 [DBG] LNWL: Could not resend transaction 3ee78d447769774ab6de8b7eb73af8322646eb747ca3f743285233faf53a19e4: -22: TX rejected: transaction already exists   2018-05-28 11:35:04.085 [DBG] LNWL: Could not resend transaction 032dd4db8bb8aef899bbf36af8b1fecf8af9a77084daf973b93c4f0d03dfd7aa: -22: TX rejected: transaction output has negative value of -1850
2018-05-28 11:35:04.086 [DBG] LNWL: Could not resend transaction 52ffee32a8f2d7a4f9f7fc640f1abd23efd35977f092189a4c017637582122e1: -22: TX rejected: transaction output has negative value of -453
2018-05-28 11:35:04.086 [DBG] LNWL: Could not resend transaction 3e534322a3f85e7283f0962f6543940849acc6a28fb45b8f86690f676be7aa5d: -22: TX rejected: transaction output has negative value of -330
2018-05-28 11:35:04.087 [DBG] LNWL: Could not resend transaction ddf46c985005f6913a06439cdd497b9e74d1bc309da5bf3da219acb6e26e5e88: -22: TX rejected: transaction output has negative value of -45
2018-05-28 11:35:04.088 [DBG] LNWL: Could not resend transaction c6b5d91543cdeabe528247ba7b1584d4df6c4bb1627714a506012cd3358285c0: -22: TX rejected: transaction output has negative value of -668
2018-05-28 11:35:04.089 [DBG] LNWL: Could not resend transaction 80d93e5a0437457c85d4434c6dc3e84162f889aaba9ff88ad79812ed05762a6f: -22: TX rejected: transaction output has negative value of -709
2018-05-28 11:35:04.092 [DBG] LNWL: Could not resend transaction d3882957348fd0cdce3882503f030a58472e1fc5b95473777824a35868d308a4: -22: TX rejected: transaction already exists
2018-05-28 11:35:04.095 [DBG] LNWL: Could not resend transaction fd28e78fe75c7c2f06590ec4c163f0c97ba9fb3ea5822accdb4473f9080a93c1: -22: TX rejected: transaction already exists
2018-05-28 11:35:04.099 [DBG] LNWL: Could not resend transaction f7b15e4977c1178fcbbdb32693e56ade7776bb08229233a5374264cc9ebe1b87: -22: TX rejected: transaction already exists
2018-05-28 11:35:04.100 [DBG] LNWL: Could not resend transaction cd93f5c587f582ba1b2fbecdd908c9e07bda0a823c3e1caa2c0c79737c19d248: -22: TX rejected: transaction output has negative value of -519
2018-05-28 11:35:04.101 [DBG] LNWL: Could not resend transaction 972a11ff5eacadb1132e47b7cf7eaa1393091b2a36067d84af30ee912c8c4a17: -22: TX rejected: transaction output has negative value of -710
2018-05-28 11:35:04.102 [DBG] LNWL: Could not resend transaction 0febf306b42f38f9a3e0fa8a369acc61ba2c36b3fef95831ab35b239908c31a8: -22: TX rejected: transaction 0febf306b42f38f9a3e0fa8a369acc61ba2c36b3fef95831ab35b239908c31a8 is not standard: transaction output 0: payment of 50 is dust
2018-05-28 11:35:04.103 [DBG] LNWL: Could not resend transaction 58f366d4778ae08e34658e9da0d594ffa61bdf567aef513db4d83220b4e7c840: -22: TX rejected: transaction 58f366d4778ae08e34658e9da0d594ffa61bdf567aef513db4d83220b4e7c840 is not standard: transaction output 0: payment of 51 is dust
2018-05-28 11:35:04.104 [DBG] LNWL: Could not resend transaction 1ccb470413dd7ac109e0fd1af4f24610dfca764188bdd5455cb59ee9c87a67ca: -22: TX rejected: transaction 1ccb470413dd7ac109e0fd1af4f24610dfca764188bdd5455cb59ee9c87a67ca is not standard: transaction output 0: payment of 282 is dust
2018-05-28 11:35:04.105 [DBG] LNWL: Could not resend transaction 3ee905b1a247777462096878a806e31e5fc32623124333e4bdbb51eb7bcc39fe: -22: TX rejected: transaction output has negative value of -1023
2018-05-28 11:35:04.107 [DBG] LNWL: Could not resend transaction dd1b4c533cc1e4fd6e6933c3c5702cf29a54eeed3d75dca9ae002941fd81ca9b: -22: TX rejected: transaction already exists
2018-05-28 11:35:04.108 [DBG] LNWL: Could not resend transaction 95a79b0f59ed2845d0e92d0307baaa2653d1163620c0a019f82196dcf309c76d: -22: TX rejected: transaction output has negative value of -98
2018-05-28 11:35:04.109 [DBG] LNWL: Could not resend transaction e285e1078e2b14cf93a791c25d6d26a02285d436cff86249b072b735b33fff7e: -22: TX rejected: transaction output has negative value of -1279
2018-05-28 11:35:04.110 [DBG] LNWL: Could not resend transaction 9ad578b3744c6cecd7d47031178e34c50e1028a3983b52dfbfc756aadec4e5c6: -22: TX rejected: transaction output has negative value of -2800
2018-05-28 11:35:04.110 [DBG] LNWL: Could not resend transaction 64040bacfd7bee3cee98ddd166a0e114ab28ee037f796374a5a89ac9c9e58e8d: -22: TX rejected: transaction 64040bacfd7bee3cee98ddd166a0e114ab28ee037f796374a5a89ac9c9e58e8d is not standard: transaction output 0: payment of 117 is dust
2018-05-28 11:35:04.111 [DBG] LNWL: Could not resend transaction b5f6d716508aaebab69cc2fd63ffd43cac12658de415cab067a0efb6d9ca08e8: -22: TX rejected: transaction output has negative value of -1213                                                                                                                                                                 2018-05-28 11:35:04.112 [DBG] LNWL: Could not resend transaction faace583c679a3614936cf82d47a08a9e3b3381fbde75e571e4219aea2661aac: -22: TX rejected: transaction faace583c679a3614936cf82d47a08a9e3b3381fbde75e571e4219aea2661aac is not standard: transaction output 0: payment of 50 is dust```


@Roasbeef
Copy link
Member

Roasbeef commented May 28, 2018

Applying this small patch will allow you to boot again:

diff --git a/contractcourt/chain_arbitrator.go b/contractcourt/chain_arbitrator.go
index c567dcf9..05ce17db 100644
--- a/contractcourt/chain_arbitrator.go
+++ b/contractcourt/chain_arbitrator.go
@@ -453,9 +453,11 @@ func (c *ChainArbitrator) Start() error {
 			return err
 		}
 	}
-	for _, arbitrator := range c.activeChannels {
+	for chanPoint, arbitrator := range c.activeChannels {
 		if err := arbitrator.Start(); err != nil {
-			return err
+			log.Warnf("unable to start channel arb for: %v",
+				chanPoint)
+			continue
 		}
 	}

We're aware of the greater issue and are working on a fix atm. The set of things in the logs are in a sense benign: simply you have a bunch of tiny channels that can't be sweep on chain at the current fee level or were just far too small.

@Stadicus
Copy link

@Roasbeef Is this issue specific to the update to v0.4.2-beta? I updated a few days ago, and with a restart today the daemon started crashing on start. After applying your patch above, it's working fine again.

Are there plans to mitigate this issue in an upcoming release? What's your advice for the moment regarding upgrades? I worry a bit about lots of people upgrading their nodes and LND just stops working. Not everybody is able to patch & compile from source.

@Roasbeef
Copy link
Member

Roasbeef commented Jun 20, 2018 via email

@Stadicus
Copy link

If I understand correctly, these are dust channels, with not enough balance to be closed, correct? If I already have a channel like this, is there anything one can do to preempt this LND crash, without manually applying this patch? Or is this already merged into master?

@Roasbeef
Copy link
Member

Roasbeef commented Jun 20, 2018 via email

@Stadicus
Copy link

Thanks for clarifying and all your incredible work!

@Stadicus
Copy link

@Roasbeef, would applying the patches regarding "bad-txns-vout-empty" again (mentioned in issue #1670) lead to the same issue that I need to "forget" the channel with dust output? Is this rpc available in LND 0.5-beta?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants