plugin: Implement the htlc_accepted hook

CHANGELOG.md

@@ -13,7 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - JSON API: `listpeers` status now shows how many confirmations until channel is open (#2405)
 - Config: Adds parameter `min-capacity-sat` to reject tiny channels.
 - JSON API: `listforwards` now includes the time an HTLC was received and when it was resolved. Both are expressed as UNIX timestamps to facilitate parsing (Issue [#2491](https://github.com/ElementsProject/lightning/issues/2491), PR [#2528](https://github.com/ElementsProject/lightning/pull/2528))
-+- JSON API: new plugin hooks `invoice_payment` for intercepting invoices before they're paid, and `openchannel` for intercepting channel opens.
++- JSON API: new plugin hooks `invoice_payment` for intercepting invoices before they're paid, `openchannel` for intercepting channel opens, and `htlc_accepted` to decide whether to resolve, reject or continue an incoming or forwarded payment..
 - plugin: the `connected` hook can now send an `error_message` to the rejected peer.
 - Protocol: we now enforce `option_upfront_shutdown_script` if a peer negotiates it.
 - JSON API: `listforwards` now includes the local_failed forwards with failcode (Issue [#2435](https://github.com/ElementsProject/lightning/issues/2435), PR [#2524](https://github.com/ElementsProject/lightning/pull/2524))

common/sphinx.c

@@ -434,6 +434,8 @@ struct route_step *process_onionpacket(
 	deserialize_hop_data(&step->hop_data, paddedheader);
 
         memcpy(&step->next->mac, step->hop_data.hmac, SECURITY_PARAMETER);
+	step->raw_payload = tal_dup_arr(step, u8, paddedheader + 1,
+					HOP_DATA_SIZE - 1 - HMAC_SIZE, 0);
 
 	memcpy(&step->next->routinginfo, paddedheader + HOP_DATA_SIZE, ROUTING_INFO_SIZE);
 

common/sphinx.h

@@ -77,6 +77,7 @@ struct route_step {
 	enum route_next_case nextcase;
 	struct onionpacket *next;
 	struct hop_data hop_data;
+	u8 *raw_payload;
 };
 
 /**

contrib/plugins/helloworld.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 from lightning import Plugin
-
+import time
 
 plugin = Plugin()
 
@@ -34,5 +34,12 @@ def on_disconnect(plugin, id):
     plugin.log("Received disconnect event for peer {}".format(id))
 
 
+@plugin.hook("htlc_accepted")
+def on_htlc_accepted(onion, htlc, plugin):
+    plugin.log('on_htlc_accepted called')
+    time.sleep(20)
+    return {'result': 'continue'}
+
+
 plugin.add_option('greeting', 'Hello', 'The greeting I should use.')
 plugin.run()

doc/PLUGINS.md

@@ -15,7 +15,7 @@ variety of ways:
  - **Hooks** are a primitive that allows plugins to be notified about
    internal events in `lightningd` and alter its behavior or inject
    custom behaviors.
-   
+
 A plugin may be written in any language, and communicates with
 `lightningd` through the plugin's `stdin` and `stdout`. JSON-RPCv2 is
 used as protocol on top of the two streams, with the plugin acting as
@@ -27,14 +27,14 @@ executable (e.g. use `chmod a+x plugin_name`)
 During startup of `lightningd` you can use the `--plugin=` option to
 register one or more plugins that should be started. In case you wish
 to start several plugins you have to use the `--plugin=` argument
-once for each plugin. An example call might look like: 
+once for each plugin. An example call might look like:
 
 ```
 lightningd --plugin=/path/to/plugin1 --plugin=path/to/plugin2
 ```
 
 `lightningd` will write JSON-RPC requests to the plugin's `stdin` and
-will read replies from its `stdout`. To initialize the plugin two RPC 
+will read replies from its `stdout`. To initialize the plugin two RPC
 methods are required:
 
  - `getmanifest` asks the plugin for command line options and JSON-RPC
@@ -346,8 +346,114 @@ the string `reject` or `continue`.  If `reject` and
 there's a member `error_message`, that member is sent to the peer
 before disconnection.
 
+#### `htlc_accepted`
+
+The `htlc_accepted` hook is called whenever an incoming HTLC is accepted, and
+its result determines how `lightningd` should treat that HTLC.
+
+The payload of the hook call has the following format:
+
+```json
+{
+  "onion": {
+    "payload": "",
+    "per_hop_v0": {
+      "realm": "00",
+      "short_channel_id": "1x2x3",
+      "forward_amount": "42msat",
+      "outgoing_cltv_value": 500014
+    }
+  },
+  "next_onion": "[1365bytes of serialized onion]",
+  "shared_secret": "0000000000000000000000000000000000000000000000000000000000000000",
+  "htlc": {
+    "amount": "43msat",
+    "cltv_expiry": 500028,
+    "cltv_expiry_relative": 10,
+    "payment_hash": "0000000000000000000000000000000000000000000000000000000000000000"
+  }
+}
+```
+
+The `per_hop_v0` will only be present if the per hop payload has format `0x00`
+as defined by the specification. If not present an object representing the
+type-length-vale (TLV) payload will be added (pending specification). For detailed information about each field please refer to [BOLT 04 of the specification][bolt4], the following is just a brief summary:
+
+ - `onion.payload` contains the unparsed payload that was sent to us from the
+   sender of the payment.
+ - `onion.per_hop_v0`:
+   - `realm` will always be `00` since that value determines that we are using
+     the `per_hop_v0` format.
+   - `short_channel_id` determines the channel that the sender is hinting
+     should be used next (set to `0x0x0` if we are the recipient of the
+     payment).
+   - `forward_amount` is the amount we should be forwarding to the next hop,
+     and should match the incoming funds in case we are the recipient.
+   - `outgoing_cltv_value` determines what the CLTV value for the HTLC that we
+     forward to the next hop should be.
+ - `next_onion` is the fully processed onion that we should be sending to the
+   next hop as part of the outgoing HTLC. Processed in this case means that we
+   took the incoming onion, decrypted it, extracted the payload destined for
+   us, and serialized the resulting onion again.
+ - `shared_secret` is the shared secret we used to decrypt the incoming
+   onion. It is shared with the sender that constructed the onion.
+ - `htlc`:
+   - `amount` is the amount that we received with the HTLC. This amount minus
+     the `forward_amount` is the fee that will stay with us.
+   - `cltv_expiry` determines when the HTLC reverts back to the
+     sender. `cltv_expiry` minus `outgoing_cltv_expiry` should be equal or
+     larger than our `cltv_delta` setting.
+   - `cltv_expiry_relative` hints how much time we still have to claim the
+     HTLC. It is the `cltv_expiry` minus the current `blockheight` and is
+     passed along mainly to avoid the plugin having to look up the current
+     blockheight.
+   - `payment_hash` is the hash whose `payment_preimage` will unlock the funds
+     and allow us to claim the HTLC.
+
+The hook response must have one of the following formats:
+
+```json
+{
+  "result": "continue"
+}
+```
+
+This means that the plugin does not want to do anything special and
+`lightningd` should continue processing it normally, i.e., resolve the payment
+if we're the recipient, or attempt to forward it otherwise. Notice that the
+usual checks such as sufficient fees and CLTV deltas are still enforced.
+
+```json
+{
+  "result": "fail",
+  "failure_code": 4301
+}
+```
+
+`fail` will tell `lightningd` to fail the HTLC with a given numeric
+`failure_code` (please refer to the [spec][bolt4-failure-codes] for details).
+
+```json
+{
+  "result": "resolve",
+  "payment_key": "0000000000000000000000000000000000000000000000000000000000000000"
+}
+```
+
+`resolve` instructs `lightningd` to claim the HTLC by providing the preimage
+matching the `payment_hash` presented in the call. Notice that the plugin must
+ensure that the `payment_key` really matches the `payment_hash` since
+`lightningd` will not check and the wrong value could result in the channel
+being closed.
+
+Warning: `lightningd` will replay the HTLCs for which it doesn't have a final
+verdict during startup. This means that, if the plugin response wasn't
+processed before the HTLC was forwarded, failed, or resolved, then the plugin
+may see the same HTLC again during startup. It is therefore paramount that the
+plugin is idempotent if it talks to an external system.
 
 [jsonrpc-spec]: https://www.jsonrpc.org/specification
 [jsonrpc-notification-spec]: https://www.jsonrpc.org/specification#notification
+[bolt4]: https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md
 [bolt4-failure-codes]: https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#failure-messages
 [bolt2-open-channel]: https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#the-open_channel-message

lightningd/json.c

@@ -308,6 +308,24 @@ void json_add_u64(struct json_stream *result, const char *fieldname,
 	json_add_member(result, fieldname, "%"PRIu64, value);
 }
 
+void json_add_s64(struct json_stream *result, const char *fieldname,
+		  int64_t value)
+{
+	json_add_member(result, fieldname, "%"PRIi64, value);
+}
+
+void json_add_u32(struct json_stream *result, const char *fieldname,
+		  uint32_t value)
+{
+	json_add_member(result, fieldname, "%d", value);
+}
+
+void json_add_s32(struct json_stream *result, const char *fieldname,
+		  int32_t value)
+{
+	json_add_member(result, fieldname, "%d", value);
+}
+
 void json_add_literal(struct json_stream *result, const char *fieldname,
 		      const char *literal, int len)
 {

lightningd/json.h

@@ -127,6 +127,15 @@ void json_add_num(struct json_stream *result, const char *fieldname,
 /* '"fieldname" : value' or 'value' if fieldname is NULL */
 void json_add_u64(struct json_stream *result, const char *fieldname,
 		  uint64_t value);
+/* '"fieldname" : value' or 'value' if fieldname is NULL */
+void json_add_s64(struct json_stream *result, const char *fieldname,
+		  int64_t value);
+/* '"fieldname" : value' or 'value' if fieldname is NULL */
+void json_add_u32(struct json_stream *result, const char *fieldname,
+		  uint32_t value);
+/* '"fieldname" : value' or 'value' if fieldname is NULL */
+void json_add_s32(struct json_stream *result, const char *fieldname,
+		  int32_t value);
 /* '"fieldname" : true|false' or 'true|false' if fieldname is NULL */
 void json_add_bool(struct json_stream *result, const char *fieldname,
 		   bool value);

lightningd/lightningd.c

@@ -720,9 +720,6 @@ int main(int argc, char *argv[])
 	/*~ Initialize the transaction filter with our pubkeys. */
 	init_txfilter(ld->wallet, ld->owned_txfilter);
 
-	/*~ Pull peers, channels and HTLCs from db. */
-	load_channels_from_wallet(ld);
-
 	/*~ Get the blockheight we are currently at, UINT32_MAX is used to signal
 	 * an uninitialized wallet and that we should start off of bitcoind's
 	 * current height */
@@ -747,6 +744,13 @@ int main(int argc, char *argv[])
 	setup_topology(ld->topology, &ld->timers,
 		       min_blockheight, max_blockheight);
 
+	/*~ Pull peers, channels and HTLCs from db. Needs to happen after the
+	 *  topology is initialized since some decisions rely on being able to
+	 *  know the blockheight. */
+	db_begin_transaction(ld->wallet->db);
+	load_channels_from_wallet(ld);
+	db_commit_transaction(ld->wallet->db);
+
 	/*~ Now create the PID file: this errors out if there's already a
 	 * daemon running, so we call before trying to create an RPC socket. */
 	pid_fd = pidfile_create(ld);