Owner API finalize_tx responds with Fee: Missing fee fields error

[marekyggdrasil]

Describe the bug
I am trying to reproduce the invoice flow using Owner API. When I receive the slatepack from the recipient and attempt to finalize it I encounter

Fee: Missing fee fields error

error. The request payload that is being attempted to get finalized is the following

{   'slate': {   'coms': [   {   'c': '09fb58da3e87439f0085ab9ccf0aff592f9317b6253ae955e76435ecbd85c94c3e'},
                             {   'c': '08baaebfa8baca00f38adee8c22c8a084c2bbad711165e3dad3da016a3d761c7e6',
                                 'p': 'b8b2662fc0a2a6b8630d2499deeaaff64e3d80938248ce41e01b8f3dbdcc79e085ffd24857a2df3f65d9707adbb5ad6187825e884224df70eb264c27c328b67b0791985815ca0f4ecdaf5a1dea35728ef88fbcb4e1a847eae021a90589c007155edff028d9987a25fb9e7d26005ec4694435912e8427c0460d11c8d598f9f330db88a4fe601d440802586b36ab146ed4c2569dfe243298860154736b8fbf24e063c000cea1f4c46d01bf8d6429a1d4a95050c083194e7b25af18ba09d7926fd9a4ee29bf2cdfddca16b87735bee526d879e1cf69afef65fc08397a8fd415460e384e17e5618070dde9efc99315cd16f9903e6f63660af9d1ddcfdd1e9d07a0130d8aa0119245f93d7d79e3235a367989e6270588e2456d4ce05cc2652360bfc7f1265fac19f39c83902ef8d7080307572a075c4c3ee310ea8da4c7618f3d839113c14a7e3a4a7810771f36e28bd1f48444117020e647d183cae2ba884f490dafa269030d54df195b19a7c780917f0eada05896c693f135346f45d6d07da976753b1014eb188fc1cc53a877fbda467f0dcd07fe8e9d68e2ac5f9853fb2e01681552deae5b87fcf1961f2cf42707ce9c0d555736fb9a696a0572f4f6881e78c78a607a1243e16fb0d271bab34b435c4e68317fdd91b53e833469c5ef08595d71d68ad2277c7ec7972b7a38a5eabdcb714960e313c41309a40bbcd9398274f96f5fe6de58be3b263585e4b8fde1cd86e5dd059cadc8127eb17e88aae395d7cae67ddc37bf1804227ef43240dc41b7a18a566fc4c8f42f6692744cc8f8ad37b8b0d7b1f17dc1384b3d20b121c23d8b59ecde517947e921a560e72456c9083df4ba3fa7c4f85215f737b007268fe366b5b277db478b7357b43d70e085b873bd0379dd6d4a5c3711357ffd8446ce5b95c72c01bb6935cec0a10423293369247ff0f5b7f17f47'}],
                 'fee': '23000000',
                 'id': '5d1d3210-044b-4c4b-9d3e-297fb5062aed',
                 'off': '93567c4ab93ae545cc2b43da18ddced825dbd119e19ad25b41133fe6ff1903bc',
                 'sigs': [   {   'nonce': '03bc3f344e0850dca43e97ddeb7703c5b8328270b2ad35841a25cdd68ea06937e9',
                                 'part': 'e93769a08ed6cd251a8435adb2708232b8c50377ebdd973ea4dc50084e343fbc768efa7c088bbf376807da22ee47f67ccc0ab8f38bc01b59e3f1bbddcb030180',
                                 'xs': '026cca4c745757544b03e01ce30da80c0b698201c686a8703756ffc5a6c8e50ca6'}],
                 'sta': 'I2',
                 'ver': '4:3'},
    'token': 'b15504ff21f639e2fa7748dece90a294086d7c190d7b9f4e21b38ff49f4c037d'}

and in the slate we can observe there is a fee attribute which confuses me.

I was trying to investigate by browsing through the wallet source but the only thing I could find is

grin-wallet/libwallet/src/internal/selection.rs

Line 661 in bdc5bd7

.ok_or_else(|| ErrorKind::Fee("Missing fee fields".into()))?;

I am reporting it as a bug, but probably I am just using it wrong.

Disclaimer: This is testnet wallet and if any sensitive data is included there, it is not relevant.

To Reproduce
Steps to reproduce the behavior:

1. Prepare two wallets: Alice and Bob
2. Run Owner API listener using Alice wallet
3. Make request to open_wallet method to avoid keychain mask error.
4. Make a request to issue_invoice_tx method. It will provide a slate.
5. Make request to create_slatepack_message, provide the above slate, sender index set to 0 and put tgrin address in the recipients list parameter. This will provide an encrypted slatepack message.
6. Using Bob's wallet run grin-wallet pay and paste this slatepack. It will reply with a new one.
7. Back to owner API, execute slate_from_slatepack_message to extract slate from the Bob's slatepack. Set secret_indices to [0] as we set that for the sender index in (4).
8. Skip outputs locking and directly execute finalize_tx using the extracted slate.
9. Owner API response will indicate an error has occurred with message Fee: Missing fee fields error

Expected behavior
I would expect to receive a finalized slate.

Screenshots
N/A

Desktop (please complete the following information):

OS:

$ uname -s -r -v -m -p -i -o
Linux 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64 unknown unknown GNU/Linux

Wallet version

$ grin-wallet -V
grin-wallet 5.1.0-alpha.1

Build at commit

$ git rev-parse HEAD
34d23eb17ada3ffc09d76b1fa66ba9778abc29ea

Additional context

Please help me Obi-Wan Kenobi. You're my only hope.

[marekyggdrasil]

Thanks @deevope for your DM! The fix was merging the fix by @pkariz to my master. Now getting the processed slate without any errors.

{'coms': [{'c': '09fb58da3e87439f0085ab9ccf0aff592f9317b6253ae955e76435ecbd85c94c3e'}, {'c': '08a73ac0cfa2243883ccca3c0bc835db3209f73ae9fced441655518032d4390f78', 'p': '388e8d3e627256eca969651fed281612c71f748a0dfe7ce2369a702e334dd49c44f611549066706fa1b267ef362fad726052b2f2215cab1b8eeecf161dae49160b2d85e2088bb1c0a2799dda5f74231b9d6b9fd8b1d9d08b57adc607bc5a51d9dc189a9273a4dd396f20aee76df5a1b726ce78585b70563619ed2d07faa023d79e233aa01e338f1218f233f98fa0606a40346328ffb281f9f1ce98bc3953b91699837e34e7169492f526aad2bbade294a5b2addcc2657c9e14577f4d0d0b3fbbd2713f317e2bdc9cb99d68d035d546d0dcd7457fdca61b2528694df9dc4c5ca48d5fa7944080755c71270f68e0371f2aea081a70b5e747dbf4d12ad9468ff19371246707cca8abb3156ab58b867f9c80eeabad9d61fdabdc2575492bc52a32491e5632d28a730ac6740568a622e09b753cb1c09a927b723c0abfdadeaa91abc5abfe650f22a3e3867b1172a58f56eaee142216ef1b60e1ec8ea8e2f797139fcbe5b900bc39780ba9c57401d6748b7f81e31d2a3e2d1ad79107626e36274515a88ca5414cf1f581baa6669036494b991eff034f02fe5be0e287581159fec03ade32097e5c46022ea50b489798906f7ad304348918c91526563072252bb87cd161a51cc9780bd71b7dd87bfc61e8998c1334de8d95d0f2665763e45ac5e943b9c091b5fb104065f935b06164d2c6ff95aac6424ef4f3dc9c6e00dce5cb38e37c2ee97c7386b756bf6634814d8952496b2cf2c1241a955558a439ad298407b3e8ce71780c438e41aa6d8757448b194a80e473b668352f35b8ce9623c1fa2b62745ec366debcaa23d42d9e7f104bb0bcf16b2ebac43e44305fad727c1f183f108bca64f5f1473b58aac05f23539c120714bb7fc9c7f87ef00fa143739ad79a721d2b4f1074c133ed2d1dafbadd973f65615efb13538cd575be162e11031ee46133c91e61a7'}, {'c': '08baaebfa8baca00f38adee8c22c8a084c2bbad711165e3dad3da016a3d761c7e6', 'p': 'b8b2662fc0a2a6b8630d2499deeaaff64e3d80938248ce41e01b8f3dbdcc79e085ffd24857a2df3f65d9707adbb5ad6187825e884224df70eb264c27c328b67b0791985815ca0f4ecdaf5a1dea35728ef88fbcb4e1a847eae021a90589c007155edff028d9987a25fb9e7d26005ec4694435912e8427c0460d11c8d598f9f330db88a4fe601d440802586b36ab146ed4c2569dfe243298860154736b8fbf24e063c000cea1f4c46d01bf8d6429a1d4a95050c083194e7b25af18ba09d7926fd9a4ee29bf2cdfddca16b87735bee526d879e1cf69afef65fc08397a8fd415460e384e17e5618070dde9efc99315cd16f9903e6f63660af9d1ddcfdd1e9d07a0130d8aa0119245f93d7d79e3235a367989e6270588e2456d4ce05cc2652360bfc7f1265fac19f39c83902ef8d7080307572a075c4c3ee310ea8da4c7618f3d839113c14a7e3a4a7810771f36e28bd1f48444117020e647d183cae2ba884f490dafa269030d54df195b19a7c780917f0eada05896c693f135346f45d6d07da976753b1014eb188fc1cc53a877fbda467f0dcd07fe8e9d68e2ac5f9853fb2e01681552deae5b87fcf1961f2cf42707ce9c0d555736fb9a696a0572f4f6881e78c78a607a1243e16fb0d271bab34b435c4e68317fdd91b53e833469c5ef08595d71d68ad2277c7ec7972b7a38a5eabdcb714960e313c41309a40bbcd9398274f96f5fe6de58be3b263585e4b8fde1cd86e5dd059cadc8127eb17e88aae395d7cae67ddc37bf1804227ef43240dc41b7a18a566fc4c8f42f6692744cc8f8ad37b8b0d7b1f17dc1384b3d20b121c23d8b59ecde517947e921a560e72456c9083df4ba3fa7c4f85215f737b007268fe366b5b277db478b7357b43d70e085b873bd0379dd6d4a5c3711357ffd8446ce5b95c72c01bb6935cec0a10423293369247ff0f5b7f17f47'}], 'fee': '23000000', 'id': '5d1d3210-044b-4c4b-9d3e-297fb5062aed', 'off': '04709c0f416ce0c105639a7f57118624d4f1a28264621be0fed2a3453c3fcdbd', 'sigs': [{'nonce': '03bc3f344e0850dca43e97ddeb7703c5b8328270b2ad35841a25cdd68ea06937e9', 'part': 'e93769a08ed6cd251a8435adb2708232b8c50377ebdd973ea4dc50084e343fbc768efa7c088bbf376807da22ee47f67ccc0ab8f38bc01b59e3f1bbddcb030180', 'xs': '026cca4c745757544b03e01ce30da80c0b698201c686a8703756ffc5a6c8e50ca6'}, {'nonce': '03c68c3e43caf841276000fcdcf064ec1440a2c2c81efb2279881fecca3d13ed1f', 'part': '1fed133dcaec1f887922fb1ec8c2a24014ec64f0dcfc00602741f8ca433e8cc69254cc696f960406657e478a0e75ffb619303d039aaea70c20c70a6a9564bc2d', 'xs': '02cb0d1fef16967a0fd6059bb6261bd767a885b00598240b3f79010be42c0fa79c'}], 'sta': 'I3', 'ver': '4:3'}

I let you guys decide if it should be closed now or if you want to wait until fix is merged to master.

[deevope]

Let it open, so we can track the issue until it's fixed on the grin wallet master branch. IIRC @pkariz was in doubt if it was the best fix for that.

[pkariz]

Let it open, so we can track the issue until it's fixed on the grin wallet master branch. IIRC @pkariz was in doubt if it was the best fix for that.

yes, that's exactly what it was. Not only if it was the best solution, but i also wasn't 100% sure it's the correct one. The last 3 commits here are what i have changed to make SRS and RSR work through wallet api

[marekyggdrasil]

Noted. We will proceed with fix from @pkariz just to get our environment work and I will pay attention to this ticket getting closed. Then we will use the latest release that would include the fix of your preference. Until then I will keep merging from master.

[marekyggdrasil]

Encountered the same problem while working on the grin-telegram-bot and reproduced it using 5.1.0 version wallet.

[pkariz]

Does that 5.1.0 you're running include the fix i made? If not then i guess this hasn't been fixed in this repo yet

[marekyggdrasil]

It does not include the patch you shared. It was just a temp solution for previous project, it was also causing some other bugs. We need to have the both SRS and RSR flows working in the Owner API without expecting users to search for and apply some patches and compile wallet from scratch. It is a basic feature that seems to got broken at some point. I will try to find some time and test wallets and identify at which commit it stopped working.

[marekyggdrasil]

So I did run some tests.

Unfortunately anything before ba9a4982df8c0a5c34583769e2c35a7669762f58 does not compile on my machine and after this starting from 1dd85690a1c08b3508385f0c11abf29a05b664d9 reproduces the problem.

Sorry I could not contribute much, but I did develop a tool for fast testing of wallets. It does RSR exchange between two local wallets saving some copy-paste time. Here it is: https://github.com/grinventions/rsr-test

[marekyggdrasil]

I decided to run some of the pre-compiled binaries from wallet's GitHub releases, but unfortunately, the nothing below 5.0.0 runs on my device due to this problem but at least this way we found that this has been fixed.

[yeastplume]

On radar, will try and reproduce over the next couple of weeks

[yeastplume]

This should be fixed now via #701, but note that existing deployments can just call finalize transaction on the foreign API instead and all transactions should work as expected.

[marekyggdrasil]

Thank you for checking, @yeastplume