feat: API: support typed errors over RPC #9061

arajasek · 2022-07-19T17:33:40Z

Related Issues

This is work towards, but does not finish #9041

Proposed Changes

Updates go-jsonrpc.
Introduces API errors, with only one case right now -- OutOfGas.
Demo test.

Additional Info

Needs Support error codes / typed errors go-jsonrpc#36 & tag
The chief limitation here is that wrapped errors do not get picked up by the RPC code, which relies on reflection. This is a non-trivial issue, although it is not a blocker. I'd like to discuss this with @magik6k, but it doesn't preclude landing this PR.

Checklist

Before you mark the PR ready for review, please make sure that:

All commits have a clear commit message.
The PR title is in the form of of <PR type>: <area>: <change being made>
- example: fix: mempool: Introduce a cache for valid signatures
- PR type: fix, feat, INTERFACE BREAKING CHANGE, CONSENSUS BREAKING, build, chore, ci, docs,perf, refactor, revert, style, test
- area: api, chain, state, vm, data transfer, market, mempool, message, block production, multisig, networking, paychan, proving, sealing, wallet, deps
This PR has tests for new functionality or change in behaviour
If new user-facing features are introduced, clear usage guidelines and / or documentation updates should be included in https://lotus.filecoin.io or Discussion Tutorials.
CI is green

codecov · 2022-07-19T17:55:01Z

Codecov Report

❗ No coverage uploaded for pull request base (release/v1.18.0@37be576). Click here to learn what that means.
The diff coverage is n/a.

❗ Current head 30bd0ed differs from pull request most recent head 3e81c56. Consider uploading reports for the commit 3e81c56 to get more accurate results

Additional details and impacted files

@@                Coverage Diff                 @@
##             release/v1.18.0    #9061   +/-   ##
==================================================
  Coverage                   ?   41.02%           
==================================================
  Files                      ?      708           
  Lines                      ?    78898           
  Branches                   ?        0           
==================================================
  Hits                       ?    32365           
  Misses                     ?    41014           
  Partials                   ?     5519

magik6k · 2022-08-01T12:08:19Z

The chief limitation here is that wrapped errors do not get picked up by the RPC code, which relies on reflection.

We could make go-jsonrpc recursively errors.Unwrap the error until it hits a registered error (with some depth limit just in case), then have it return a "wrapped" version of that error:

What the error code would be in that case is a good question, we can go one of a few routes:
- Pre-register the 'wrapped' error type, which would marshal into the {errorMsg, subErrCode, subErr} - probably closest how errors are wrapped in Go, but weird in non-Go apps - you can't tell the error type just by looking at the error code in the jsonrpc response
- Register wrapped error types for each wrapable error type - you can tell the error just by looking at the error code, but now there are 2 error codes per error type.
- Mix of the above - when registering an error specify if it's wrapable - if it is, always marshal it as wrapped json, if it is not, keep current behavior.

magik6k · 2022-09-21T20:19:34Z

cmd/lotus/daemon.go

 	"github.com/filecoin-project/lotus/api"
+	lapi "github.com/filecoin-project/lotus/api"


duplicate import

magik6k · 2022-09-22T11:45:10Z

Tests seem to be broken because of this really robust error check: https://github.com/filecoin-project/lotus/blob/master/itests/kit/blockminer.go#L194 (which we can now make actually robust with this PR)

magik6k · 2022-09-23T07:42:47Z

lib/retry/retry_test.go

 func TestRetryErrorIsInTrue(t *testing.T) {
 	errorsToRetry := []error{&jsonrpc.RPCConnectionError{}}
-	require.True(t, ErrorIsIn(&jsonrpc.RPCConnectionError{}, errorsToRetry))
+	require.True(t, api.ErrorIsIn(&jsonrpc.RPCConnectionError{}, errorsToRetry))
 }

 func TestRetryErrorIsInFalse(t *testing.T) {
 	errorsToRetry := []error{&jsonrpc.RPCConnectionError{}}
-	require.False(t, ErrorIsIn(xerrors.Errorf("random error"), errorsToRetry))
+	require.False(t, api.ErrorIsIn(xerrors.Errorf("random error"), errorsToRetry))
 }

 func TestRetryWrappedErrorIsInTrue(t *testing.T) {
 	errorsToRetry := []error{&jsonrpc.RPCConnectionError{}}
-	require.True(t, ErrorIsIn(xerrors.Errorf("wrapped: %w", &jsonrpc.RPCConnectionError{}), errorsToRetry))
+	require.True(t, api.ErrorIsIn(xerrors.Errorf("wrapped: %w", &jsonrpc.RPCConnectionError{}), errorsToRetry))
 }


Technically those tests should also be moved to api/

magik6k · 2022-09-23T07:44:20Z

itests/kit/blockminer.go

@@ -191,7 +192,7 @@ func (bm *BlockMiner) MineBlocksMustPost(ctx context.Context, blocktime time.Dur
 			reportSuccessFn := func(success bool, epoch abi.ChainEpoch, err error) {
 				// if api shuts down before mining, we may get an error which we should probably just ignore
 				// (fixing it will require rewriting most of the mining loop)
-				if err != nil && !strings.Contains(err.Error(), "websocket connection closed") {
+				if err != nil && !strings.Contains(err.Error(), "websocket connection closed") && !api.ErrorIsIn(err, []error{&jsonrpc.RPCConnectionError{}}) {


supernit:

Suggested change

if err != nil && !strings.Contains(err.Error(), "websocket connection closed") && !api.ErrorIsIn(err, []error{&jsonrpc.RPCConnectionError{}}) {

if err != nil && !strings.Contains(err.Error(), "websocket connection closed") && !api.ErrorIsIn(err, []error{new(jsonrpc.RPCConnectionError)}) {

dirkmc · 2022-09-23T08:13:47Z

Another use case where this would be helpful:
https://github.com/filecoin-project/boost/blob/a58ea5a92a4237a018f9e7dc700936a5cebb0688/cmd/booster-bitswap/remoteblockstore/remoteblockstore.go#L97-L102

Backport: #9061 rpc errors

hunjixin · 2022-10-10T08:23:38Z

@magik6k sorry for these questions after the code merge

Recently venus intended to submit some of venus changes and bug fixes for rpc to the upstream, but when pushing it, it found many conflicts with TypeError

In PR, I think the current implementation of TypeError is a bit complicated.

I'm not sure if users really need the MarshalJson/UnMarshalJson functionality, but I'm looking at lotus/boost usage and it's enough to determine the type of error, and in venus requirements it's enough to determine the type of error.
in addition, I have the impression that adding meta fields may cause errors in the jsonrpc library in rust, which does not allow additional fields. meta is not a in standard jsonrpc protocol.

ErrorCode is a good idea, but maybe more better to define ErrorCode as true error type like below code

type ErrorCode int

func (e ErrorCode) Error() string {
    return fmt.Sprintf("error code %d", e)
}

for rpc server , user fmt.Error to return specific error

if res.MsgRct.ExitCode == exitcode.SysErrOutOfGas {
        return -1, rpc.ErrorCode(exitcode.SysErrOutOfGas)
}

or convert all exitcode to rpc error

if res.MsgRct.ExitCode != exitcode.Ok {
        return -1, xerrors.Errorf("message execution failed: exit %w, reason: %s", rpc.ErrorCode(res.MsgRct.ExitCode), res.Error)
}

for rpc client, use this code to check error type

    require.True(t, xerrors.Is(err, rpc.EOutOfGas))

If write it this way, not need to add the Register function in Rpc, and the Rpc implementation can be simpler without adding the meta field in Respnse.

arajasek requested a review from a team as a code owner July 19, 2022 17:33

arajasek changed the title ~~feat: support typed errors over RPC~~ feat: API: support typed errors over RPC Jul 19, 2022

arajasek force-pushed the asr/rpc-errors branch from 24ed596 to 30bd0ed Compare July 19, 2022 17:37

arajasek force-pushed the asr/rpc-errors branch from 30bd0ed to 0eee34b Compare July 20, 2022 17:15

arajasek mentioned this pull request Jul 20, 2022

feat: cli: Make compact partitions cmd better #9070

Merged

5 tasks

arajasek force-pushed the asr/rpc-errors branch from 0eee34b to b05f410 Compare July 20, 2022 22:57

arajasek force-pushed the asr/rpc-errors branch 2 times, most recently from 9f475ba to af56893 Compare September 20, 2022 20:27

arajasek changed the base branch from master to release/v1.18.0 September 21, 2022 15:17

arajasek mentioned this pull request Sep 21, 2022

fix: setup RPCConnectionError with pointer filecoin-project/go-jsonrpc#74

Merged

arajasek force-pushed the asr/rpc-errors branch 2 times, most recently from 10e6284 to f7ddb86 Compare September 21, 2022 18:03

magik6k reviewed Sep 21, 2022

View reviewed changes

arajasek added 3 commits September 22, 2022 11:44

feat: support typed errors over RPC

93f94d4

chore: deps: update to go-jsonrpc 0.1.8

6189932

remove duplicate import

5b96496

arajasek force-pushed the asr/rpc-errors branch from cb62e53 to 6c23b7b Compare September 22, 2022 16:50

arajasek mentioned this pull request Sep 22, 2022

fix: build: set PropagationDelaySecs correctly #9358

Merged

5 tasks

arajasek force-pushed the asr/rpc-errors branch 2 times, most recently from 5e91725 to 8022543 Compare September 22, 2022 17:14

fix: itest: check for closed connection

0626286

arajasek force-pushed the asr/rpc-errors branch from 8022543 to 0626286 Compare September 22, 2022 17:58

magik6k approved these changes Sep 23, 2022

View reviewed changes

arajasek added 2 commits September 23, 2022 11:37

chore: refactor: move retry test to API

3c195e4

address magik supernit

3e81c56

arajasek force-pushed the asr/rpc-errors branch from 7479cc8 to 3e81c56 Compare September 23, 2022 15:42

arajasek merged commit d89b196 into release/v1.18.0 Sep 23, 2022

arajasek deleted the asr/rpc-errors branch September 23, 2022 16:57

This was referenced Sep 26, 2022

[NV17]网络升级准备 filecoin-project/venus#5311

Closed

[venus] nv17 代码待合并 filecoin-project/venus#5287

Closed

magik6k mentioned this pull request Sep 27, 2022

Backport: #9061 rpc errors #9384

Merged

5 tasks

shrenujbansal added a commit that referenced this pull request Sep 27, 2022

Merge pull request #9384 from filecoin-project/backport/asr/rpc-errors

d11248b

Backport: #9061 rpc errors

Fatman13 mentioned this pull request Sep 30, 2022

[RPC] RPC支持加入错误类型error type？ filecoin-project/venus#5334

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: API: support typed errors over RPC #9061

feat: API: support typed errors over RPC #9061

arajasek commented Jul 19, 2022 •

edited

Loading

codecov bot commented Jul 19, 2022 •

edited

Loading

magik6k commented Aug 1, 2022

magik6k Sep 21, 2022

arajasek Sep 21, 2022

magik6k commented Sep 22, 2022

magik6k Sep 23, 2022

magik6k Sep 23, 2022

dirkmc commented Sep 23, 2022

hunjixin commented Oct 10, 2022 •

edited

Loading

		"github.com/filecoin-project/lotus/api"
		lapi "github.com/filecoin-project/lotus/api"

	if err != nil && !strings.Contains(err.Error(), "websocket connection closed") && !api.ErrorIsIn(err, []error{&jsonrpc.RPCConnectionError{}}) {
	if err != nil && !strings.Contains(err.Error(), "websocket connection closed") && !api.ErrorIsIn(err, []error{new(jsonrpc.RPCConnectionError)}) {

feat: API: support typed errors over RPC #9061

feat: API: support typed errors over RPC #9061

Conversation

arajasek commented Jul 19, 2022 • edited Loading

Related Issues

Proposed Changes

Additional Info

Checklist

codecov bot commented Jul 19, 2022 • edited Loading

Codecov Report

magik6k commented Aug 1, 2022

magik6k Sep 21, 2022

Choose a reason for hiding this comment

arajasek Sep 21, 2022

Choose a reason for hiding this comment

magik6k commented Sep 22, 2022

magik6k Sep 23, 2022

Choose a reason for hiding this comment

magik6k Sep 23, 2022

Choose a reason for hiding this comment

dirkmc commented Sep 23, 2022

hunjixin commented Oct 10, 2022 • edited Loading

arajasek commented Jul 19, 2022 •

edited

Loading

codecov bot commented Jul 19, 2022 •

edited

Loading

hunjixin commented Oct 10, 2022 •

edited

Loading