[rpc] --rpc-max-response-size doesn't seem work correctly

[ianhe8x]

Is there an existing issue?

• I have searched the existing issues

Experiencing problems? Have you tried our Stack Exchange first?

• This is not a support question.

Description of bug

According to doc, the default value of --rpc-max-response-size is 15 MB, which I have found in the source code as well.

When I make a traceblock rpc to my node, I got exceeds max payload size error which no matter how I adjust this argument, I can not get pass it.
I'm pretty sure something wrong with the implementation. The proof is (against a polkadot node)

curl -s -X POST -H 'Content-type: application/json' --data '{"jsonrpc":"2.0","method":"state_traceBlock",
    "params":["0x6308efcc9c004279bec98653e36c45cfbac73a85bd0ccb7e84efdfda44a65e9c", "state", "", "Put"]
    ,"id":1}' 'https://my-polkadot-node' > res.log
curl -s -X POST -H 'Content-type: application/json' --data '{"jsonrpc":"2.0","method":"state_traceBlock",
    "params":["0x6308efcc9c004279bec98653e36c45cfbac73a85bd0ccb7e84efdfda44a65e9c", "state", "", "Append"]
    ,"id":1}' 'https://my-polkadot-node' > res2.log
curl -s -X POST -H 'Content-type: application/json' --data '{"jsonrpc":"2.0","method":"state_traceBlock",
    "params":["0x6308efcc9c004279bec98653e36c45cfbac73a85bd0ccb7e84efdfda44a65e9c", "state", "", "Put,Append"]
    ,"id":1}' 'https://my-polkadot-node' > res3.log

res.log: 1.1M
res2.log: 321K
res3.log: Payload likely exceeds max payload size of RPC server.

Doesn't make sense if default limit is 15MB.

Steps to reproduce

No response

[bkchr]

Cc @niklasad1

[niklasad1]

Hey @ianhe8x

Can you provide the error message that you got on the too big response?
I suspect it could be the traceBlock rpc implementation that has its own limit which triggers that, https://github.com/paritytech/substrate/blob/master/client/service/src/builder.rs#L678

Can you try with --rpc-max-payload instead?

We could remove that limit completely now because JSON-RPC server doesn't kill the connection with a too big response if my hunch is correct :P

[ianhe8x]

The error msg is {"jsonrpc":"2.0","result":{"traceError":{"error":"Payload likely exceeds max payload size of RPC server."}},"id":1}

---

Overall --rpc-max-payload does solve the problem. but I'm wondering what is the unit of --rpc-max-payload, is it MB?

I got same error with --rpc-max-payload=18, but passed with --rpc-max-payload=20 for "Put,Append" test.
The response is just 1.4MB, doesn't explain why it failed with the first config.

[niklasad1]

Overall --rpc-max-payload does solve the problem. but I'm wondering what is the unit of --rpc-max-payload, is it MB?

Indeed, it's MB but it uses "an estimation for the size" which is BASE_PAYLOAD + events.len() * AVG_EVENT + spans.len() * AVG_SPAN which is too strict apparently.

However, I have removed it now as it's not needed anymore.
This was used before on the old RPC server because a too big response would kill the connection.