Unnecessary escaping in op JSON

[yoonk-msft]

Fluid op messages in websocket are bloated with unnecessary escaping in the JSON, like the following:

42/fluid,["submitOp","c20ad82d-ed12-4327-8032-35668f4b5a8b",[[{"clientSequenceNumber":10,"contents":"
{\"type\":\"chunkedOp\",\"contents\":{\"chunkId\":1,\"contents\":\"{\\\"address\\\":\\\"defaultComponent\\\",\\\"contents\\\":
{\\\"content\\\":{\\\"address\\\":\\\"23fd7090-3805-49a0-9438-c270de8835fd\\\",\\\"contents\\\":{\\\"type\\\":0,\\\"edit\\\":
{\\\"id\\\":\\\"4cdf2714-e632-4078-a2bf-84fbaf7aa0ae\\\",\\\"changes\\\":[{\\\"destination\\\":0,\\\"source\\\":[{\\\"traits\\\":
{\\\"3e029a25-a671-459d-a63b-61ff015f3a5a\\\":[{\\\"traits\\\":{\\\"ebbf1545-ba08-48e6-ae20-3962cb0537c3\\\":[{\\\"traits\\\":
{},\\\"definition\\\":\\\"208c7c2e-733d-4f47-af22-ced77889fc46\\\",\\\"identifier\\\":\\\"5dcdc4d0-ec00-4a7c-955c-
f5e79739dad4\\\"}],\\\"d905d3cd-3421-4460-a8a7-97928df1ac4e\\\":[{\\\"traits\\\":{\\\"04f7b6e9-ee61-4d84-b099-
0507a060cbcc\\\":[{\\\"traits\\\":{\\\"c80c5c72-a0a0-48ce-8e60-9f867c33c6f4\\\":[{\\\"traits\\\":{},\\\"payload\\\":
{\\\"base64\\\":\\\"AAAAAAAAP0A=\\\"},\\\"definition\\\":\\\"227b8b24-4c1e-40fe-bd65-
ad26fbbfab27\\\",\\\"identifier\\\":\\\"3fcc7f34-8a81-49a8-88b9-f4ed4b49603b\\\"}]
...

This extra level of string escaping makes up around 20% of our op message size. Could this be optimized?

[DLehenbauer]

• 'ContainerRuntime.submit()' stringifies the content to see if it exceeds 'maxOpSize' (escape level 1)
• 'DeltaManager.submit()' stringifies the content, and then wraps an IDocumentMessage. (escape level 2)
• 'DocumentDeltaConnection.submit()' passes an array of IDocumentMessages to Socket.IO, which implicitly stringifies (escape level 3)

[vladsud]

# 2 above can be easily moved into runtime (and thus merged with # 1). That said, I wonder why we do so at all (and not leave JSON here as is) - I think code long time ago was leaving JSON (I remember seeing a bunch of back-compat code dealing with JSON), but for some reason it was changed to be a string. Curious why.

That said, I think it's also worth re-examining

• why we use 16K limit (that hits # 1) - why not use much higher limit? I think 16K is default, it's controlled by ordering service.
• Why particular application / workflow has such large messages (to hit # 1)? While in some cases it's unavoidable (copy/paste of 1M of text), it should not be the norm. Note that there is hard limit of 1M due to Kafka limitations, so if app consistently hits large payloads, it will eventually hit 1Mb hard limit.

[DLehenbauer]

On the summarization side, @scarlettjlee noticed that ITreeEntry.contents is typed to 'string'.

[microsoft-github-policy-service]

This PR has been automatically marked as stale because it has had no activity for 60 days. It will be closed if no further activity occurs within 8 days of this comment. Thank you for your contributions to Fluid Framework!