Cookie size for cut/copy clipboard too large

[petschki]

Since moving to Zope4/Python3 we're experiencing problems with cut/copy & paste actions because of the browsers cookie size restrictions to 4kB ... I made some investigations and want to share it here:

I've made new clean Plone 5.2.x/py3 installation, selected all items on the main level and you get this list of ids which OFS.CopySupport should compress for later saving in a cookie:

(1, [('', 'Plone2', 'front-page'), ('', 'Plone2', 'news'), ('', 'Plone2', 'events'), ('', 'Plone2', 'Members')])

this gets encoded to a json_bytes string with 112 Bytes:

b'[1, [["", "Plone2", "front-page"], ["", "Plone2", "news"], ["", "Plone2", "events"], ["", "Plone2", "Members"]]]'

and zlib compressed to 65 Bytes:

b'x^\x8b6\xd4Q\x88\x8eVR\xd2QP\n\xc8\xc9\xcfK5\x02\xb1\xd2\x8a\xf2\xf3Jt\x0b\x12\xd3S\x95b\x81\xd2h\xb2y\xa9\xe5\xc5\xd8\xc4S\xcbR\xf3J\xb0\xca\xf8\xa6\xe6&\xa5\x16\x01\xa5bc\x01\xb22\x1f\x12'

now it is quoted and returned ... for py3 additionally it gets decoded to a latin-1 text before quoting. After quoting its size is:

PY2: 153 Bytes
'x%5E%8B6%D4Q%88%8EVR%D2QP%0A%C8%C9%CFK5%02%B1%D2%8A%F2%F3Jt%0B%12%D3S%95b%81%D2h%B2y%A9%E5%C5%D8%C4S%CBR%F3J%B0%CA%F8%A6%E6%26%A5%16%01%A5bc%01%B22%1F%12'
PY3: 252 Bytes
'x%5E%C2%8B6%C3%94Q%C2%88%C2%8EVR%C3%92QP%0A%C3%88%C3%89%C3%8FK5%02%C2%B1%C3%92%C2%8A%C3%B2%C3%B3Jt%0B%12%C3%93S%C2%95b%C2%81%C3%92h%C2%B2y%C2%A9%C3%A5%C3%85%C3%98%C3%84S%C3%8BR%C3%B3J%C2%B0%C3%8A%C3%B8%C2%A6%C3%A6%26%C2%A5%16%01%C2%A5bc%01%C2%B22%1F%12'

so the saved value for the cookie is 2.25 times larger.

In a project we have a folder with 20 items which id's are about 100 characters long and the compressed/quoted cookie was too large to save in the browser. Debug logs for the clipboard:

2020-06-18 13:44:17,041 INFO    [OFS:671][waitress] json_bytes: 3467B
2020-06-18 13:44:17,041 INFO    [OFS:673][waitress] squashed_bytes: 947B
2020-06-18 13:44:17,041 INFO    [OFS:678][waitress] returned quoted cookie: 3801B

Has anybody an idea how to downsize the content of the cookie or even better: store the clipboard somewhere else than a cookie?

[d-maurer]

Peter Mathis wrote at 2020-6-18 04:47 -0700:

... and zlib compressed to 65 Bytes: ``` b'x^\x8b6\xd4Q\x88\x8eVR\xd2QP\n\xc8\xc9\xcfK5\x02\xb1\xd2\x8a\xf2\xf3Jt\x0b\x12\xd3S\x95b\x81\xd2h\xb2y\xa9\xe5\xc5\xd8\xc4S\xcbR\xf3J\xb0\xca\xf8\xa6\xe6&\xa5\x16\x01\xa5bc\x01\xb22\x1f\x12' ``` now it is quoted and returned ... for py3 additionally it gets decoded to a `latin-1` text before quoting.

"latin-1" encoding/decoding is the typical way to convert between a sequence of bytes and text representing the same sequence of bytes. It does not increase the length. Thus, you should get the same length for Python2 and for Python3. Quoting can be important. The cookie value is transported to the browser together with other information (parameters such as "domain", "path", "expires", ...). The value must therefore not contain syntactically relevant characters. The best way to achieve this is via quoting. Url-quoting is not the best way to quote binary data (such as compressed information). The value should instead be base64 encoded.