Update to CCF Specification 1.0 #7

fxamacker · 2024-10-22T23:21:32Z

This PR updates CCF Specs to version 1.0 and matches the deployed version of CCF Codec.

Primarily, this PR adds new Cadence types and other updates to match changes in Cadence:

Update for rename restricted type to intersection type cadence#2581
Update for Add support for FunctionType.Purity in CCF codec cadence#3107
Update for Add support for Attachment and AttachmentType in CCF codec cadence#3131
Update for Add more support for entitlements (ReferenceType.Authorization) to CCF codec cadence#3139

Other changes in this PR:

Update copyright for Flow Foundation.
Added "Version: 1.0.0" and removed "Revision".
Removed "Status: RC3" and "Status of this Document".
Update "Scope" to highlight some scope-related items earlier:
- Some requirements are explicitly defined as optional.
- CCF-based formats or protocols MUST specify when optional requirements are used.
- "Deterministic CCF Encoding Requirements" is optional so individual formats and protocols MUST specify when to use it (as previously stated deeper in this document).
Update "Scope" to explicitly mention how to encode version numbers is outside the scope of this document. CCF-based formats and protocols are free to specify if version numbers are encoded and how to encode them.
Update "CCF Examples" section to make it easier to read and understand.
Mention some items that might not be known to all readers.
Improve grammar and fix typos.

Note

Scope of specs hasn't changed because the added text only highlights what is stated deeper inside the document.

Changes: - Update copyright for Flow Foundation. - Removed "Status: RC3" - Added "Version: 1.0.0" and removed "Revision". - Update "Scope" to highlight some scope-related items earlier: - Some requirements are explicitly defined as optional. - CCF-based formats or protocols MUST specify when optional requirements are used. - "Deterministic CCF Encoding Requirements" is optional so individual formats and protocols MUST specify when to use it (as previously stated deeper in this document).

This update corresponds to Cadence PR 3107: onflow/cadence#3107 This change does not affect CCF Codec because it was already implemented and deployed in the codec.

turbolent

Nice!

This update corresponds to Cadence PR 3131: onflow/cadence#3131 This change does not affect CCF Codec because it was already implemented and deployed in the codec.

This update corresponds to Cadence PR 2581: onflow/cadence#2581

This update corresponds to Cadence PR 2523 and subsequent changes to inclusiverange and inclusiverange-type: onflow/cadence#2523 This change does not affect CCF Codec because it was already implemented and deployed in CCF codec.

This update corresponds to Cadence PR 3139: onflow/cadence#3139 This change does not affect CCF Codec because it was already implemented and deployed in CCF codec.

This change does not affect CCF Codec because it was already implemented and deployed in CCF codec.

This change does not affect CCF Codec because it was already removed (deprecated) and deployed in CCF codec.

turbolent

Great work! Thank you for bringing this up-to-date 👏

Minor updates to text that do not affect CCF codecs.

Removed list of CBOR codec implementations. Inspired by seeing a draft RFC that includes a list of implementations with a note to remove that list before publication (it was useful to keep during draft status).

It is outside the scope of CCF Specification to specify encoding of version numbers for CCF itself, CCF-based formats, or CCF-based protocols. For example, CCF-based formats and protocols are free to specify an encoding that uses SemVer, sequence-based versioning, other versioning, or no versioning. Each CCF message is a CBOR data item. Given this, some CCF-based formats or protocols may want to specify using CBOR Sequences (RFC 8742) to provide a version number in the first CBOR data item, followed by CBOR data item(s) encoded in CCF.

Update "Why CBOR" to be clearer and flow better. Mention "CBOR is a binary data format..." in case reader never heard of it. Replace "IETF RFC" with just "RFC" to be more concise. Add this to "Interoperability and Reuse of CBOR Codecs" because this important aspect is often overlooked: When evaluating and comparing codecs, benchmarks and tests should include decoding malicious data. These changes do not affect CCF codecs.

Update text for readability (grammar) and fix some typos. The only other change was updating a CDDL comment to replace the word "info" with "information". This change does not affect CCF codecs.

This commit makes it harder to miss the difference between SHOULD and MUST related to deterministic encoding. Added: "However, some CCF-based protocols may not require deterministic CCF encodings." Moved this to the start of new paragraph to make it harder to miss: "CCF-based protocols MUST specify when encoders are required to emit deterministic CCF encodings. For example:"

The word "must" was replaced with "MUST" in a sentence to make it more clear it is an absolute requirement.

fxamacker · 2025-03-21T16:47:48Z

Out of the last 10 commits, 9 were done in the past couple of days.

However, github.com is incorrectly showing 5 of those were done months ago. Fortunately, clicking on the commit shows the correct date.

These edits to the specs were done from a web browser (and I reboot linux regularly), so maybe this is a github.com website bug that is triggered depending on navigation used to reach the edit button for the updated document.

While at it, also update "Scope" section to improve readability.

Changes were done at github.com/fxamacker/ccf_draft to reduce noise here during the weekend. For details (individual commits), see: - fxamacker/ccf_draft#103 - fxamacker/ccf_draft#104 NOTE: There were no changes to CDDL.

fxamacker · 2025-03-31T03:38:05Z

Commit 1d5dd35 here includes all changes from March 28-30, 2025 at https://github.com/fxamacker/ccf_draft.

Changes were done at github.com/fxamacker/ccf_draft to reduce noise here during the weekend.

For details (individual commits), see:

NOTE: There were no changes to CDDL.

This commit clarifies text in all the examples. Also fixed some formatting for consistency.

fxamacker · 2025-03-31T18:06:59Z

Commit cf54d91 matches the latest updates from https://github.com/fxamacker/ccf_draft/blob/main/ccf_specs.md.

NOTE: There were no changes to CDDL.

fxamacker · 2025-03-31T18:12:20Z

ccf_specs.md


 - `composite-type.id` MUST be unique in `ccf-typedef-message` or `ccf-typedef-and-value-message`.

 - `composite-type.cadence-type-id` MUST be unique in `ccf-typedef-message` or `ccf-typedef-and-value-message`.

- `field-name` MUST be unique in `composite-type`.
+- `field-name` MUST be unique in `composite-type.fields`.


This is more explicit and doesn't affect existing CCF codec implementation.

fxamacker · 2025-03-31T18:19:02Z

ccf_specs.md

+
+It is outside the scope of this document to specify individual CCF-based protocols (e.g., events). For example, CCF-based protocols MUST specify when encoders are required to emit CCF encodings that satisfy "Deterministic CCF Encoding Requirements."
+
+This document does not specify how to encode version numbers of CCF itself or CCF-based protocols. CCF-based protocols can specify an encoding that uses CalVer, SemVer, sequence-based versioning, any other versioning, or no versioning. Some CCF-based protocols may want to use CBOR Sequences ([RFC 8742](https://www.rfc-editor.org/rfc/rfc8742.html)) to provide a version number in the first CBOR data item, followed by CBOR data item(s) representing CCF message(s).


Versioning is explicitly mentioned because people sometimes have strong preferences about versioning. Also, different CCF-based protocols can have very different priorities when choosing how to encode metadata like version numbers.

fxamacker · 2025-03-31T18:21:20Z

ccf_specs.md


 ## Abstract

-Cadence Compact Format (CCF) is a data format designed for compact, efficient, and deterministic encoding of [Cadence](https://github.com/onflow/cadence) external values. Cadence is a modern resource-oriented programming language used by [Flow](https://github.com/onflow/flow-go) blockchain.
+Cadence Compact Format (CCF) is a binary data format designed for compact, efficient, and deterministic encoding of [Cadence](https://github.com/onflow/cadence) external values. Cadence is a modern resource-oriented programming language used by [Flow](https://github.com/onflow/flow-go) blockchain.


Added "binary" because some readers might not know CCF is a binary data format.

fxamacker · 2025-03-31T18:25:18Z

ccf_specs.md


-  - More compact encoding. Cadence type info is not repeatedly encoded unnecessarily in a message. E.g. for a homogeneous array of a Cadence composite type, each element will not have its Cadence composite type info encoded repeatedly.
+- Cadence external values: CCF supports all Cadence built-in types and user-defined types (e.g., composite types). For extensibility, CCF reserves multiple ranges of CBOR tag numbers (unassigned by IANA) for future Cadence built-in data types.


This replaces two bullet points and clarifies extensibility (we use CBOR tags and reserve ranges of CBOR tag numbers for future Cadence built-in types).

fxamacker · 2025-03-31T18:33:44Z

ccf_specs.md


-CBOR codecs are available in various programming languages. Examples include:
+When evaluating or comparing codecs, benchmarks should include decoding malicious data.


Added this because it is important and often overlooked by teams evaluating codec implementations or data format designs.

fxamacker · 2025-03-31T18:37:23Z

ccf_specs.md

-  - `fxamacker/cbor` was designed with security in mind and passed multiple security assessments in 2022. A [nonconfidential security assessment](https://github.com/veraison/go-cose/blob/v1.0.0-rc.1/reports/NCC_Microsoft-go-cose-Report_2022-05-26_v1.0.pdf) produced by NCC Group for Microsoft Corporation includes parts of fxamacker/cbor.
-  - `fxamacker/cbor` is used in projects by Arm Ltd., Cisco, Dapper Labs, EdgeX&nbsp;Foundry, Fraunhofer&#8209;AISEC, Linux&nbsp;Foundation, Microsoft, Mozilla, Tailscale, Teleport, [and&nbsp;others](https://github.com/fxamacker/cbor#who-uses-fxamackercbor). Notably, it was already [used by Cadence](https://github.com/onflow/cadence/blob/master/runtime/interpreter/encode.go) for internal value encoding.
-  - `fxamacker/cbor` is maintained by the author of this document.
+Projects implementing a CCF codec should evaluate more than one CBOR codec for standards compliance, security, and other factors.


Removed list of specific CBOR codec implementations in various languages because it doesn't really belong in a data format specification (except maybe during early drafts).

turbolent · 2025-04-01T15:45:13Z

👌

fxamacker added the documentation Improvements or additions to documentation label Oct 22, 2024

fxamacker self-assigned this Oct 22, 2024

fxamacker requested review from ramtinms and turbolent as code owners October 22, 2024 23:21

Add function-value.purity in CDDL

6b39848

This update corresponds to Cadence PR 3107: onflow/cadence#3107 This change does not affect CCF Codec because it was already implemented and deployed in the codec.

turbolent approved these changes Oct 22, 2024

View reviewed changes

fxamacker added 2 commits October 22, 2024 18:50

Add attachment and attachment-type in CDDL

f1702cd

This update corresponds to Cadence PR 3131: onflow/cadence#3131 This change does not affect CCF Codec because it was already implemented and deployed in the codec.

Rename restricted-type to intersection-type in CDDL

f93d24f

This update corresponds to Cadence PR 2581: onflow/cadence#2581

fxamacker changed the title ~~Update copyright and scope in ccf_specs.md~~ Update to CCF Specification 1.0 Oct 23, 2024

fxamacker mentioned this pull request Oct 23, 2024

Update CCF CDDL for IntersectionType to match CCF codec onflow/cadence#3640

Merged

6 tasks

fxamacker added 5 commits October 23, 2024 11:20

Add inclusiverange and inclusiverange-type in CDDL

d72b68c

This update corresponds to Cadence PR 2523 and subsequent changes to inclusiverange and inclusiverange-type: onflow/cadence#2523 This change does not affect CCF Codec because it was already implemented and deployed in CCF codec.

Add entitlement types and type values in CDDL

305c1ba

This update corresponds to Cadence PR 3139: onflow/cadence#3139 This change does not affect CCF Codec because it was already implemented and deployed in CCF codec.

Add workflow_dispatch to cddl.yml

1517dd7

Fix CDDL syntax broken in prior commit

da5ac16

Update simple-type-id enum in CDDL

c2cb8ae

This change does not affect CCF Codec because it was already implemented and deployed in CCF codec.

fxamacker mentioned this pull request Oct 24, 2024

Fix and improve CCF CDDL in comments onflow/cadence#3643

Merged

6 tasks

fxamacker added 2 commits October 23, 2024 20:25

Remove path-capability-value from CDDL

5e91602

This change does not affect CCF Codec because it was already removed (deprecated) and deployed in CCF codec.

Mention attachment uses composite-value in CDDL

22fa0dd

This was referenced Oct 24, 2024

Unplanned work - Q4 2024 onflow/flow-go#6563

Closed

Update CCF specs to 1.0.0 by adding support for new types added to Cadence #6

Closed

turbolent approved these changes Nov 15, 2024

View reviewed changes

turbolent requested review from SupunS and removed request for ramtinms November 15, 2024 20:05

fxamacker mentioned this pull request Nov 26, 2024

CCF specification 1.0 release onflow/flow-go#6764

Closed

4 tasks

fxamacker added 5 commits January 26, 2025 20:35

Update ccf_specs.md

e37a928

Minor updates to text that do not affect CCF codecs.

Reword positive integer as unsigned integer

305d379

Remove list of CBOR codecs

1e93503

Removed list of CBOR codec implementations. Inspired by seeing a draft RFC that includes a list of implementations with a note to remove that list before publication (it was useful to keep during draft status).

fxamacker added 5 commits March 20, 2025 15:43

Remove a sentence in "Why CBOR"

7b13fb5

Improve readability of text and fix typos

9f20441

Update text for readability (grammar) and fix some typos. The only other change was updating a CDDL comment to replace the word "info" with "information". This change does not affect CCF codecs.

Update some text for consistency

bf3d570

Use key word "MUST" in a sentence for absolute requirement

45d061b

The word "must" was replaced with "MUST" in a sentence to make it more clear it is an absolute requirement.

fxamacker added 2 commits March 27, 2025 09:01

Update document date

6703acb

While at it, also update "Scope" section to improve readability.

Update with changes from fxamacker/ccf_draft PR 103 and 104

1d5dd35

Changes were done at github.com/fxamacker/ccf_draft to reduce noise here during the weekend. For details (individual commits), see: - fxamacker/ccf_draft#103 - fxamacker/ccf_draft#104 NOTE: There were no changes to CDDL.

Clarify all examples in "CCF Examples"

cf54d91

This commit clarifies text in all the examples. Also fixed some formatting for consistency.

fxamacker commented Mar 31, 2025

View reviewed changes

fxamacker merged commit 2217b44 into main Mar 31, 2025
1 check passed

fxamacker mentioned this pull request Apr 1, 2025

🎯 Cadence VM (execution environment) goals - Q1 2025 onflow/flow-go#6881

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to CCF Specification 1.0 #7

Update to CCF Specification 1.0 #7

fxamacker commented Oct 22, 2024 •

edited

Loading

turbolent left a comment

turbolent left a comment

fxamacker commented Mar 21, 2025 •

edited

Loading

fxamacker commented Mar 31, 2025 •

edited

Loading

fxamacker commented Mar 31, 2025

fxamacker Mar 31, 2025 •

edited

Loading

fxamacker Mar 31, 2025 •

edited

Loading

fxamacker Mar 31, 2025

fxamacker Mar 31, 2025

fxamacker Mar 31, 2025

fxamacker Mar 31, 2025

turbolent commented Apr 1, 2025


		It is outside the scope of this document to specify individual CCF-based protocols (e.g., events). For example, CCF-based protocols MUST specify when encoders are required to emit CCF encodings that satisfy "Deterministic CCF Encoding Requirements."

		This document does not specify how to encode version numbers of CCF itself or CCF-based protocols. CCF-based protocols can specify an encoding that uses CalVer, SemVer, sequence-based versioning, any other versioning, or no versioning. Some CCF-based protocols may want to use CBOR Sequences ([RFC 8742](https://www.rfc-editor.org/rfc/rfc8742.html)) to provide a version number in the first CBOR data item, followed by CBOR data item(s) representing CCF message(s).


		- More compact encoding. Cadence type info is not repeatedly encoded unnecessarily in a message. E.g. for a homogeneous array of a Cadence composite type, each element will not have its Cadence composite type info encoded repeatedly.
		- Cadence external values: CCF supports all Cadence built-in types and user-defined types (e.g., composite types). For extensibility, CCF reserves multiple ranges of CBOR tag numbers (unassigned by IANA) for future Cadence built-in data types.


		CBOR codecs are available in various programming languages. Examples include:
		When evaluating or comparing codecs, benchmarks should include decoding malicious data.

Update to CCF Specification 1.0 #7

Update to CCF Specification 1.0 #7

Conversation

fxamacker commented Oct 22, 2024 • edited Loading

turbolent left a comment

Choose a reason for hiding this comment

turbolent left a comment

Choose a reason for hiding this comment

fxamacker commented Mar 21, 2025 • edited Loading

fxamacker commented Mar 31, 2025 • edited Loading

fxamacker commented Mar 31, 2025

fxamacker Mar 31, 2025 • edited Loading

Choose a reason for hiding this comment

fxamacker Mar 31, 2025 • edited Loading

Choose a reason for hiding this comment

fxamacker Mar 31, 2025

Choose a reason for hiding this comment

fxamacker Mar 31, 2025

Choose a reason for hiding this comment

fxamacker Mar 31, 2025

Choose a reason for hiding this comment

fxamacker Mar 31, 2025

Choose a reason for hiding this comment

turbolent commented Apr 1, 2025

fxamacker commented Oct 22, 2024 •

edited

Loading

fxamacker commented Mar 21, 2025 •

edited

Loading

fxamacker commented Mar 31, 2025 •

edited

Loading

fxamacker Mar 31, 2025 •

edited

Loading

fxamacker Mar 31, 2025 •

edited

Loading