diff --git a/README.md b/README.md index 6f5ac657..14344a06 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,15 @@ IPLD Specifications =================== -IPLD is not a single specification it is a set of specifications. - -The goal of IPLD is to enable decentralized data-structures -which in turn will enable more decentralized applications. +The goal of IPLD is to enable decentralized data-structures that are universally addressable and linkable which in turn will enable more decentralized applications. +These data-structures allow us to do for data what URLs and links did for HTML web pages. +IPLD is not a single specification it is a set of specifications. Many of the specifications in IPLD are inter-dependent. -# IPLD Layer Model +## IPLD Layer Model -This layer model is a simplified heirarchy of IPLD concepts and requirements. +This layer model is a simplified hierarchy of IPLD concepts and requirements. ``` ┌──────────────────────────────────────────────────┐ @@ -31,226 +30,123 @@ This layer model is a simplified heirarchy of IPLD concepts and requirements. └──────────────────────────────────────────────────┘ ``` -## Block Layer (Layer 0) - -The block layer encompasses all content addressed block formats and specifies how blocks -are addressed,how they self-describe their codec for encoding/decoding, and how blocks link -between each other. - -This layer alone is enough to accomplish basic graph replication across a variety of formats (eth, bitcoin, git, -dag-pb, dag-cbor, etc). - -This layer does not define data structures or types, although many codecs may convert these formats into native types, -there are no type requirements or assurances about types at the block layer. - -## Data Model Layer (Layer 1) - -The Data Model Layer describes a set of base required types to be implemented by a subset of IPLD codecs. - -With these basic types authors can create various single-block data structures which can be read with predictable -paths and selectors. - -With just the data model layer, several data structures can be authored and put into a single block. These data -structures can also link to one another, but a *single* collection (Map or List) cannot be spread across many -blocks with only the Data Model. - -Since different systems and transports may impose block size limits (often 2mb or more) in order to control memory usage, -larger collections need to be sharded over many blocks at the Schema Layer. +### Block Layer (Layer 0) -## Schema Layer (Layer 2) +The block layer encompasses all content addressed block formats and specifies how blocks are addressed, how they self-describe their codec for encoding/decoding, and how blocks link between each other. -IPLD Schemas define a mapping from the Data Model Layer (Layer 1) to -instantiated data structures comprising complex layouts. The Schema Layer -adds the ability to extend the IPLD Data Model to the wide variety of types -required for typical programatic interaction with a data source without the -need to implement custom translation abstractions. +This layer alone is enough to accomplish basic graph replication across a variety of formats (eth, bitcoin, git, dag-pb, dag-cbor, etc). -The Schema Layer will also serve as an enabling layer for complex multi-block -data structures by providing stability and consistency of data model use within -individual blocks and defined interaction points for the logic required for -building and interacting with advanced data layouts, such as multi-block Maps, -Lists and Sets. - -# Specification Repo Layout - -* [/IPLD-Data-Model-v1.md](/IPLD-Data-Model-v1.md) -* [/IPLD-Path.md](/IPLD-Path.md) -* [/CID.md](/CID.md) -* [/Codecs](/Codecs) - * [/Codecs/DAG-JSON.md](/Codecs/DAG-JSON.md) - * [/Codecs/DAG-CBOR.md](/Codecs/DAG-CBOR.md) -* [/data-structures/multiblock-collections.md](/data-structures/multiblock-collections.md) -* [/graphsync/graphsync.md](/graphsync/graphsync.md) -* [/selectors/selectors.md](/selectors/selectors.md) - -## Discussion - -Discussion of specific specifications happens in [this repository's issues](https://github.com/ipld/specs/issues). - -Discussion of IPLD more generally happens in the [IPLD repository](https://github.com/ipld/ipld/issues). - -## Contribute - -Suggestions, contributions, criticisms are welcome. Though please make sure to familiarize yourself deeply with IPLD and IPFS, the models it adopts, and the principles it follows. [Join in in the issues](https://github.com/ipld/specs/issues). - -Please be aware that specs are really hard to design by committee. Treat this space like you would the workshop of an artist. Please suggest improvements, but please don't be disappointed if we say no to something. What we leave out is often more important than what we add in. - -Check out our [contributing document](https://github.com/ipld/ipld/blob/master/contributing.md) for more information on how we work, and about contributing in general. Please be aware that all interactions related to IPLD are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md). - -Small note: If editing the README, please conform to the [standard-readme](https://github.com/RichardLitt/standard-readme) specification. - -## License +This layer does not define data structures or types, although many codecs may convert these formats into native types, there are no type requirements or assurances about types at the block layer. -This repository is only for documents. All of these are licensed under the [CC-BY-SA 3.0](https://ipfs.io/ipfs/QmVreNvKsQmQZ83T86cWSjPu2vR3yZHGPm5jnxFuunEB9u) license, © 2016 Protocol Labs Inc. +**Documents:** -# Terminology +| | | +|-----|------| +| [Concept: Block](block-layer/block.md) | [block-layer/block.md](block-layer/block.md) | +| [Concept: Content Addressability](block-layer/content-addressability.md) | [block-layer/content-addressability.md](block-layer/content-addressability.md) | +| [Concept: Multihash](block-layer/multihash.md) | [block-layer/multihash.md](block-layer/multihash.md) | +| [Concept: Serialization and Formats](block-layer/serialization-and-formats.md) | [block-layer/serialization-and-formats.md](block-layer/serialization-and-formats.md) | +| [Specification: CIDs](block-layer/CID.md) | [block-layer/CID.md](block-layer/CID.md) | +| [Specification: DAG-CBOR](block-layer/codecs/DAG-CBOR.md) | [block-layer/codecs/DAG-CBOR.md](block-layer/codecs/DAG-CBOR.md) | +| [Specification: DAG-JSON](block-layer/codecs/DAG-JSON.md) | [block-layer/codecs/DAG-JSON.md](block-layer/codecs/DAG-JSON.md) | +| [Specification: IPLD Selectors](block-layer/selectors/selectors.md) | [block-layer/selectors/selectors.md](block-layer/selectors/selectors.md) | +| [Specification: Graphsync](block-layer/graphsync/graphsync.md) | [block-layer/graphsync/graphsync.md](block-layer/graphsync/graphsync.md) | -## Description of IPLD +### Data Model Layer (Layer 1) -IPLD is a set of standards and implementations for creating decentralized data-structures that are -universally addressable and linkable. These structures allow us to do for data what URLs and -links did for HTML web pages. +The Data Model Layer describes a set of base required types to be implemented by a subset of IPLD codecs. -## Generic Terms +With these basic types authors can create various single-block data structures which can be read with predictable paths and selectors. -### Content Addressability +With just the data model layer, several data structures can be authored and put into a single block. +These data structures can also link to one another, but a *single* collection (Map or List) cannot be spread across many blocks with only the Data Model. -"Content addressability" refers to the ability to refer to content by a trustless identifier. +Since different systems and transports may impose block size limits (often 2mb or more) in order to control memory usage, larger collections need to be sharded over many blocks at the Schema Layer. -Rather than referring to content by a string identifier or URL, content addressable systems refer to content -by a cryptographic hash. This allows complete decentralization of the content as the identifier -does not specify the retrieval method and provides a secure way to verify the content. +**Documents:** -## IPLD Terms +| | | +|-----|------| +| [Specification: IPLD Data Model](data-model-layer/paths.md) | [data-model-layer/paths.md](data-model-layer/paths.md) | +| [Specification: IPLD Paths](data-model-layer/data-model.md) | [data-model-layer/data-model.md](data-model-layer/data-model.md) -### Multihash +### Schema Layer (Layer 2) -Multihash is hash format that is not specific to a single hashing algorithm. +IPLD Schemas define a mapping from the Data Model Layer (Layer 1) to instantiated data structures comprising complex layouts. +The Schema Layer adds the ability to extend the IPLD Data Model to the wide variety of types required for typical programatic interaction with a data source without the need to implement custom translation abstractions. -A multihash describes the algorithm used for the hash as well as the hash value. +The Schema Layer will also serve as an enabling layer for complex multi-block data structures by providing stability and consistency of data model use within individual blocks and defined interaction points for the logic required for building and interacting with advanced data layouts, such as multi-block Maps, Lists and Sets. -``` -+-----------+----------------------------+ -| Hash Type | Hash Value | -+-----------+----------------------------+ -``` +**Documents:** -SHA-256 example. +| | | +|-----|------| +| [Specification: IPLD Schemas](schema-layer/schemas/schemas.md) | [schema-layer/schemas/schemas.md](schema-layer/schemas/schemas.md) | +| [Concept: IPLD Multi-block Collections](schema-layer/data-structures/multiblock-collections.md) | [schema-layer/data-structures/multiblock-collections.md](schema-layer/data-structures/multiblock-collections.md) | -``` -+---------+------------------------------------------------------------------+ -| SHA-256 | 2413fb3709b05939f04cf2e92f7d0897fc2596f9ad0b8a9ea855c7bfebaae892 | -+---------+------------------------------------------------------------------+ -``` +## Specification document status -Note: these examples are simplifications of the concepts. For a complete description visit the [project and its specs](https://github.com/multiformats/multihash). +Specification documents in this repository fit into one of two categories and have one of three possible statuses: -### CID (Content Identifier) +- Prescriptive + - Exploratory + - Draft + - Final +- Descriptive + - Draft + - Final -Hash based content identifier. Includes the `codec` and `multihash`. +**Prescriptive** specifications are intended to describe future implementations or, in some cases, changes to existing implementations. -CID's +**Descriptive** specifications describe existing behavior. +In many cases these specifications are not intended to drive new implementations and are only written to understand existing behaviors. -``` -+-------+------------------------------+ -| Codec | Multihash | -+-------+------------------------------+ -``` +Documents labelled "Specification" in this repository will also be labelled with a descriptor that indicates the category and status. +e.g. _"Status: Prescriptive - Draft"_ or _"Status: Descriptive - Final"_. -The long version -``` -+------------------------------+ -|Codec | -+------------------------------+ -|Multihash | -| +----------+---------------+ | -| |Hash Type | Hash Value | | -| +----------+---------------+ | -| | -+------------------------------+ -``` +## Contributing & Discussion -Note: these examples are simplifications of the concepts. For a complete description visit the [spec](./CID.md). +Suggestions, contributions, criticisms are welcome. -### Block +Discussion of specifications happens in [this repository's issues](https://github.com/ipld/specs/issues) or via pull request. Discussion of IPLD more generally happens in the [IPLD repository](https://github.com/ipld/ipld/issues). -A CID and the binary data value for that CID. +Check out our [contributing document](https://github.com/ipld/ipld/blob/master/contributing.md) for more information on how we work, and about contributing in general. +Please be aware that all interactions related to IPLD are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md). -The short version. -``` -+-----+--------------------------------+ -| CID | Data | -+-----+--------------------------------+ -``` +## Governance -The long version. -``` -+-----------------------------------+------------------+ -| CID | Binary Data | -| +------------------------------+ | | -| |Codec | | | -| +------------------------------+ | | -| |Multihash | | | -| | +----------+---------------+ | | | -| | |Hash Type | Hash Value | | | | -| | +----------+---------------+ | | | -| | | | | -| +------------------------------+ | | -| | | -+-----------------------------------+------------------+ -``` - -### IPLD Path - -A string identifier used for deep references into IPLD -graphs. Follows similar escape and segmentation rules as URI Paths. +All changes to documents must take place via pull request. -[Read the full specification for more details.](./IPLD-Path.md) +Pull requests are governed by different rules depending on the document type and status of that document: -### IPLD Data Model +Specifications: -The IPLD Data Model describes a set of base types. Codecs that support these base types -can be used by any of the data-structures built on top of the IPLD Data Model. + * Exploratory Stage + * Authors can merge changes into exploratory specifications at their own discretion + * Draft Stage + * Authors must attempt to reach a consensus between all active participants before a merge + * If no objections are raised in a 48 hours period changes can be merged + * If objections cannot be resolved the change can be voted on by the IPLD Team + * Final Stage + * Improvements that have a consensus can be merged + * Changes to behavior should not be merged unless absolutely necessary and agreed upon by a vote of the IPLD Team -Codecs that support the IPLD Data Model: +Concepts and other documents (including README.md): -* [DAG-CBOR](/Codecs/DAG-CBOR.md) -* WIP: [DAG-JSON](/Codecs/DAG-JSON.md) + * Authors must attempt to reach a consensus between all active participants before a merge + * If no objections are raised in a 48 hours period changes can be merged + * If objections cannot be resolved the change can be voted on by the IPLD Team -### Codec +## IPLD Team -A codec exposes serialization and deserialization for IPLD blocks. If it also supports -content addressable links then the codec exposes those links as `CID`'s. A codec -also supports atomic IPLD Path lookups on the block. +The IPLD Team consists of currently active IPLD developers. -#### Serializer, Deserializer and Format +* @mikeal +* @vmx +* @warpfork +* @rvagg -A logical separation exists in any given IPLD codec between the **format** and the **serializer/deserializer**. - -``` -+--------------------+ +--------------------+ -| | | | -| Serializer | | Deserializer | -| | | | -+---------+----------+ +---------+----------+ - | ^ - | Sent to another peer | - v | -+---------+----------+ +----------+---------+ -| | | | -| Format +-------------> Format | -| | | | -+--------------------+ +--------------------+ -``` - -A **format** may represent object types and tree structures any -way it wishes. This includes existing representations (JSON, BSON, CBOR, -Protobuf, msgpack, etc) or even new custom serializations. - -Therefor, a **format** is the standardized representation of IPLD Links and Paths. It describes how to translate between structured data and binary. - -It is worth noting that **serializers** and **deserializers** differ by programming language while the **format** does not and MUST remain consistent across all codec implementations. - -#### Representation +## License -The in-memory representation of a de-serialized IPLD value. +This repository is only for documents. +All of these are licensed under the [CC-BY-SA 3.0](https://ipfs.io/ipfs/QmVreNvKsQmQZ83T86cWSjPu2vR3yZHGPm5jnxFuunEB9u) license, © 2016 Protocol Labs Inc. diff --git a/REQUIREMENTS.md b/REQUIREMENTS.md deleted file mode 100644 index ff187e72..00000000 --- a/REQUIREMENTS.md +++ /dev/null @@ -1,160 +0,0 @@ -# IPLD Requirements - -This document outlines parts of IPLD that should and should not be changed to -ensure the success of future improvements (especially type systems). - -**Definitions** - -* Block: A block is a chunk of an IPLD DAG, encoded in a format. Blocks have CIDs. -* Fragment: A piece of an IPLD DAG. Blocks contain fragments. -* Node: A node is a *point* in an IPLD DAG (object, array, number, etc.). -* Link: A link is an IPLD Node that points to another IPLD Node. -* Path: A paths a human readable pointer to an IPLD Node. - -## Linked - -IPLD must support linking to any IPLD node (even if the node is in the middle of -a block). That is, IPLD must support arbitrary IPLD paths in links. - -**Motivation:** Considering this in the context of programming languages, not -being able to *store* a pointer to a struct *inside* of another struct would be -severely limiting. - -NOTE: We don't currently support arbitrary paths but, in the context of -programming, we really need to. - -## Immutable - -IPLD links must be immutable. We'll likely define a mutable link spec on top of -IPLD but there needs to be an immutable layer at the bottom. - -**Motivation:** *Having* an immutable layer is important for a lot of analysis, -memoization, type checking, etc. - -## Multicodecs Are Not Types - -It's impossible to understand IPLD data at a *structural* level if we don't know -the format. Therefore, we should avoid introducing new formats unnecessarily as -*every* IPLD implementation needs to support these new formats. - -## No Non-Local Reasoning - -An IPLD block should never be interpreted in the context of *anything* not -contained in the block (and CID). - -For example, assuming we add support for relative links, the following -definition of `foo` would not be a valid IPLD block: - -``` -var foo = { - // points outside of the current block, into the parent's "baz" field. - "baz": {"/": "../../baz"} -} -var bar = { - "foo": CidOf(foo), - // `/foo/baz` points here. - "baz": "something" -} - -// resolution throug block `foo` depends on block `bar`. -Resolve("/ipld/${CidOf(bar)}/foo/baz/") -``` - -For the same reason, IPLD links can't rely on an authority (e.g., a blockchain). - -Note: Links like this can still be encoded at the application level but they -won't be handled by the IPLD resolver (and won't get the special "link" type). - -**Motivation:** IPLD needs to be easy to reason about. - -**Caveat:** - -We *may* want to relax this if we want to move schemas into separate, -deduplicated blocks (referenced by CID). If we do that, we'd need to fetch a -block's schema before being able to interpret the it. - -However, we need to *thoroughly* discuss any changes to this requirement. - -1. The space savings may not be worth it given the size of CIDs (>40 bytes), - compression, smart transports, and smart datastores. -2. This change would introduce some weird interface complexities and potential - network dependencies. - -## No Cycles - -IPLD links must not be cyclic, even if we add support for relative links. - -**Motivation:** - -1. Security: Forbidding link cycles ensures that any graph traversal terminates. - This makes it easier to correctly and securely implement some graph - algorithms, even on potentially untrusted data in a distributed system. -2. Consistency: Without complex hacks, it's impossible to create link cycles - *between* hash-linked blocks. Allowing link cycles one one level (e.g., in - relative links within a single block) but not on a larger scale (between - blocks) is inconsistent (even though there's nothing we can do about it). - -## Stable Pathing - -An IPLD path always means the same thing, everywhere, every time. Importantly, -users shouldn't be able to configure their IPLD library to change how path -resolution works. They should be able to use alternative path resolution -algorithms for *non*-IPLD paths (e.g., IPFS paths) but those should be built -on top of IPLD. - -**Motivation:** Deterministic computations on top of a IPLD need to produce the -same result every time. - -## Link Transparent Pathing - -Path resolution must transparently traverse links. - -**Motivation:** Deduplication and composability. If links are *not* transparent, -programmers will have to either make many small objects (lots of hashing) or -inline data into large objects (lots of duplication and copying). - -## Primitives - -The "recommended" IPLD format (currently DagCBOR) needs to support *at a minimum*: - -* 32/64 bit integers without losing information. -* 32/64 bit floats without losing information. -* Unicode strings. -* Binary strings. -* Objects (with string keys, at least). -* Arrays. -* Booleans. -* A bottom type (null). - -**Motivation:** Convenience, really. - -## Non-Cyclic, Block-Local Relative Links - -That is, relative links that don't traverse out the back of an object. See the -conclusions from: [#1](https://github.com/ipld/specs/issues/1). - -**Motivation:** This is required to efficiently represent a highly connected DAG -of tiny nodes. - -**Caveat:** This brings in some sticky issues around mutability. Depending on -the implementation, relative links within an object may be act like mutable -links (from the perspective of the user). The concern here is that we don't want -users to bundle nodes together into single block *because* they want this -mutability. - -# To Do - -Working through this, I realized we have a few things we really need to finish a few things before we can -call IPLD ready. - -* **Path links.** Pointers that can only point to objects at block boundaries - are useful but severely gimped. We've been fine up till now because we - generally don't *edit* complicated datastructures but this will change. - ([#83](https://github.com/ipld/specs/issues/83)) -* **Slice links.** For the same reason, we really should support - `/ipld/QmID/start..stop` as a syntax for slicing an array. Most programming - languages support this so *not* supporting it would be a bit awkward. - ([#84](https://github.com/ipld/specs/issues/84)) -* **Link Spec.** We need to specify a complete and formal link spec and stick - with it. -* **Relative Links.** [#1](https://github.com/ipld/specs/issues/1). diff --git a/CID.md b/block-layer/CID.md similarity index 85% rename from CID.md rename to block-layer/CID.md index 5243c099..16cf484c 100644 --- a/CID.md +++ b/block-layer/CID.md @@ -1,10 +1,34 @@ -# CIDv1 +# Specification: CIDs -# Content IDs +**Status: Descriptive - Final** -This document will use the words Content IDs or CIDs. +This document will use the words "Content IDs" or "CIDs" interchangeably. -Prior base58 multihash links to protobuf data be called CID Version 0. +Prior Base58 Multihash links to Protobuf data is referred to as CID Version 0. + +## Summary + +A CID is a hash-based content identifier. Includes the `codec` and `multihash`. + +``` ++-------+------------------------------+ +| Codec | Multihash | ++-------+------------------------------+ +``` + +The long version: + +``` ++------------------------------+ +|Codec | ++------------------------------+ +|Multihash | +| +----------+---------------+ | +| |Hash Type | Hash Value | | +| +----------+---------------+ | +| | ++------------------------------+ +``` ## CIDs Version 1 diff --git a/block-layer/block.md b/block-layer/block.md new file mode 100644 index 00000000..41527d3b --- /dev/null +++ b/block-layer/block.md @@ -0,0 +1,27 @@ +# Concept: Block + +A IPLD Block is a CID and the binary data value for that CID. + +The short version: +``` ++-----+--------------------------------+ +| CID | Data | ++-----+--------------------------------+ +``` + +The long version: +``` ++-----------------------------------+------------------+ +| CID | Binary Data | +| +------------------------------+ | | +| |Codec | | | +| +------------------------------+ | | +| |Multihash | | | +| | +----------+---------------+ | | | +| | |Hash Type | Hash Value | | | | +| | +----------+---------------+ | | | +| | | | | +| +------------------------------+ | | +| | | ++-----------------------------------+------------------+ +``` \ No newline at end of file diff --git a/Codecs/DAG-CBOR.md b/block-layer/codecs/DAG-CBOR.md similarity index 81% rename from Codecs/DAG-CBOR.md rename to block-layer/codecs/DAG-CBOR.md index 1899b925..437f6414 100644 --- a/Codecs/DAG-CBOR.md +++ b/block-layer/codecs/DAG-CBOR.md @@ -1,8 +1,10 @@ -# [WIP] DagCBOR Spec +# Specification: DAG-CBOR -DAG-CBOR supports the full ["IPLD Data Model v1."](../IPLD-Data-Model-v1.md) +**Status: Descriptive - Final** -CBOR already natively supports all ["IPLD Data Model v1: Simple Types."](../IPLD-Data-Model-v1.md#simple-types) +DAG-CBOR supports the full [IPLD Data Model](../data-model-layer/data-model.md). + +CBOR already natively supports all [IPLD Data Model Kinds](../data-model-layer/data-model.md#kinds). ## Format diff --git a/Codecs/DAG-JSON.md b/block-layer/codecs/DAG-JSON.md similarity index 76% rename from Codecs/DAG-JSON.md rename to block-layer/codecs/DAG-JSON.md index 488f8fbe..a97e00b5 100644 --- a/Codecs/DAG-JSON.md +++ b/block-layer/codecs/DAG-JSON.md @@ -1,6 +1,8 @@ -# [WIP] DAG-JSON v1 +# Specification: DAG-JSON -DAG-JSON supports the full ["IPLD Data Model v1."](../IPLD-Data-Model-v1.md) +**Status: Descriptive - Final** + +DAG-JSON supports the full [IPLD Data Model](../data-model-layer/data-model.md). ## Format diff --git a/block-layer/codecs/README.md b/block-layer/codecs/README.md new file mode 100644 index 00000000..36d50dec --- /dev/null +++ b/block-layer/codecs/README.md @@ -0,0 +1,5 @@ +# IPLD Codecs + +A codec exposes serialization and deserialization for IPLD blocks. +If it also supports content addressable links then the codec exposes those links as CIDs. +A codec also supports atomic IPLD Path lookups on the block. diff --git a/block-layer/content-addressability.md b/block-layer/content-addressability.md new file mode 100644 index 00000000..6bee5ad8 --- /dev/null +++ b/block-layer/content-addressability.md @@ -0,0 +1,7 @@ +# Concept: Content Addressability + +"Content addressability" refers to the ability to refer to content by a trustless identifier. + +Rather than referring to content by a string identifier or URL, content addressable systems refer to content +by a cryptographic hash. This allows complete decentralization of the content as the identifier +does not specify the retrieval method and provides a secure way to verify the content. diff --git a/graphsync/graphsync.md b/block-layer/graphsync/graphsync.md similarity index 99% rename from graphsync/graphsync.md rename to block-layer/graphsync/graphsync.md index eae52b24..9bdc0cd6 100644 --- a/graphsync/graphsync.md +++ b/block-layer/graphsync/graphsync.md @@ -1,4 +1,6 @@ -# graphsync +# Graphsync + +**Status: Prescriptive - Draft** A protocol to synchronize graphs across peers. diff --git a/block-layer/multihash.md b/block-layer/multihash.md new file mode 100644 index 00000000..193d982a --- /dev/null +++ b/block-layer/multihash.md @@ -0,0 +1,21 @@ +# Concept: Multihash + +Multihash is hash format that is not specific to a single hashing algorithm. + +A multihash describes the algorithm used for the hash as well as the hash value. + +``` ++-----------+----------------------------+ +| Hash Type | Hash Value | ++-----------+----------------------------+ +``` + +SHA-256 example. + +``` ++---------+------------------------------------------------------------------+ +| SHA-256 | 2413fb3709b05939f04cf2e92f7d0897fc2596f9ad0b8a9ea855c7bfebaae892 | ++---------+------------------------------------------------------------------+ +``` + +Note: these examples are simplifications of the concepts. For a complete description visit the [project and its specs](https://github.com/multiformats/multihash). diff --git a/selectors/selectors.jpg b/block-layer/selectors/selectors.jpg similarity index 100% rename from selectors/selectors.jpg rename to block-layer/selectors/selectors.jpg diff --git a/selectors/selectors.md b/block-layer/selectors/selectors.md similarity index 99% rename from selectors/selectors.md rename to block-layer/selectors/selectors.md index 5280a4c9..8aa9dbe7 100644 --- a/selectors/selectors.md +++ b/block-layer/selectors/selectors.md @@ -1,4 +1,6 @@ -# IPLD Selectors +# Specification: IPLD Selectors + +**Status: Prescriptive - Draft** This document is a designdoc for IPLD Selectors. diff --git a/block-layer/serialization-and-formats.md b/block-layer/serialization-and-formats.md new file mode 100644 index 00000000..af398dc1 --- /dev/null +++ b/block-layer/serialization-and-formats.md @@ -0,0 +1,27 @@ +# Concept: Serialization and Formats + +A logical separation exists in any given IPLD codec between the **format** and the **serializer/deserializer**. + +``` +┌────────────────────┐ ┌────────────────────┐ +│ │ │ │ +│ Serializer │ │ Deserializer │ +│ │ │ │ +└─────────┬──────────┘ └──────────^─────────┘ + │ │ + │ Sent to another peer │ + │ │ +┌─────────v──────────┐ ┌──────────┴─────────┐ +│ │ │ │ +│ Format ├─────────────> Format │ +│ │ │ │ +└────────────────────┘ └────────────────────┘ +``` + +A **format** may represent object types and tree structures any way it wishes. +This includes existing representations (JSON, BSON, CBOR, Protobuf, msgpack, etc) or even new custom serializations. + +Therefor, a **format** is the standardized representation of IPLD Links and Paths. +It describes how to translate between structured data and binary. + +It is worth noting that **serializers** and **deserializers** differ by programming language while the **format** does not and MUST remain consistent across all codec implementations. diff --git a/IPLD-Data-Model-v1.md b/data-model-layer/data-model.md similarity index 97% rename from IPLD-Data-Model-v1.md rename to data-model-layer/data-model.md index 8f64665e..537f77c3 100644 --- a/IPLD-Data-Model-v1.md +++ b/data-model-layer/data-model.md @@ -1,5 +1,6 @@ -IPLD Data Model -=============== +# Specification: IPLD Data Model + +**Status: Descriptive - Draft** The IPLD Data Model is a core part of the IPLD specification, which describes what data is representable in IPLD -- diff --git a/IPLD-Path.md b/data-model-layer/paths.md similarity index 76% rename from IPLD-Path.md rename to data-model-layer/paths.md index 1119f496..a9184832 100644 --- a/IPLD-Path.md +++ b/data-model-layer/paths.md @@ -1,13 +1,19 @@ -# [WIP] IPLD Path v1 +# Specification: IPLD Paths -An IPLD Path is a string identifier used for deep references into IPLD -graphs. +**Status: Descriptive - Draft** + +## Summary + +An IPLD "Path" is a string identifier used for deep references into IPLD graphs. +Paths follow similar escape and segmentation rules as URI paths. + +An IPLD Path is a string identifier used for deep references into IPLD graphs. IPLD Path's are constructed following the same constraints as [URI Paths](https://tools.ietf.org/html/rfc3986#section-3.3). Similarly, the string `?` is reserved for future use as a query separator. -# Path Resolution +## Path Resolution Path resolution is broken into two parts: full path resolution and block level resolution. diff --git a/data-structures/multiblock-collections.md b/schema-layer/data-structures/multiblock-collections.md similarity index 99% rename from data-structures/multiblock-collections.md rename to schema-layer/data-structures/multiblock-collections.md index 80c3719c..e03265f8 100644 --- a/data-structures/multiblock-collections.md +++ b/schema-layer/data-structures/multiblock-collections.md @@ -1,4 +1,4 @@ -# IPLD Multi-block Collections +# Concept: IPLD Multi-block Collections Organizing IPLD data into usable, efficient, complex collections spanning many blocks aimed for use by end-user applications. diff --git a/schema/representations.md b/schema-layer/schemas/representations.md similarity index 99% rename from schema/representations.md rename to schema-layer/schemas/representations.md index c9ff99a3..03be7d2b 100644 --- a/schema/representations.md +++ b/schema-layer/schemas/representations.md @@ -1,4 +1,3 @@ - Kinds and their Representations ------------------------------- diff --git a/schema/schema-kinds.md b/schema-layer/schemas/schema-kinds.md similarity index 99% rename from schema/schema-kinds.md rename to schema-layer/schemas/schema-kinds.md index 27e28d7d..84e2815f 100644 --- a/schema/schema-kinds.md +++ b/schema-layer/schemas/schema-kinds.md @@ -1,5 +1,3 @@ - - Schema Kinds ------------ diff --git a/schema-layer/schemas/schemas.md b/schema-layer/schemas/schemas.md new file mode 100644 index 00000000..dfdd6bd6 --- /dev/null +++ b/schema-layer/schemas/schemas.md @@ -0,0 +1,3 @@ +# Specification: IPLD Schemas + +**Status: Prescriptive - Exploratory** diff --git a/schema/README.md b/schema/README.md deleted file mode 100644 index 6938a6a6..00000000 --- a/schema/README.md +++ /dev/null @@ -1,2 +0,0 @@ -IPLD Schemas -============