docs(x/data): update module documentation (#975)

* docs(x/data): update module documentation

* wip

* docs(x/data): update module documentation

* update vuepress config

docs/.vuepress/config.js

@@ -147,8 +147,10 @@ module.exports = {
             '/modules/data/01_concepts',
             '/modules/data/02_state',
             '/modules/data/03_messages',
-            '/modules/data/04_events',
-            '/modules/data/05_client',
+            '/modules/data/04_queries',
+            '/modules/data/05_events',
+            '/modules/data/06_types',
+            '/modules/data/07_client',
           ],
         },
       ],

proto/regen/data/v1/tx.proto

@@ -7,7 +7,7 @@ import "regen/data/v1/types.proto";
 
 option go_package = "github.com/regen-network/regen-ledger/x/data";
 
-// Msg is the regen.data.v1alpha1 Msg service
+// Msg is the regen.data.v1 Msg service
 service Msg {
   // Anchor "anchors" a piece of data to the blockchain based on its secure
   // hash, effectively providing a tamper resistant timestamp.

x/data/spec/01_concepts.md

@@ -1,9 +1,57 @@
 # Concepts
 
-## Data Anchoring
+### IRI
 
-Proving that a piece of data was known to exist at a certain point in time. This can also be referred to as "secure timestamping".
+An Internationalized Resource Identifier (IRI) is used within the data module to identify a piece of data. The IRI for a piece of data is always unique and generated deterministically. The IRI contains the full [content hash](#content-hash) enabling support for stateless conversions between IRI and content hash.
 
-## Data Signing
+The IRI for a [raw content hash](#raw-content-hash) and a [graph content hash](#graph-content-hash) follow the same format but differ in how the content hash is encoded to accommodate for different properties. Each IRI includes a prefix, base58 encoded data (the encoded content hash), and a file extension.
 
-Asserting to the veracity and validity of a piece of data. Signing implies that the contents of the data are generally accepted to be true by the signer.
+The format for a content hash:
+
+```
+regen:113gdjFKcVCt13Za6vN7TtbgMM6LMSjRnu89BMCxeuHdkJ1hWUmy.rdf
+```
+
+The pattern for a raw content hash:
+
+```
+regen:{base58check(concat(byte(0x0), byte(digest_algorithm), hash))}.{media_type extension}
+```
+
+The pattern for a graph content hash:
+
+```
+regen:{base58check(concat(byte(0x1), byte(canonicalization_algorithm), byte(merkle_tree), byte(digest_algorithm), hash))}.rdf
+```
+
+### Content Hash
+
+A content hash is a hash-based content identifier for a piece of data. A content hash can either be of type [raw](#raw-content-hash) or [graph](#graph-content-hash). A content hash defines the hash (the content hash itself) and the digest algorithm used to generate the hash. Each type defines additional properties specific to its type.
+
+#### Raw Content Hash
+
+A raw content hash specifies "raw" data that does not use deterministic, canonical encoding. Users of raw content hashes must maintain a copy of the hashed data that is preserved bit by bit. In addition to defining the hash (the content hash itself) and the digest algorithm, a raw content hash also defines the media type (e.g. TXT, JSON, CSV, XML, PDF, etc.). For a complete list of the supported media types for a raw content hash, see [RawMediaType](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.RawMediaType).
+
+#### Graph Content Hash
+
+A graph content hash specifies "graph" data that conforms to the [RDF data model](https://www.w3.org/TR/rdf11-concepts/) and therefore uses deterministic, canonical encoding allowing implementations to choose from various formats for content hash encoding while maintaining the guarantee that the underlying canonical hash will not change. In addition to defining the hash (the content hash itself) and the digest algorithm, a graph content hash also defines the canonicalization algorithm and the type of merkle tree. In the current implementation, Universal RDF Dataset Canonicalization Algorithm 2015 (URDNA2015) is the only canonicalization algorithm supported and no merkle tree types are supported.
+
+### Anchor
+
+Anchoring data is a way to prove a piece of data was known to exist at a certain point in time. This can also be referred to as "secure timestamping". When data is anchored, the content hash is converted to a unique deterministic identifier (an [IRI](#iri)) that is stored on chain alongside a timestamp representing the time at which the data was anchored (i.e. the block time of the transaction).
+
+When anchoring data, the sender of the transaction is not attesting to the veracity of the data. The sender can simply be an intermediary providing secure timestamping services. [Attest](#attest) should be used to attest to the veracity of the data.
+
+If the data is altered in any way, both the content hash and the resulting identifier would be different and the data would need to be anchored again, creating a new content entry for the updated data and leaving the previous content entry as a record of the data prior to any changes.
+
+### Attest
+
+Attesting to data is a way to validate a piece of data. Attesting to data is comparable to signing a legal document and implies the contents of the data are accepted to be true by the attestor to the best of their knowledge. When data is attested to, an attestor entry is stored on chain including the identifier ([IRI](#iri)) of the anchored data, the address of the attestor, and a timestamp representing the time at which the data was attested to. The attestor entry can be thought of as a digital signature (separate from but dependant upon a transaction signature).
+
+Attesting to data is only supported with data that uses deterministic, canonical encoding (i.e. the attestor must provide a [graph content hash](#graph-content-hash)). If the data has not already been anchored, attesting to data will also anchor the data. If the same attestor attempts to attest to the same piece of data, the attestation will be ignored (i.e. the previous entry will not be updated and a new entry will not be added). Any number of different attestors can attest to the same piece of data. 
+
+### Resolver
+
+A resolver is used to retrieve data that has been stored off chain and anchored on chain. A resolver defines a URL that refers to an HTTP service where the data is served and should be made available with a simple GET request that takes the [IRI](#iri) of the anchored data.
+
+When defining a resolver, the address that defines the resolver becomes the manager of the resolver. The manager has the ability to register data to the resolver. When registering data that has not been anchored, the data will be automatically anchored before being registered to the resolver.

x/data/spec/02_state.md

@@ -1,3 +1,11 @@
 # State
 
-The `data` module uses a lookup table for storing content hashes that adhere to a canonical regen IRI scheme. The lookup table creates a short, unique binary identifier for the longer content hash using an efficient, non-cryptographic hash function.
+The `data` module uses the `orm` package as an abstraction layer over the `KVStore` that enables the creation of database tables with primary and secondary keys. The tables used within the data module are defined in the proto files available on [Buf Schema Registry](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1).
+
+<!-- listed alphabetically -->
+
+- [DataAnchor](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.DataAnchor)
+- [DataAttestor](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.DataAttestor)
+- [DataID](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.DataID)
+- [DataResolver](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.DataResolver)
+- [Resolver](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Resolver)

x/data/spec/03_messages.md

@@ -1,30 +1,12 @@
 # Msg Service
 
-## Anchor
+The data module provides a message service for interacting with the state of the data module. The messages are defined in the proto files available on [Buf Schema Registry](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1).
 
-`Anchor` "anchors" a piece of data to the blockchain based on its secure hash, effectively providing a tamper resistant timestamp. The sender in `Anchor` is not attesting to the veracity of the underlying data. They can simply be an intermediary providing timestamp services. `Attest` should be used to create a digital signature attesting to the veracity of some piece of data. 
+For examples on how to interact with state using the `regen` binary, see [regen tx data](../../commands/regen_tx_data.md).
 
-+++ https://github.com/regen-network/regen-ledger/blob/55d39dc7d9768e5a0bd18c48ca78cb4a02145e81/proto/regen/data/v1alpha2/tx.proto#L44-L53
+<!-- listed alphabetically -->
 
-### Validation:
-
-- `sender` must ba a valid address, and their signature must be present in the transaction
-- `hash` must be a valid content hash, either raw data that does not specify a canonical encoding or graph data that conforms to the RDF data model
-
-## Attest
-
-`Attest` allows for signing of an arbitrary piece of data on the blockchain. By attesting to data, the attestor is making a statement about the veracity of the data itself. It is like signing a legal document, meaning that I agree to all conditions and to the best of my knowledge everything is true. When anchoring data, the sender is not attesting to the veracity of the data, they are simply communicating that it exists.
-
-On-chain signatures have the following benefits:
-- on-chain identities can be managed using different cryptographic keys that change over time through key rotation practices
-- an on-chain identity may represent an organization and through delegation individual members may sign on behalf of the group
-- the blockchain transaction envelope provides built-in replay protection and timestamping
-
-`Attest` implicitly calls `Anchor` if the data was not already anchored. `Attest` can be called multiple times for the same content hash with different attestors and those attestors will be appended to the list of attestors.
-
-+++ https://github.com/regen-network/regen-ledger/blob/55d39dc7d9768e5a0bd18c48ca78cb4a02145e81/proto/regen/data/v1alpha2/tx.proto#L64-L77
-
-### Validation:
-
-- `attestors` must be a valid addresses, and their signatures must be present in the transaction
-- `hash` must be a valid content hash, either raw data that does not specify a canonical encoding or graph data that conforms to the RDF data model
+- [Anchor](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Msg.Anchor)
+- [Attest](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Msg.Attest)
+- [DefineResolver](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Msg.DefineResolver)
+- [RegisterResolver](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Msg.RegisterResolver)

x/data/spec/04_queries.md

@@ -0,0 +1,21 @@
+# Query Service
+
+The data module provides a query service for querying the state of the data module. The queries are defined in the proto files available on [Buf Schema Registry](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1).
+
+For examples on how to query state using gRPC and REST, see the [data client](07_client.md) documentation, and for examples using the `regen` binary, see [regen query data](../../commands/regen_query_data.html).
+
+<!-- listed alphabetically -->
+
+- [AttestorsByHash](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.AttestorsByHash)
+- [AttestorsByIRI](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.AttestorsByIRI)
+- [ByAttestor](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.ByAttestor)
+- [ByHash](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.ByHash)
+- [ByIRI](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.ByIRI)
+- [HashByIRI](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.HashByIRI)
+- [IRIByGraphHash](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.IRIByGraphHash)
+- [IRIByHash](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.IRIByHash)
+- [IRIByRawHash](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.IRIByRawHash)
+- [Resolver](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.Resolver)
+- [ResolversByIRI](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.ResolversByIRI)
+- [ResolversByHash](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.ResolversByHash)
+- [ResolversByUrl](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.Query.ResolversByUrl)

x/data/spec/05_events.md

@@ -0,0 +1,10 @@
+# Events
+
+The data module emits events upon the successful execution of messages. The events for the data module are defined in the proto files available on [Buf Schema Registry](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1).
+
+<!-- listed alphabetically -->
+
+- [EventAnchor](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.EventAnchor)
+- [EventAttest](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.EventAttest)
+- [EventDefineResolver](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.EventDefineResolver)
+- [EventRegisterResolver](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.EventRegisterResolver)

x/data/spec/06_types.md

@@ -0,0 +1,16 @@
+# Types
+
+The `data` module provides the following types used in multiple services. The types for the data module are defined in the proto files available on [Buf Schema Registry](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1).
+
+<!-- listed alphabetically -->
+
+- [AttestorEntry](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.AttestorEntry)
+- [ContentEntry](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.ContentEntry)
+- [ContentHash](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.ContentHash)
+- [ContentHash.Graph](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.ContentHash.Graph)
+- [ContentHash.Raw](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.ContentHash.Raw)
+- [ContentHashes](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.ContentHashes)
+- [DigestAlgorithm](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.DigestAlgorithm)
+- [GraphCanonicalizationAlgorithm](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.GraphCanonicalizationAlgorithm)
+- [GraphMerkleTree](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.GraphMerkleTree)
+- [RawMediaType](https://buf.build/regen/regen-ledger/docs/main:regen.data.v1#regen.data.v1.RawMediaType)

x/data/spec/README.md

@@ -21,10 +21,11 @@ The data module also enables the creation and management of data resolvers. When
 ## Contents
 
 1. **[Concepts](01_concepts.md)**
-2. **[State](02_state.md)**
-3. **[Msg Service](03_messages.md)**
-4. **[Events](04_events.md)**
-5. **[Client](05_client.md)**
+1. **[State](02_state.md)**
+1. **[Msg Service](03_messages.md)**
+1. **[Events](05_events.md)**
+1. **[Types](06_types.md)**
+1. **[Client](07_client.md)**
 
 ## Protobuf