-
Notifications
You must be signed in to change notification settings - Fork 56
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Add/update documentation for connector kit (#138)
Co-authored-by: Florian Rusch (ZF Friedrichshafen AG) <florian.rusch.external@zf.com> Co-authored-by: Stephan Bauer <stephan@be-muc.de> Co-authored-by: stephanbcbauer <84396022+stephanbcbauer@users.noreply.github.com>
- Loading branch information
1 parent
1e841a7
commit 61689ee
Showing
18 changed files
with
680 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
--- | ||
id: Adoption View | ||
title: Adoption View | ||
description: 'Connector Kit' | ||
sidebar_position: 1 | ||
--- | ||
|
||
The ConnectorKit provides a connector framework, based on the [Eclipse Dataspace Connector][edc-url] for sovereign, cross-enterprise data exchange. | ||
|
||
![EDC Overview](images/edc_overview.png) | ||
|
||
Trust, interoperability and data sovereignty, these are the objectives and values for secure and sustainable peer-to-peer data exchange between organizations and companies. The claim is data sovereignty: Whoever makes data available retains control and decides individually who is involved in the data exchange, how, when, where and under what conditions. | ||
|
||
A corresponding concept was developed in the context of [Gaia-X][gaiax-url] and the [International Data Space Association][idsa-url]. The essential software component is the connector. | ||
|
||
With the [EDC][edc-url], a new central communication component was created, which implements the following architectural principles: | ||
|
||
- Simple, maintaining a small and efficient core with as few external dependencies as possible | ||
- Interoperable, independent of platforms and ecosystems | ||
- Decentralized, software components with the necessary capabilities for participating in a data room are located on the partners' side, data is only exchanged between the agreed points. | ||
- Data protection is more important than data sharing, data to be transmitted are fundamentally linked to policies via contracts; a transfer without a contract is not possible. | ||
- Separation of metadata and data enables high throughput rates for the actual data transfer. | ||
- Consistent semantics for the data is the basis for the consistency of digital value creation. | ||
- As far as possible, all processes, starting with determining the identity, through ensuring the contractually agreed regulations to data transmission, are automated. | ||
- Existing standards and protocols ([GAIA-X][gaiax-url] and [IDSA][idsa-url]) are used as far as possible. | ||
|
||
The [EDC][edc-url] as a connector implements a framework agreement for sovereign, cross-organizational data exchange. The International Data Spaces Standard (IDS) and relevant principles in connection with [GAIA-X][gaiax-url] were implemented. The connector is designed to be extensible to support alternative protocols and to be integrated into different ecosystems. | ||
|
||
The objective is to set up a decentralized software component on the part of the respective partner, which bundles the skills required to participate in a data room and enables peer-to-peer connections between participants. | ||
The focus here is particularly on the data sovereignty of the independent companies. | ||
The functionality required for this is bundled in the open-source project "Eclipse Dataspace Connectors", to which the Catena-X partners contribute as part of the Eclipse Foundation. | ||
|
||
The main difference between the EDC and the previous connectors of the [IDSA][idsa-url] is the separation of the communication into a channel for the metadata and one for the actual data exchange. The channel for the data supports various transmission protocols via so-called data plane extensions. The metadata is transmitted directly via the EDC interface, while the actual data exchange then takes place via the appropriate channel extension. In this way, a highly scalable data exchange is made possible. | ||
|
||
![EDC Architecture](images/edc_architecture.png) | ||
|
||
The architecture of the EDC combines various services that are necessary for the above principles: | ||
|
||
- An interface to the Identity Provider service, currently [IDSA][idsa-url]'s [Dynamic Attribute Provisioning System][daps-url]. This central service provides the identity and the corresponding authentication of the participants in the data exchange. (There is no authorization at this point). Decentralized solutions will also be supported in the future. | ||
- The provision of possible offers (contract offering) which, on the one hand, stipulates the data offered and the associated terms of use (policies) in corresponding contracts. | ||
- An interface for manual selection of data and associated contract offers. | ||
- The actual data transfer via the data plane extension | ||
- The connection of software systems on the customer and provider side | ||
|
||
[edc-url]: https://github.com/eclipse-edc/Connector | ||
[gaiax-url]: https://www.data-infrastructure.eu/GAIAX/Navigation/EN/Home/home.html | ||
[idsa-url]: https://internationaldataspaces.org/ | ||
[daps-url]: https://www.dataspaces.fraunhofer.de/en/software/identity_provider.html |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,64 @@ | ||
--- | ||
id: Domain Model | ||
title: Domain Model | ||
description: 'Connector Kit' | ||
sidebar_position: 2 | ||
--- | ||
|
||
![domain-model](images/domain-model.png) | ||
|
||
> The shown picture illustrates only a generic view of the Domain Model and is not intended to show all aspects of the project. | ||
## Asset | ||
|
||
An asset represents data (databases, files, cache information, etc.) which should be published and shared between | ||
organizations. For each asset, a [`DataAddress`](#data-address) needs to be resolvable. | ||
|
||
## Data address | ||
|
||
A data address is a pointer into the physical storage location where an asset will be stored. | ||
|
||
## Contract | ||
|
||
A contract always contains one or more [`Assets`](#asset) and a single [`Policy`](#policy). The contract construct is | ||
used to define the arrangement between two parties ("consumer" and "provider"). Regarding this arrangement, the contract | ||
passes several stages which are explained below: | ||
|
||
### Contract definition | ||
|
||
Contract definitions associate a policy with assets. A `ContractDefinition` object contains an access policy, a contract | ||
policy, and an asset selector which links the contract to one or more assets. | ||
|
||
### Contract offer | ||
|
||
The contract offer is a dynamic representation of the [`ContractDefinition`](#contract-definition) | ||
for a specific consumer and serves as protocol's data transfer object (DTO) for a particular contract negotiation. | ||
Contract offers are not persisted and will be regenerated on every request. The connector acting as data provider will | ||
generate contract offers only for contract definitions dedicated to the organization or data space participant | ||
operating the requesting connector acting as data consumer. A contract offer is always related to a single asset of | ||
the `ContractDefinition` object (e.g. for a `ContractDefinition` containing three `Asset` objects, the connector will | ||
generate three `ContractOffer` objects). | ||
|
||
### Contract negotiation | ||
|
||
A `ContractNegotiation` captures the current state of the negotiation of a contract (`ContractOffer` -> | ||
`ContractAgreement`) between two parties. This process is inherently asynchronous. | ||
|
||
### Contract agreement | ||
|
||
A contract agreement represents the agreed-upon terms of access and usage of an asset's data between two data space | ||
participants, including a start and an end date and further relevant information. | ||
|
||
## Policy | ||
|
||
Contract policies represent permitted and prohibited actions over a certain asset. These actions can be limited further | ||
by constraints (temporal or spatial) and duties ("e.g. deletion of the data after 30 days"). | ||
|
||
## Transfer process | ||
|
||
After a successful contract negotiation, a `DataRequest` is sent from a consumer connector to a provider connector to | ||
initiate the data transfer. It references the requested [`Asset`](#asset) and [`ContractAgreement`](#contract-agreement) | ||
as well as information about the [data destination](#data-address). | ||
|
||
Similar to the `ContractNegotiation`, this object captures the current state of a data transfer. This process is | ||
inherently asynchronous, so the `TransferProcess` objects are stored in a backing data store (`TransferProcessStore`). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# Development View | ||
|
||
## Project Overview | ||
|
||
TractusX is an initiative of companies under the umbrella of the Eclipse Foundation. | ||
It is a pilot for the larger initiative of CatenaX. | ||
A broader overview of the project can be found on the initiative's [Github page][tractusx-edc-link] | ||
or the homepage of the [Eclipse Foundation](https://projects.eclipse.org/projects/automotive.tractusx). | ||
|
||
## The EDC | ||
|
||
The Eclipse Dataspace Connector is one of the core components facilitating TractusX. | ||
|
||
:::note TractusX EDC or Core EDC? | ||
|
||
This documentation is for TractusX EDC. | ||
It includes the Core EDC with all of its functionality. | ||
However, this core is supplemented by extensions that allow for the use of additional backends and connection types. | ||
Furthermore, the provided Helm charts, build configuration and tests allow for a smoother deployment. | ||
::: | ||
|
||
You can find the repository for the TractusX EDC [here][tractusx-edc-link]. | ||
|
||
[tractusx-edc-link]: https://github.com/eclipse-tractusx/tractusx-edc |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# The Eclipse Foundation | ||
|
||
## Eclipse Development Process | ||
|
||
This Eclipse Foundation open project is governed by the Eclipse Foundation | ||
Development Process and operates under the terms of the Eclipse IP Policy. | ||
|
||
* <https://eclipse.org/projects/dev_process> | ||
* <https://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf> | ||
|
||
## Eclipse Contributor Agreement | ||
|
||
In order to be able to contribute to Eclipse Foundation projects you must | ||
electronically sign the Eclipse Contributor Agreement (ECA). | ||
|
||
* <http://www.eclipse.org/legal/ECA.php> | ||
|
||
The ECA provides the Eclipse Foundation with a permanent record that you agree | ||
that each of your contributions will comply with the commitments documented in | ||
the Developer Certificate of Origin (DCO). Having an ECA on file associated with | ||
the email address matching the "Author" field of your contribution's Git commits | ||
fulfills the DCO's requirement that you sign-off on your contributions. | ||
|
||
For more information, please see the Eclipse Committer Handbook: | ||
<https://www.eclipse.org/projects/handbook/#resources-commit> | ||
|
||
## License | ||
|
||
Code in TractusX EDC is published under the [Apache License](https://github.com/eclipse-tractusx/tractusx-edc/blob/main/LICENSE). | ||
|
||
## Contact | ||
|
||
Contact the project developers via the project's "dev" list. | ||
|
||
* <https://accounts.eclipse.org/mailing-list/tractusx-dev> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
# Repository Structure | ||
|
||
The repository for TractusX EDC can be found [here](https://github.com/eclipse-tractusx/tractusx-edc). | ||
It contains the following components: | ||
|
||
## EDC Extensions | ||
|
||
The core EDC is extensible by design. | ||
TractusX EDC provides such extensions. | ||
These extensions and their documentation are available | ||
[here](https://github.com/eclipse-tractusx/tractusx-edc/blob/main/edc-extensions/README.md). | ||
|
||
## Gradle Files for EDC Builds | ||
|
||
Builds of TractusX EDC are performed via Gradle. | ||
To allow for different configurations, different builds are provided. | ||
For example separate secrets backends are supported, but require separate builds of EDC. | ||
Therefor, different builds are available for both | ||
[data plane](https://github.com/eclipse-tractusx/tractusx-edc/blob/main/edc-dataplane/README.md) | ||
and [control plane](https://github.com/eclipse-tractusx/tractusx-edc/blob/main/edc-controlplane/README.md), | ||
|
||
## Helm Charts for EDC Deployment | ||
|
||
To facilitate deployment of these different builds and their prerequisites, | ||
Helm charts are provided. The charts and their documentation can be found | ||
[here](https://github.com/eclipse-tractusx/tractusx-edc/blob/main/charts/README.md). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# Project Structure | ||
|
||
## Issue Tracking | ||
|
||
Issues are maintained in [GitHub Issues](https://github.com/eclipse-tractusx/tractusx-edc/issues). | ||
|
||
## Reporting Vulnerabilities | ||
|
||
Vulnerabilities in the TractusX code base are best reported directly to the | ||
[Eclipse Foundation](https://www.eclipse.org/security/). | ||
|
||
## Git Flow | ||
|
||
The TractusX EDC repository uses a Git Flow, with `main` as the development branch and `releases` as the release branch. | ||
Other branches should follow the naming conventions of `feature/x` or `hotfix/x`, though this is not strictly enforced. | ||
|
||
## Tooling | ||
|
||
We use Java 11 with Gradle for dependencies and builds. | ||
We use [Spotless](https://github.com/diffplug/spotless) for code formatting. | ||
Releases are in the form of Docker containers and Helm charts. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# Software Operation View | ||
|
||
## Introduction | ||
|
||
The following documentation will guide you through the TractusX EDC deployment. | ||
You will be setting up multiple controllers and enabling communication between them. | ||
|
||
:::note TractusX EDC or Core EDC? | ||
|
||
The following guide assumes the use of the TractusX EDC. | ||
It includes the Core EDC with all of its functionality. | ||
However, this core is supplemented by extensions that allow for the use of additional backends and connection types. | ||
Furthermore, the provided Helm charts, build configuration and tests allow for a smoother deployment. | ||
::: | ||
|
||
## Connector Components | ||
|
||
In a usual EDC environment, each participant would operate at least one connector. | ||
Each of these connectors consists of a control plane and a data plane. | ||
The control plane functions as administration layer and is responsible for resource management, contract negotiation and administering data transfer. | ||
The data plane does the heavy lifting of transferring and receiving data streams. | ||
|
||
Each of these planes comes in several variants, allowing for example secrets to be stored in Azure Vault or a Hashicorp Vault. | ||
The setup on the following pages assumes the use of Hashicorp Vault for secrets and PostgreSQL for data storage. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
# Technical Prerequisites | ||
|
||
## Obtaining Releases | ||
|
||
The most recent release of TractusX EDC can be obtained under `https://github.com/eclipse-tractusx/tractusx-edc/releases`. | ||
To create your own build, you can clone the repository at `https://github.com/eclipse-tractusx/tractusx-edc` and consult the provided README.md. | ||
This can be useful if you want to use non-standard extensions or configuration. | ||
|
||
## Container Environment | ||
|
||
TractusX releases come in the form of Docker containers and corresponding Helm charts. | ||
As such, recent versions of the following are required. | ||
|
||
- Docker | ||
- Kubernetes | ||
- Helm | ||
|
||
Seeing as these are standard tools, TractusX EDC will run on any cloud environment that can accept Helm charts. | ||
|
||
## Backend Dependencies | ||
|
||
The EDC requires backend services for persistence of data and secrets. The following backends are currently supported. | ||
|
||
Data Storage: | ||
|
||
- PostgreSQL database | ||
- In memory database | ||
|
||
Secret Storage: | ||
|
||
- Hashicorp Vault | ||
- Azure Vault | ||
|
||
The default setup assumes data storage via PostgreSQL database. | ||
In memory storage is only recommended for running tests. | ||
Hashicorp Vault is the default secret provider, because it is platform-agnostic. | ||
|
||
Helm charts are provided to set up these services locally. | ||
**These are not suited for production environments.** | ||
|
||
## All-in-one deployment | ||
|
||
An all-in-one deployment is no longer in scope and will not be provided. |
Oops, something went wrong.