Transfering data with Qilin.Cloud Pipelines
Qilin.Cloud is a composable cloud platform designed to effortlessly move data between various sources and destinations. For instance, it facilitates the seamless transfer of product information from an ERP system to Marketplaces, and vice versa.
The Qilin pipeline comprises a series of organized procedures aimed at efficiently transferring and modifying data from one system to another. Initially, data is collected from diverse sources like ERP, PIM, and WMS. Subsequently, this data undergoes processes like cleansing, aggregation, or enrichment. Once transformed, the data is directed to various destination systems such as marketplaces, storefronts, or other sales channels for further utilization. These data pipelines play a pivotal role in automating data flow and are integral for facilitating data-driven decision-making within businesses.
A simplified representation of the Qilin pipeline is illustrated below:
https://github.com/QilinCloud/QilinCloud/wiki/Definitions-&-Terminology#connector
The ingression module is an important part of Qilin.Cloud, playing a crucial role in handling downstream data for Qilin. This module encompasses essential tasks, including receiving incoming data, comparing variations, conducting processing, performing business validation, and asynchronously storing data into Qilin.Cloud.
The Egression module, an integral component of Qilin.Cloud, assumes responsibility for overseeing upstream data for Qilin prior to its transmission. This module encompasses pivotal tasks, including scheduling data transfers, conducting validation, and maintaining the status of data subsequent to its dispatch to the Output Connector.
Data routing serves as a mechanism for tracking the origin and destination of data within Qilin. Upon traversing the Data Routing component, data is duplicated and concurrently triggered across multiple pipelines, operating in parallel and independently. This ensures that if a particular pipeline encounters a failure, it remains isolated, preserving the functionality and integrity of other pipelines.
Data Filtering sift through and exclude unnecessary data before dispatching it to a target system. This functionality proves invaluable when dealing with voluminous data in the source system, allowing users to selectively push only data that satisfies specific criteria to the destination. It functions as an optional component, offering flexibility to apply filtering to individual object types such as Product, Order, or Offer, with distinct filter conditions defined by FilterPredicate.
-
Data Enrichment is part of the Data Pipeline, offering optional placement both before and after Data Filtering.
-
It facilitates the modification of data payloads prior to their transmission to a target system, encompassing functionalities such as currency conversion, text translation, address validation, and filtering out prohibited words. This versatility enables us to tailor and adapt data according to the unique requirements of each target channel.
-
Data Enrichment comprises a series of EnrichmentBlocks, wherein data undergoes sequential enrichment by each block. The output of one block serves as the input for the subsequent block, ensuring a systematic flow. Additionally, enrichment flow can be prematurely terminated within any block by setting a special flag known as IsCompleted, indicating that further processing is unnecessary.
-
The core of Data Enrichment are EnrichmentActions, which represent the fundamental operators capable of modifying data through mathematical operations, text truncation, and more. This system operates as a plugin framework, allowing for the development of Enrichment Actions not only by Qilin but also by external developers. For example, someone provided a plugins that receive current product information (title, description, ...) then send to a custom trained generative AI model to optimize these texts.
-
Within a single EnrichmentBlock:
- It contains a list of PredicateEnrichmentBranches along with a default PredicateBranch.
- When PredicateEnrichmentBranches is populated, the system prioritizes the first branch where the Predicate evaluation yields a True result for execution.
- In the absence of executed PredicateBranches, the DefaultBranch is triggered.
- Each Branch is equipped with a flag named IsCompleted, facilitating premature termination of the Enrichment process.
- During the execution of an EnrichmentBranch, objects undergo step-by-step enrichment by EnrichmentActions.
Note: To learn more about EnrichmentBlock, EnrichmentAction please read in Data models & API specification.
A Qilin user, who is a business owner aiming to streamline their online e-commerce operations, here are their expectations:
- They currently utilize an ERP system to centralize product information management and operate two distinct marketplaces, X and Y.
- They need to sell only "book" products at Marketplace X, "phone" products at Marketplace Y.
- Automated synchronization of product details from the ERP system to the respective Marketplaces whenever a product is created or updated.
- Before initiating data synchronization to Marketplace Y for "phone" products, they seek specific modifications based on predefined conditions:
- Append a postfix to the English product title:
- If the condition is labeled as "AsNew", append "(As New)" to the title.
- If the condition is labeled as "Demo", append "(Demo)" to the title.
- Otherwise, no modification is required.
- Translate the product title from English to German if the German title field is empty.
- Append a postfix to the English product title:
The first step is create our channels. After created successfully, we need to save 3 channel ids returned by Qilin for using later.
-
The ERP where to manage products
-
Marketplace X
-
Marketplace Y
The next, we use channel ids from previous step to create pipelines. Then save pipeline id returned by Qilin for using later.
- Pipeline From ERP to MP X
- Pipeline From ERP to MP Y
- Marketplace X only sells "book" Products
- Marketplace Y only sells "phone" Products
- Marketplace X: Not set up
- Marketplace Y
- Enrichment before Filtering: Update title postfix
- Enrichment after Filtering: Using DeepL API to translate German title, we put after Filtering to reduce call to external API
For testing our config, we will use our logs provided by Data Flow Tracking (DFT). We should need to look details on "blocks" field return by DFT API, this is a list of items showing how data pass to every single Qilin Component. Each block with has it Id and ParentId which refer to the previous block.
Because the tags is empty => Product not sync to any MarketPlace We could see two Filtering blocks in DFT has status Stopped
View Full Product Payload
{
"sourceId": "e4afc249-90f9-4ea7-8829-fec8cc225c2c",
"productId": "prod_demo0205_001",
"title": {
"en-GB": "Super product"
},
"brandId": "brand_hh_nike",
"description": {
"en-GB": "Product description"
},
"GTINs": [
"12341234"
],
"SKUs": [
"sku_hh_{{$randomPassword}}"
],
"MPNs": [
"mnp_1",
"mpn_2"
],
"shortDescription": {
"en-GB": "this is short desc for product1"
},
"tags": [],
"deepLinks": [
"https://example.com/",
"https://example2.com/"
],
"seoTitle": {
"en-GB": "this is seo title for product1"
},
"seoKeywords": {
"en-GB": [
"product1",
"seo-product1",
"ontop"
]
},
"seoDescription": {
"en-GB": "this is seo desc for product1"
},
"length": 10,
"width": 10,
"height": 10,
"weightNet": 10,
"weightGross": 10,
"variations": [],
"variationGroupId": "",
"categoryIds": [],
"shippingClassificationId": "ship-1",
"productGroup": {
"en-GB": "shoe"
},
"crossSellingIds": null,
"ISBNs": [
"978-3-16-148410-0"
],
"ASINs": [
"B017VXKVXE"
],
"hazardCodes": [
"hazardCode-1"
],
"TARICs": [
"taric-1",
"taric-2"
],
"countryOfOrigin": "VNM",
"MSRP": {
"amountInFractionalUnit": 10,
"currency": "USD",
"divisor": 100
},
"unit": 1,
"container": "container-1",
"basePriceReferenceVolume": 10,
"basePriceReferenceUnit": 1,
"minimumPurchaseQuantity": 10,
"purchaseQuantity": 12,
"condition": 1,
"flexibleAttributes": null,
"productStatus": "open",
"updatedDateExternal": null
}View Full DFT
{
"message": null,
"resultTime": "2024-05-02T10:10:26.5167557Z",
"body": {
"transaction": {
"id": "01HWWCGC6ND7798AS6H6B3AF8G",
"subscriptionId": "bfc12c9e-821d-4351-93db-4092508f5a6e",
"createdAt": "2024-05-02T10:09:55.021+00:00",
"fields": {
"objectType": "Product",
"action": "create",
"internalId": "54d8f50e-92e4-4ddd-8c91-1de8514ec6b3",
"externalId": "prod_demo0205_001",
"sourceId": "e4afc249-90f9-4ea7-8829-fec8cc225c2c",
"requestId": null,
"correlationId": null,
"targetIds": [
"0f262e69-0aa5-4988-83cd-1a7b4e9444a3",
"adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2"
]
},
"progress": {
"prePipelines": null,
"pipelines": {
"0f262e69-0aa5-4988-83cd-1a7b4e9444a3": {
"step": "Filtering",
"status": "Stopped"
},
"adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2": {
"step": "Filtering",
"status": "Stopped"
}
}
},
"status": "Passed",
"additionalInformation": {}
},
"blocks": [
{
"id": "01HWWCGERN4Z8GC3F3C81K3B10",
"transactionId": "01HWWCGC6ND7798AS6H6B3AF8G",
"parentId": "01HWWCGC6SCKW03TSM4T4M1PAY",
"createdAt": "2024-05-02T10:09:58.915+00:00",
"step": "Routing",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
},
{
"id": "01HWWCGERTR4Y6YSECKE024RHH",
"transactionId": "01HWWCGC6ND7798AS6H6B3AF8G",
"parentId": "01HWWCGERN4Z8GC3F3C81K3B10",
"createdAt": "2024-05-02T10:09:58.917+00:00",
"step": "Pipeline",
"status": "Passed",
"additionalInformation": {
"TargetName": null,
"TargetId": "0f262e69-0aa5-4988-83cd-1a7b4e9444a3"
},
"exceptionMessage": null
},
{
"id": "01HWWCGERZGRPZ2HCB0C1JNMMS",
"transactionId": "01HWWCGC6ND7798AS6H6B3AF8G",
"parentId": "01HWWCGERTR4Y6YSECKE024RHH",
"createdAt": "2024-05-02T10:09:58.917+00:00",
"step": "Filtering",
"status": "Stopped",
"additionalInformation": {
"TargetId": "0f262e69-0aa5-4988-83cd-1a7b4e9444a3",
"Policy": {
"Operator": "arrayContains",
"Path": "$.tags",
"Value": "book",
"Conditions": null
}
},
"exceptionMessage": null
},
{
"id": "01HWWCGES0NYMH0WAFD0TYWQ0F",
"transactionId": "01HWWCGC6ND7798AS6H6B3AF8G",
"parentId": "01HWWCGERN4Z8GC3F3C81K3B10",
"createdAt": "2024-05-02T10:09:58.917+00:00",
"step": "Pipeline",
"status": "Passed",
"additionalInformation": {
"TargetName": null,
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2"
},
"exceptionMessage": null
},
{
"id": "01HWWCGES8XG15DQ02V49BKH8W",
"transactionId": "01HWWCGC6ND7798AS6H6B3AF8G",
"parentId": "01HWWCGES0NYMH0WAFD0TYWQ0F",
"createdAt": "2024-05-02T10:09:58.917+00:00",
"step": "Enrichment",
"status": "Passed",
"additionalInformation": {
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2",
"EnrichmentBlocks": [
{
"Name": "Add Postfix base on ConditionName",
"Description": null,
"PredicateBranches": [
{
"Predicate": {
"Operator": "eq",
"Path": "$.conditionName",
"Value": "AsNew",
"Conditions": null
},
"Actions": [
{
"Action": "addPostfix",
"Path": "$.title.en-GB",
"IsFailAccepted": false
}
],
"IsCompleted": false
},
{
"Predicate": {
"Operator": "eq",
"Path": "$.conditionName",
"Value": "Demo",
"Conditions": null
},
"Actions": [
{
"Action": "addPostfix",
"Path": "$.title.en-GB",
"IsFailAccepted": false
}
],
"IsCompleted": false
}
],
"DefaultBranch": {
"Actions": [],
"IsCompleted": false
}
}
],
"Diff": [
{
"Op": "replace",
"Path": "$.title.en-GB",
"Value": "Super product (As New)",
"From": null
}
]
},
"exceptionMessage": null
},
{
"id": "01HWWCGES85Q0RDHY35QSD63KD",
"transactionId": "01HWWCGC6ND7798AS6H6B3AF8G",
"parentId": "01HWWCGES8XG15DQ02V49BKH8W",
"createdAt": "2024-05-02T10:09:58.917+00:00",
"step": "Filtering",
"status": "Stopped",
"additionalInformation": {
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2",
"Policy": {
"Operator": "arrayContains",
"Path": "$.tags",
"Value": "phone",
"Conditions": null
}
},
"exceptionMessage": null
},
{
"id": "01HWWCGC6SCKW03TSM4T4M1PAY",
"transactionId": "01HWWCGC6ND7798AS6H6B3AF8G",
"parentId": null,
"createdAt": "2024-05-02T10:09:59.965+00:00",
"step": "Ingression",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
}
]
},
"pagination": null
}Product now sync to Marketplace X only. Marketplace Y filtered out.
View Full Product Payload
{
"sourceId": "e4afc249-90f9-4ea7-8829-fec8cc225c2c",
"productId": "prod_demo0205_003",
"title": {
"en-GB": "Super product"
},
"brandId": "brand_hh_nike",
"description": {
"en-GB": "Product description"
},
"GTINs": [
"12341234"
],
"SKUs": [
"sku_hh_{{$randomPassword}}"
],
"MPNs": [
"mnp_1",
"mpn_2"
],
"shortDescription": {
"en-GB": "this is short desc for product1"
},
"tags": ["book"],
"deepLinks": [
"https://example.com/",
"https://example2.com/"
],
"seoTitle": {
"en-GB": "this is seo title for product1"
},
"seoKeywords": {
"en-GB": [
"product1",
"seo-product1",
"ontop"
]
},
"seoDescription": {
"en-GB": "this is seo desc for product1"
},
"length": 10,
"width": 10,
"height": 10,
"weightNet": 10,
"weightGross": 10,
"variations": [],
"variationGroupId": "",
"categoryIds": [],
"shippingClassificationId": "ship-1",
"productGroup": {
},
"crossSellingIds": null,
"ISBNs": [
"978-3-16-148410-0"
],
"ASINs": [
"B017VXKVXE"
],
"hazardCodes": [
"hazardCode-1"
],
"TARICs": [
"taric-1",
"taric-2"
],
"countryOfOrigin": "VNM",
"MSRP": {
"amountInFractionalUnit": 10,
"currency": "USD",
"divisor": 100
},
"unit": 1,
"container": "container-1",
"basePriceReferenceVolume": 10,
"basePriceReferenceUnit": 1,
"minimumPurchaseQuantity": 10,
"purchaseQuantity": 12,
"condition": 1,
"flexibleAttributes": null,
"productStatus": "open",
"updatedDateExternal": null
}View Full DFT
{
"message": null,
"resultTime": "2024-05-02T10:35:50.2001768Z",
"body": {
"transaction": {
"id": "01HWWDY6Q3B0X56RXC06X6RT36",
"subscriptionId": "bfc12c9e-821d-4351-93db-4092508f5a6e",
"createdAt": "2024-05-02T10:34:56.712+00:00",
"fields": {
"objectType": "Product",
"action": "create",
"internalId": "0ace4844-2e00-4831-8c25-877195e08852",
"externalId": "prod_demo0205_003",
"sourceId": "e4afc249-90f9-4ea7-8829-fec8cc225c2c",
"requestId": null,
"correlationId": null,
"targetIds": [
"0f262e69-0aa5-4988-83cd-1a7b4e9444a3",
"adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2"
]
},
"progress": {
"prePipelines": null,
"pipelines": {
"0f262e69-0aa5-4988-83cd-1a7b4e9444a3": {
"step": "OutputConnector",
"status": "Passed"
},
"adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2": {
"step": "Filtering",
"status": "Stopped"
}
}
},
"status": "Passed",
"additionalInformation": {}
},
"blocks": [
{
"id": "01HWWDY6Q76R23KSACMA0B7K39",
"transactionId": "01HWWDY6Q3B0X56RXC06X6RT36",
"parentId": null,
"createdAt": "2024-05-02T10:34:57.133+00:00",
"step": "Ingression",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
},
{
"id": "01HWWDY7ZVGGAAZW029BZ2WVZ3",
"transactionId": "01HWWDY6Q3B0X56RXC06X6RT36",
"parentId": "01HWWDY6Q76R23KSACMA0B7K39",
"createdAt": "2024-05-02T10:34:59.173+00:00",
"step": "Routing",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
},
{
"id": "01HWWDY7ZVFGD6X39PV553DPHX",
"transactionId": "01HWWDY6Q3B0X56RXC06X6RT36",
"parentId": "01HWWDY7ZVGGAAZW029BZ2WVZ3",
"createdAt": "2024-05-02T10:34:59.173+00:00",
"step": "Pipeline",
"status": "Passed",
"additionalInformation": {
"TargetName": null,
"TargetId": "0f262e69-0aa5-4988-83cd-1a7b4e9444a3"
},
"exceptionMessage": null
},
{
"id": "01HWWDY7ZVEC1EXMVCB8ATRMM4",
"transactionId": "01HWWDY6Q3B0X56RXC06X6RT36",
"parentId": "01HWWDY7ZVFGD6X39PV553DPHX",
"createdAt": "2024-05-02T10:34:59.173+00:00",
"step": "Filtering",
"status": "Passed",
"additionalInformation": {
"TargetId": "0f262e69-0aa5-4988-83cd-1a7b4e9444a3",
"Policy": {
"Operator": "arrayContains",
"Path": "$.tags",
"Value": "book",
"Conditions": null
}
},
"exceptionMessage": null
},
{
"id": "01HWWDY7ZVFGK71J1MJ1DQ2GG4",
"transactionId": "01HWWDY6Q3B0X56RXC06X6RT36",
"parentId": "01HWWDY7ZVGGAAZW029BZ2WVZ3",
"createdAt": "2024-05-02T10:34:59.173+00:00",
"step": "Pipeline",
"status": "Passed",
"additionalInformation": {
"TargetName": null,
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2"
},
"exceptionMessage": null
},
{
"id": "01HWWDY7ZV13CECR3XGX9K644H",
"transactionId": "01HWWDY6Q3B0X56RXC06X6RT36",
"parentId": "01HWWDY7ZVFGK71J1MJ1DQ2GG4",
"createdAt": "2024-05-02T10:34:59.173+00:00",
"step": "Enrichment",
"status": "Passed",
"additionalInformation": {
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2",
"EnrichmentBlocks": [
{
"Name": "Add Postfix base on ConditionName",
"Description": null,
"PredicateBranches": [
{
"Predicate": {
"Operator": "eq",
"Path": "$.conditionName",
"Value": "AsNew",
"Conditions": null
},
"Actions": [
{
"Action": "addPostfix",
"Path": "$.title.en-GB",
"IsFailAccepted": false
}
],
"IsCompleted": false
},
{
"Predicate": {
"Operator": "eq",
"Path": "$.conditionName",
"Value": "Demo",
"Conditions": null
},
"Actions": [
{
"Action": "addPostfix",
"Path": "$.title.en-GB",
"IsFailAccepted": false
}
],
"IsCompleted": false
}
],
"DefaultBranch": {
"Actions": [],
"IsCompleted": false
}
}
],
"Diff": [
{
"Op": "replace",
"Path": "$.title.en-GB",
"Value": "Super product (As New)",
"From": null
}
]
},
"exceptionMessage": null
},
{
"id": "01HWWDY7ZWDZTMXNZ7XFYZBHWV",
"transactionId": "01HWWDY6Q3B0X56RXC06X6RT36",
"parentId": "01HWWDY7ZV13CECR3XGX9K644H",
"createdAt": "2024-05-02T10:34:59.173+00:00",
"step": "Filtering",
"status": "Stopped",
"additionalInformation": {
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2",
"Policy": {
"Operator": "arrayContains",
"Path": "$.tags",
"Value": "phone",
"Conditions": null
}
},
"exceptionMessage": null
},
{
"id": "01HWWDY9GGESVD0M7N7PN9DDVR",
"transactionId": "01HWWDY6Q3B0X56RXC06X6RT36",
"parentId": "01HWWDY7ZVEC1EXMVCB8ATRMM4",
"createdAt": "2024-05-02T10:35:02.256+00:00",
"step": "Egression",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
},
{
"id": "01HWWDY9GG9JB992NM5Y87X6CN",
"transactionId": "01HWWDY6Q3B0X56RXC06X6RT36",
"parentId": "01HWWDY9GGESVD0M7N7PN9DDVR",
"createdAt": "2024-05-02T10:35:02.256+00:00",
"step": "OutputConnector",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
}
]
},
"pagination": null
}Product is synced to Marketplace Y with:
- English title has Postfix (As New)
- German title translated from English
View Full Product Payload
{
"sourceId": "e4afc249-90f9-4ea7-8829-fec8cc225c2c",
"productId": "prod_demo0205_005",
"title": {
"en-GB": "Super phone",
"de-DE": ""
},
"brandId": "brand_hh_nike",
"description": {
"en-GB": "Product description"
},
"GTINs": [
"12341234"
],
"SKUs": [
"sku_hh_{{$randomPassword}}"
],
"MPNs": [
"mnp_1",
"mpn_2"
],
"shortDescription": {
"en-GB": "this is short desc for product1"
},
"tags": ["phone"],
"deepLinks": [
"https://example.com/",
"https://example2.com/"
],
"seoTitle": {
"en-GB": "this is seo title for product1"
},
"seoKeywords": {
"en-GB": [
"product1",
"seo-product1",
"ontop"
]
},
"seoDescription": {
"en-GB": "this is seo desc for product1"
},
"length": 10,
"width": 10,
"height": 10,
"weightNet": 10,
"weightGross": 10,
"variations": [],
"variationGroupId": "",
"categoryIds": [],
"shippingClassificationId": "ship-1",
"productGroup": {
},
"crossSellingIds": null,
"ISBNs": [
"978-3-16-148410-0"
],
"ASINs": [
"B017VXKVXE"
],
"hazardCodes": [
"hazardCode-1"
],
"TARICs": [
"taric-1",
"taric-2"
],
"countryOfOrigin": "VNM",
"MSRP": {
"amountInFractionalUnit": 10,
"currency": "USD",
"divisor": 100
},
"unit": 1,
"container": "container-1",
"basePriceReferenceVolume": 10,
"basePriceReferenceUnit": 1,
"minimumPurchaseQuantity": 10,
"purchaseQuantity": 12,
"condition": "AsNew",
"flexibleAttributes": null,
"productStatus": "open",
"updatedDateExternal": null
}View Full DFT
{
"message": null,
"resultTime": "2024-05-02T10:56:02.893357Z",
"body": {
"transaction": {
"id": "01HWWF3GSCRK7X49XEMZVBAVM4",
"subscriptionId": "bfc12c9e-821d-4351-93db-4092508f5a6e",
"createdAt": "2024-05-02T10:55:19.361+00:00",
"fields": {
"objectType": "Product",
"action": "create",
"internalId": "945209b5-ef59-4642-8a42-84a5216d1dc9",
"externalId": "prod_demo0205_005",
"sourceId": "e4afc249-90f9-4ea7-8829-fec8cc225c2c",
"requestId": null,
"correlationId": null,
"targetIds": [
"0f262e69-0aa5-4988-83cd-1a7b4e9444a3",
"adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2"
]
},
"progress": {
"prePipelines": null,
"pipelines": {
"0f262e69-0aa5-4988-83cd-1a7b4e9444a3": {
"step": "Filtering",
"status": "Stopped"
},
"adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2": {
"step": "OutputConnector",
"status": "Passed"
}
}
},
"status": "Passed",
"additionalInformation": {}
},
"blocks": [
{
"id": "01HWWF3GSC6P871V2XX5CND185",
"transactionId": "01HWWF3GSCRK7X49XEMZVBAVM4",
"parentId": null,
"createdAt": "2024-05-02T10:55:23.297+00:00",
"step": "Ingression",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
},
{
"id": "01HWWF3J0MJQXCV6YA3WY6MZ79",
"transactionId": "01HWWF3GSCRK7X49XEMZVBAVM4",
"parentId": "01HWWF3GSC6P871V2XX5CND185",
"createdAt": "2024-05-02T10:55:26.39+00:00",
"step": "Routing",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
},
{
"id": "01HWWF3J0MYCZD0XAAC8S92YF6",
"transactionId": "01HWWF3GSCRK7X49XEMZVBAVM4",
"parentId": "01HWWF3J0MJQXCV6YA3WY6MZ79",
"createdAt": "2024-05-02T10:55:26.39+00:00",
"step": "Pipeline",
"status": "Passed",
"additionalInformation": {
"TargetName": null,
"TargetId": "0f262e69-0aa5-4988-83cd-1a7b4e9444a3"
},
"exceptionMessage": null
},
{
"id": "01HWWF3J0NKRJPPJJPQMPPKBNM",
"transactionId": "01HWWF3GSCRK7X49XEMZVBAVM4",
"parentId": "01HWWF3J0MYCZD0XAAC8S92YF6",
"createdAt": "2024-05-02T10:55:26.39+00:00",
"step": "Filtering",
"status": "Stopped",
"additionalInformation": {
"TargetId": "0f262e69-0aa5-4988-83cd-1a7b4e9444a3",
"Policy": {
"Operator": "arrayContains",
"Path": "$.tags",
"Value": "book",
"Conditions": null
}
},
"exceptionMessage": null
},
{
"id": "01HWWF3J0NAH3E50AJDV0MTY8P",
"transactionId": "01HWWF3GSCRK7X49XEMZVBAVM4",
"parentId": "01HWWF3J0MJQXCV6YA3WY6MZ79",
"createdAt": "2024-05-02T10:55:26.39+00:00",
"step": "Pipeline",
"status": "Passed",
"additionalInformation": {
"TargetName": null,
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2"
},
"exceptionMessage": null
},
{
"id": "01HWWF3J0NVBWN0WNZHCBMF32F",
"transactionId": "01HWWF3GSCRK7X49XEMZVBAVM4",
"parentId": "01HWWF3J0NAH3E50AJDV0MTY8P",
"createdAt": "2024-05-02T10:55:26.39+00:00",
"step": "Enrichment",
"status": "Passed",
"additionalInformation": {
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2",
"EnrichmentBlocks": [
{
"Name": "Add Postfix base on ConditionName",
"Description": null,
"PredicateBranches": [
{
"Predicate": {
"Operator": "eq",
"Path": "$.conditionName",
"Value": "AsNew",
"Conditions": null
},
"Actions": [
{
"Action": "addPostfix",
"Path": "$.title.en-GB",
"IsFailAccepted": false
}
],
"IsCompleted": false
},
{
"Predicate": {
"Operator": "eq",
"Path": "$.conditionName",
"Value": "Demo",
"Conditions": null
},
"Actions": [
{
"Action": "addPostfix",
"Path": "$.title.en-GB",
"IsFailAccepted": false
}
],
"IsCompleted": false
}
],
"DefaultBranch": {
"Actions": [],
"IsCompleted": false
}
}
],
"Diff": [
{
"Op": "replace",
"Path": "$.title.en-GB",
"Value": "Super phone (As New)",
"From": null
}
]
},
"exceptionMessage": null
},
{
"id": "01HWWF3J0NFNCRWSTN7SNB4P3W",
"transactionId": "01HWWF3GSCRK7X49XEMZVBAVM4",
"parentId": "01HWWF3J0NVBWN0WNZHCBMF32F",
"createdAt": "2024-05-02T10:55:26.39+00:00",
"step": "Filtering",
"status": "Passed",
"additionalInformation": {
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2",
"Policy": {
"Operator": "arrayContains",
"Path": "$.tags",
"Value": "phone",
"Conditions": null
}
},
"exceptionMessage": null
},
{
"id": "01HWWF3QKG3C29ZD3CY4TFD9FN",
"transactionId": "01HWWF3GSCRK7X49XEMZVBAVM4",
"parentId": "01HWWF3J0NFNCRWSTN7SNB4P3W",
"createdAt": "2024-05-02T10:55:26.39+00:00",
"step": "Enrichment",
"status": "Passed",
"additionalInformation": {
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2",
"EnrichmentBlocks": [
{
"Name": "Translating to German by DeepL",
"Description": null,
"PredicateBranches": [
{
"Predicate": {
"Operator": "empty",
"Path": "$.title.de-DE",
"Value": null,
"Conditions": null
},
"Actions": [
{
"Action": "deepLTranslate",
"Path": "$.title.de-DE",
"IsFailAccepted": false
}
],
"IsCompleted": false
}
],
"DefaultBranch": {
"Actions": [],
"IsCompleted": false
}
}
],
"Diff": [
{
"Op": "replace",
"Path": "$.title.de-DE",
"Value": "Super Telefon (wie neu)",
"From": null
}
]
},
"exceptionMessage": null
},
{
"id": "01HWWF3SB0VV9M0FP9X37JKJ24",
"transactionId": "01HWWF3GSCRK7X49XEMZVBAVM4",
"parentId": "01HWWF3QKG3C29ZD3CY4TFD9FN",
"createdAt": "2024-05-02T10:55:29.427+00:00",
"step": "Egression",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
},
{
"id": "01HWWF3SB09WTRCNX6AD9W1FT6",
"transactionId": "01HWWF3GSCRK7X49XEMZVBAVM4",
"parentId": "01HWWF3SB0VV9M0FP9X37JKJ24",
"createdAt": "2024-05-02T10:55:29.427+00:00",
"step": "OutputConnector",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
}
]
},
"pagination": null
}Product is synced to Marketplace Y with:
- English title has Postfix (Demo)
- German title not translated from English, keep same as beginning when creating
View Full Product Payload
{
"sourceId": "e4afc249-90f9-4ea7-8829-fec8cc225c2c",
"productId": "prod_demo0205_006",
"title": {
"en-GB": "Super phone",
"de-DE": "German original title"
},
"brandId": "brand_hh_nike",
"description": {
"en-GB": "Product description"
},
"GTINs": [
"12341234"
],
"SKUs": [
"sku_hh_{{$randomPassword}}"
],
"MPNs": [
"mnp_1",
"mpn_2"
],
"shortDescription": {
"en-GB": "this is short desc for product1"
},
"tags": ["phone"],
"deepLinks": [
"https://example.com/",
"https://example2.com/"
],
"seoTitle": {
"en-GB": "this is seo title for product1"
},
"seoKeywords": {
"en-GB": [
"product1",
"seo-product1",
"ontop"
]
},
"seoDescription": {
"en-GB": "this is seo desc for product1"
},
"length": 10,
"width": 10,
"height": 10,
"weightNet": 10,
"weightGross": 10,
"variations": [],
"variationGroupId": "",
"categoryIds": [],
"shippingClassificationId": "ship-1",
"productGroup": {
},
"crossSellingIds": null,
"ISBNs": [
"978-3-16-148410-0"
],
"ASINs": [
"B017VXKVXE"
],
"hazardCodes": [
"hazardCode-1"
],
"TARICs": [
"taric-1",
"taric-2"
],
"countryOfOrigin": "VNM",
"MSRP": {
"amountInFractionalUnit": 10,
"currency": "USD",
"divisor": 100
},
"unit": 1,
"container": "container-1",
"basePriceReferenceVolume": 10,
"basePriceReferenceUnit": 1,
"minimumPurchaseQuantity": 10,
"purchaseQuantity": 12,
"condition": "Demo",
"flexibleAttributes": null,
"productStatus": "open",
"updatedDateExternal": null
}View Full DFT
{
"message": null,
"resultTime": "2024-05-02T10:59:53.1292842Z",
"body": {
"transaction": {
"id": "01HWWFB5VB48J3GF4DCY856AG0",
"subscriptionId": "bfc12c9e-821d-4351-93db-4092508f5a6e",
"createdAt": "2024-05-02T10:59:30.405+00:00",
"fields": {
"objectType": "Product",
"action": "create",
"internalId": "0c7decc4-f14c-4523-aa97-b236ba675c9a",
"externalId": "prod_demo0205_006",
"sourceId": "e4afc249-90f9-4ea7-8829-fec8cc225c2c",
"requestId": null,
"correlationId": null,
"targetIds": [
"0f262e69-0aa5-4988-83cd-1a7b4e9444a3",
"adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2"
]
},
"progress": {
"prePipelines": null,
"pipelines": {
"0f262e69-0aa5-4988-83cd-1a7b4e9444a3": {
"step": "Filtering",
"status": "Stopped"
},
"adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2": {
"step": "OutputConnector",
"status": "Passed"
}
}
},
"status": "Passed",
"additionalInformation": {}
},
"blocks": [
{
"id": "01HWWFB7079K93CSK90C83KA5G",
"transactionId": "01HWWFB5VB48J3GF4DCY856AG0",
"parentId": "01HWWFB5VFN2MHDXGZQ90EZK4H",
"createdAt": "2024-05-02T10:59:32.476+00:00",
"step": "Routing",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
},
{
"id": "01HWWFB707128C4W4PAHRSDW47",
"transactionId": "01HWWFB5VB48J3GF4DCY856AG0",
"parentId": "01HWWFB7079K93CSK90C83KA5G",
"createdAt": "2024-05-02T10:59:32.476+00:00",
"step": "Pipeline",
"status": "Passed",
"additionalInformation": {
"TargetName": null,
"TargetId": "0f262e69-0aa5-4988-83cd-1a7b4e9444a3"
},
"exceptionMessage": null
},
{
"id": "01HWWFB707SB39QGKKZ37JQRXX",
"transactionId": "01HWWFB5VB48J3GF4DCY856AG0",
"parentId": "01HWWFB707128C4W4PAHRSDW47",
"createdAt": "2024-05-02T10:59:32.476+00:00",
"step": "Filtering",
"status": "Stopped",
"additionalInformation": {
"TargetId": "0f262e69-0aa5-4988-83cd-1a7b4e9444a3",
"Policy": {
"Operator": "arrayContains",
"Path": "$.tags",
"Value": "book",
"Conditions": null
}
},
"exceptionMessage": null
},
{
"id": "01HWWFB707BH3EHG517MBMG6B4",
"transactionId": "01HWWFB5VB48J3GF4DCY856AG0",
"parentId": "01HWWFB7079K93CSK90C83KA5G",
"createdAt": "2024-05-02T10:59:32.476+00:00",
"step": "Pipeline",
"status": "Passed",
"additionalInformation": {
"TargetName": null,
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2"
},
"exceptionMessage": null
},
{
"id": "01HWWFB70768AB84T4DKW8M49C",
"transactionId": "01HWWFB5VB48J3GF4DCY856AG0",
"parentId": "01HWWFB707BH3EHG517MBMG6B4",
"createdAt": "2024-05-02T10:59:32.476+00:00",
"step": "Enrichment",
"status": "Passed",
"additionalInformation": {
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2",
"EnrichmentBlocks": [
{
"Name": "Add Postfix base on ConditionName",
"Description": null,
"PredicateBranches": [
{
"Predicate": {
"Operator": "eq",
"Path": "$.conditionName",
"Value": "AsNew",
"Conditions": null
},
"Actions": [
{
"Action": "addPostfix",
"Path": "$.title.en-GB",
"IsFailAccepted": false
}
],
"IsCompleted": false
},
{
"Predicate": {
"Operator": "eq",
"Path": "$.conditionName",
"Value": "Demo",
"Conditions": null
},
"Actions": [
{
"Action": "addPostfix",
"Path": "$.title.en-GB",
"IsFailAccepted": false
}
],
"IsCompleted": false
}
],
"DefaultBranch": {
"Actions": [],
"IsCompleted": false
}
}
],
"Diff": [
{
"Op": "replace",
"Path": "$.title.en-GB",
"Value": "Super phone (Demo)",
"From": null
}
]
},
"exceptionMessage": null
},
{
"id": "01HWWFB707A3QZ6JEKHH105AT7",
"transactionId": "01HWWFB5VB48J3GF4DCY856AG0",
"parentId": "01HWWFB70768AB84T4DKW8M49C",
"createdAt": "2024-05-02T10:59:32.476+00:00",
"step": "Filtering",
"status": "Passed",
"additionalInformation": {
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2",
"Policy": {
"Operator": "arrayContains",
"Path": "$.tags",
"Value": "phone",
"Conditions": null
}
},
"exceptionMessage": null
},
{
"id": "01HWWFB707NC5M87QJPTWC1AD3",
"transactionId": "01HWWFB5VB48J3GF4DCY856AG0",
"parentId": "01HWWFB707A3QZ6JEKHH105AT7",
"createdAt": "2024-05-02T10:59:32.476+00:00",
"step": "Enrichment",
"status": "Passed",
"additionalInformation": {
"TargetId": "adaa84e1-75dc-48f8-9f8d-14da1f4ac6a2",
"EnrichmentBlocks": [
{
"Name": "Translating to German by DeepL",
"Description": null,
"PredicateBranches": [
{
"Predicate": {
"Operator": "empty",
"Path": "$.title.de-DE",
"Value": null,
"Conditions": null
},
"Actions": [
{
"Action": "deepLTranslate",
"Path": "$.title.de-DE",
"IsFailAccepted": false
}
],
"IsCompleted": false
}
],
"DefaultBranch": {
"Actions": [],
"IsCompleted": false
}
}
],
"Diff": []
},
"exceptionMessage": null
},
{
"id": "01HWWFB5VFN2MHDXGZQ90EZK4H",
"transactionId": "01HWWFB5VB48J3GF4DCY856AG0",
"parentId": null,
"createdAt": "2024-05-02T10:59:35.203+00:00",
"step": "Ingression",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
},
{
"id": "01HWWFB8BTC9GCN0YGXSJJHB41",
"transactionId": "01HWWFB5VB48J3GF4DCY856AG0",
"parentId": "01HWWFB707NC5M87QJPTWC1AD3",
"createdAt": "2024-05-02T10:59:35.525+00:00",
"step": "Egression",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
},
{
"id": "01HWWFB8BTF4Q70W6PK8YB2FWY",
"transactionId": "01HWWFB5VB48J3GF4DCY856AG0",
"parentId": "01HWWFB8BTC9GCN0YGXSJJHB41",
"createdAt": "2024-05-02T10:59:35.525+00:00",
"step": "OutputConnector",
"status": "Passed",
"additionalInformation": {},
"exceptionMessage": null
}
]
},
"pagination": null
}