Skip to content

Commit

Permalink
Merge pull request #1717 from vtexdocs/review-data-consistency-md
Browse files Browse the repository at this point in the history
review data consistency guide
  • Loading branch information
julia-rabello authored Feb 7, 2025
2 parents 7f14160 + 783d85b commit d48e330
Showing 1 changed file with 21 additions and 20 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -15,47 +15,48 @@ This consistency is achieved through two approaches: strong consistency and even

With strong consistency, any change to data is immediately reflected and accessible after it occurs. This is achieved by locking access to the updated data until all instances have been synchronized.

This approach ensures that all users see the same data at the same time, without delay. For example, when using the `/documents` API to store and retrieve data, strong consistency ensures that the data is immediately up-to-date after any changes to a Document.
This approach ensures that all users see the same data at the same time, without delay. For example, when using the `/documents` API to store and retrieve data, strong consistency ensures that the data is immediately up-to-date after any changes to a document.

## Eventual consistency

In eventual consistency, changes take some time to propagate across instances. The updates are queued and processed asynchronously, meaning users may not see the updated data immediately. This approach is used in resource-intensive tasks, especially in large datasets. As a result, performance is optimized while synchronization occurs gradually.
In eventual consistency, changes take some time to propagate across instances, and synchronization occurs gradually. Updates are queued and processed asynchronously, meaning users may not see the updated data immediately. This approach enhances scalability and performance, especially for large datasets.

For example, the `/search` API uses eventual consistency. After a Document is updated, the change will appear in the search results only after the system has processed it.
For example, the `/search` API uses eventual consistency. After a document is updated, the change will appear in the search results only after the system has processed it. While this allows for optimized performance, there is a delay before updates become visible in search queries.

When data is persisted in Master Data, the Storage is updated in an atomic operation. A message is sent to the Master Data Worker to trigger actions like sending the updated Document to the Search Engine. Only after the Search Engine is updated will the changes be available through the `/search` API.
When data is persisted in Master Data, the Storage update occurs in an atomic operation, meaning all changes happen at once, ensuring data integrity. A message is sent to the Master Data Worker, which then triggers actions like forwarding the updated document to the Search engine. Only after the Search engine is updated changes are available through the `/search` API.

For more information on the update process in the Search Engine, refer to [Schema Lifecycle](https://developers.vtex.com/docs/guides/master-data-schema-lifecycle).

## Example

Consider the following Document:
Consider the following document:

```
```json
{
"id": "0e860678-83cc-4c21-8194-3377adcee0b7",
"firstName": "Jhon"
"firstName": "John"
}
```

If the `firstName` value is updated to `Super Jhon`, two subsequent API requests made immediately after the update may yield different results:

If the `firstName` value is updated to `Richard`, two subsequent API requests made immediately after the update may yield different results:

|Endpoint|Consistency Type|API Request|Response|
|-----|-------|-----------|---------|
|`/documents`|Strongly Consistent|`GET /documents/0e860678-83cc-4c21-8194-3377adcee0b7?_fields=id,firstName`|`{ "id": "0e860678-83cc-4c21-8194-3377adcee0b7", "firstName": "Super Jhon" }`|
|`/search`|Eventually Consistent|`GET /search?_where=firstName="Super Jhon"&_fields=id,firstName`|`{}`|
| Endpoint | Consistency type | API request | Response |
| - | - | - | - |
| `/documents` | Strong consistency |`GET /documents/0e860678-83cc-4c21-8194-3377adcee0b7?_fields=id,firstName` | `{ "id": "0e860678-83cc-4c21-8194-3377adcee0b7", "firstName": "Richard" }` |
| `/search` | Eventual consistency |`GET /search?_where=firstName="Richard"&_fields=id,firstName` | `{}` |

Note that the `/documents` API provides the updated data immediately (strong consistency), whereas the `/search` API reflects the change only after the update is processed asynchronously (eventual consistency).

## Consistency level by API endpoint
## Consistency level by endpoint

The table below shows the consistency level for each API endpoint.
The table below shows the consistency level for each Master Data API path.

| Name | Path | Level |
| - | - | - |
| Documents | `/documents` | `Strongly Consistent` |
| Indices | `/indices` | `Strongly Consistent` |
| Schemas | `/schemas` | `Strongly Consistent` |
| Search | `/search` | `Eventually Consistent` |
| Scroll | `/scroll` | `Eventually Consistent` |
| Documents | `/documents` | Strong consistency |
| Indices | `/indices` | Strong consistency |
| Schemas | `/schemas` | Strong consistency |
| Search | `/search` | Eventual consistency |
| Scroll | `/scroll` | Eventual consistency |

For more details on specific endpoints, refer to the [Master Data API - v1](https://developers.vtex.com/docs/api-reference/masterdata-api) and [Master Data API - v2](https://developers.vtex.com/docs/api-reference/master-data-api-v2) reference.

0 comments on commit d48e330

Please sign in to comment.