Skip to content

Commit

Permalink
dial in explanation
Browse files Browse the repository at this point in the history
  • Loading branch information
frankmcsherry committed Feb 7, 2025
1 parent cfb63c6 commit a559415
Showing 1 changed file with 10 additions and 7 deletions.
17 changes: 10 additions & 7 deletions posts/2024-11-25.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,23 @@ Many other systems provide [eventual consistency](https://en.wikipedia.org/wiki/
That's bad news when the change is continual: the outputs may be always nonsense.

Materialize provides a much clearer experience.
Every output Materialize produces is correct for some recent time, and we can even tell you what that time is.
These times are usually "within a second", and you can dial around your requirements here, but the results are always correct.
Materialize draws in your input data, and records the changes in a common timeline.
Every output Materialize produces corresponds exactly to the input data at some recent time.
It is as if you paused the world to evaluate your query.
We can tell you what that time is, or you can choose.

Many folks have been surprised about this guarantee when working with multiple upstream sources of data.
Indeed, one of Materialize's primary contributions is the introduction of structure such that it:
Many folks have been surprised about this claim when working with multiple upstream sources of data.
Indeed, one of Materialize's primary contributions is the introduction of a common timeline that:
1. faithfully reflects the transactional transitions of each input source,
2. brings together multiple transactional inputs into one common timeline,
3. produces results that are exactly correct for each moment on that timeline.
2. brings together multiple independent transactional sources of input data,
3. enables results that are exactly correct for every moment on that timeline.

How Materialize pulls this off is both subtle and at the same time surprisingly straight-forward.
It is not magic beans that violate fundamental theorems of distributed systems, but a relatively direct and potentially unsurprising combination of [virtual time](https://dl.acm.org/doi/10.1145/3916.3988) and [incremental computation](https://en.wikipedia.org/wiki/Incremental_computing).
Materialize interleaves the independent serializations of your upstream databases into one, not unlike how you might shuffle together two decks of cards: the order within each deck stays the same, but the interleaving of the decks is up to us.

We'll unpack how this unfolds, starting from your transactional source of truth, extended to other upstream sources, and across many independently authored and maintained SQL views.
By the end of the post, you should have a clear understanding of how Materialize aligns its input transactional data, and provides always consistent outputs.
By the end of the post, you should have a clear understanding of how Materialize aligns its input transactional data, and provides outputs that exactly track all changes in their inputs.

## Consistency and Change Data Capture (CDC)

Expand Down

0 comments on commit a559415

Please sign in to comment.