Accept frontier changes in subgraph set-up #360
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes a bug that mysteriously hasn't shown up until some stress testing over in MaterializeInc/database-issues#1731.
The issue is that in set-up, a Subgraph will propagate pointstamp information among its operators and then schedule them for their first time. However, it would not first accept updates from the outside world, which meant that the initial calls into operator logic would not reflect "safe" frontiers, especially if an operator's input was from outside the scope (there would be no initial capability locally within the subgraph to present at it). The observed behavior was a spurious empty frontier in the first call, followed by legit frontiers after that call.
In fact, this manifests in the doctest for
enterleave::enter_at
with enough panicky double-checking code (and indeed that example has an operator that consumes directly from outside the scope).This fix is very local, but another take is that
set_external_summary
could plausibly be deleted, as the surrounding scope no longer presents a summary downwards, and initial frontiers are presented in the initial population of the SharedProgress struct rather than by method call.