Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept frontier changes in subgraph set-up #360

Merged
merged 1 commit into from
Feb 17, 2021

Conversation

frankmcsherry
Copy link
Member

@frankmcsherry frankmcsherry commented Feb 17, 2021

This PR fixes a bug that mysteriously hasn't shown up until some stress testing over in MaterializeInc/database-issues#1731.

The issue is that in set-up, a Subgraph will propagate pointstamp information among its operators and then schedule them for their first time. However, it would not first accept updates from the outside world, which meant that the initial calls into operator logic would not reflect "safe" frontiers, especially if an operator's input was from outside the scope (there would be no initial capability locally within the subgraph to present at it). The observed behavior was a spurious empty frontier in the first call, followed by legit frontiers after that call.

In fact, this manifests in the doctest for enterleave::enter_at with enough panicky double-checking code (and indeed that example has an operator that consumes directly from outside the scope).

This fix is very local, but another take is that set_external_summary could plausibly be deleted, as the surrounding scope no longer presents a summary downwards, and initial frontiers are presented in the initial population of the SharedProgress struct rather than by method call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant