Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref: Buffer envelopes for broken project states #1856

Merged
merged 17 commits into from
Feb 22, 2023

Conversation

olksdr
Copy link
Contributor

@olksdr olksdr commented Feb 16, 2023

Check request only for the valid project configs, otherwise we reject incoming events immediately with invalid data (project_state), since there is nothing to check against.

With this change we will still keep buffering even if the resolved project state is invalid. After the state is updated, or pending envelopes will be processed again.

#skip-changelog

fix: #1787

@olksdr olksdr requested a review from a team February 16, 2023 10:56
@olksdr olksdr self-assigned this Feb 16, 2023
Copy link
Member

@jjbayer jjbayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jan-auer what is the reason we treat invalid states different than missing states? To me it seems that in both cases, we want to buffer events and retry to fetch the config. So maybe we can get rid of the concept of an invalid state altogether (except maybe for logging purposes)?

Copy link
Member

@jan-auer jan-auer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed again together with @jjbayer:

  1. In update_state, we never re-trigger fetching of states if there's an invalid state. This is now a problem, because we want to keep envelopes queued. This will require a change to update_state.
  2. In enqueue_validation, we should skip states that are invalid
  3. You change in check_envelope covers the fast-path. Potentially we might want to catch this in handle_check_envelope though?

We explicitly take the risk of overflowing buffers now, which will be addressed through persistent queueing.

// Intentionally ignore all errors. Fallback sampling behavior applies in this case.
if let Some(state) = self.valid_state() {
if self.valid_state().is_some() {
Copy link
Member

@jan-auer jan-auer Feb 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check is redundant now, since get_cached_state already checks for expiry. We could document on flush_sampling that this should not be called with an expired state.

Thinking through this again, I suggested a wrong change earlier. As the comment above states, we want to skip invalid and missing project states for this assignment. This is important because the public key from the DSC can point to a project that does not exist or is unavailable in the same Sentry instance. In this case, fallback sampling behavior must apply.

Copy link
Member

@jan-auer jan-auer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@olksdr olksdr merged commit 966edb3 into master Feb 22, 2023
@olksdr olksdr deleted the ref/keep-envelopes-for-invalid-configs branch February 22, 2023 10:06
jan-auer added a commit that referenced this pull request Feb 23, 2023
* master:
  feat(metrics): Tag the sample decision on count_per_root_project (#1870)
  feat(protocol): Add sourcemap debug image type to protocol (#1869)
  ref(statsd): Revert back the adition of metric names as tag on Sentry errors (#1873)
  feat(profiling): Add PHP support (#1871)
  fix(panic): revert sentry-types to 0.20.1 (#1872)
  ref(server): Use async/await in all endpoints (#1862)
  ref: Buffer envelopes for broken project states (#1856)
  meta: Remove accidentally added GeoIP file (#1866)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Buffer envelopes after resolving broken project states
3 participants