Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: redesign source and sinks to store features #57

Merged
merged 3 commits into from
Feb 10, 2023

Conversation

sighphyre
Copy link
Member

This redesigns the way we store data in the memory and Redis providers to store incoming features by environment rather the token string. This means that incoming features for an environment will be merged in the case that the come from the same environment but different tokens. To handle this in the sources, outgoing data is filtered by project

Copy link
Member

@nunogois nunogois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

.environment
.as_ref()
.map(|environment| format!("{FEATURE_PREFIX}{environment}"))
.expect("Tying to resolve features for a token that hasn't been validated")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.expect("Tying to resolve features for a token that hasn't been validated")
.expect("Resolving features for a validated token")

Just words, but when I read the expect (might be because I'm unfamiliar with Rust yet), it seems that you're expecting a non validated token...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's fair. Expect should loosely read as "I expected this value and if I didn't get it, crash with the enclosed error message"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also agree that expect is a bit confusing readability wise. It's almost like it should be a except instead.

Copy link
Contributor

@gastonfournier gastonfournier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine, just some comments

let features_to_store =
if let Some(stored_features) = con.get::<&str, Option<String>>(key.as_str()).await? {
let stored_features = serde_json::from_str::<ClientFeatures>(&stored_features)?;
stored_features.merge(features.clone())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a problem for later, but this could be an always-growing collection. We won't remove data and the TTL might get updated so stored_features will never expire...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point! Yes, we should fix this but probably in a different PR

Comment on lines +16 to +25
self.iter()
.filter(|feature| {
if let Some(feature_project) = &feature.project {
token.projects.contains(&"*".to_string())
|| token.projects.contains(feature_project)
} else {
false
}
})
.cloned()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a big deal, but this code is pretty similar to the one in server/src/data_sources/redis_provider.rs, maybe something to extract

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, basically the same code. It'll be the same code for every source. Nice target for a trait impl in future

Comment on lines +23 to +29
fn build_features_key(token: &EdgeToken) -> String {
token
.environment
.as_ref()
.map(|environment| format!("{FEATURE_PREFIX}{environment}"))
.expect("Tying to resolve features for a token that hasn't been validated")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use the function from redis_provider here? Maybe it's a thing about function scopes and visibility that makes it difficult to do that

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I intentionally didn't do that because I don't want to expose that function. I figure it's okay because if you'll break these tests hard if the backing implementation changes. Don't mind a bit of duplication in tests to preserve sanity in the public API

@sighphyre sighphyre merged commit 869294b into main Feb 10, 2023
@sighphyre sighphyre deleted the chore/data-sources-and-sinks-for-features branch February 10, 2023 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants