-
Notifications
You must be signed in to change notification settings - Fork 77
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Statsig access and erasure #4429
Merged
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
ea14311
Added Statsig Connector
MarcGEthyca b238a9a
statsig initial
MarcGEthyca 14fb1d2
Adding new config and dataset for Statsig Conn
MarcGEthyca 81480a4
Trying without client config for update
MarcGEthyca 748ed2c
Modifications for erasure connector for Statsig
MarcGEthyca 7295587
added comment for future development
MarcGEthyca 366220c
Passing connection test
MarcGEthyca 82afaa7
Renaming Statsig to Statsig Enterprise - also fixed a doc link for sl…
MarcGEthyca 8942bf6
Fixing up renaming mess
MarcGEthyca 7511b53
Removing old files
MarcGEthyca f6a3144
Merge branch 'main' into MG-con-203
galvana 8a29881
Changelog
MarcGEthyca b17d339
Merge branch 'main' into MG-con-203
galvana 17664c8
Adding request override
galvana 0da08d5
Corrected Changelog
MarcGEthyca d371bcd
Updating change log
galvana cced452
Merge branch 'main' into MG-con-203
galvana File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
saas_config: | ||
fides_key: <instance_fides_key> | ||
name: Statsig Enterprise | ||
type: statsig_enterprise | ||
description: A sample schema representing the Statsig connector for Fides | ||
user_guide: https://docs.ethyca.com/user-guides/integrations/saas-integrations/statsig-enterprise | ||
version: 0.1.0 | ||
|
||
connector_params: | ||
- name: domain | ||
label: Domain | ||
default_value: api.statsig.com | ||
description: Your Statsig URL | ||
- name: server_secret_key | ||
label: Server secret key | ||
description: The server secret key available at console.statsig.com/api_keys | ||
sensitive: True | ||
|
||
external_references: | ||
- name: statsig_user_id | ||
label: Statsig user ID field | ||
description: Dataset reference to the location of Statsig user IDs | ||
|
||
client_config: | ||
protocol: https | ||
host: <domain> | ||
authentication: | ||
strategy: api_key | ||
configuration: | ||
headers: | ||
- name: STATSIG-API-KEY | ||
value: <server_secret_key> | ||
|
||
test_request: | ||
method: POST | ||
path: /v1/log_event | ||
body: | | ||
{ | ||
"events": [ | ||
{ | ||
"eventName": "fides_test_connection" | ||
} | ||
] | ||
} | ||
|
||
endpoints: | ||
- name: user | ||
requests: | ||
read: | ||
request_override: statsig_enterprise_user_read | ||
param_values: | ||
- name: user_id | ||
references: | ||
- statsig_user_id | ||
delete: | ||
method: POST | ||
path: /v1/delete_user_data | ||
body: | | ||
{ | ||
"unit_type": "user_id", | ||
"ids": "<user_id>", | ||
"request_id": "<privacy_request_id>" | ||
} | ||
param_values: | ||
- name: user_id | ||
references: | ||
- dataset: <instance_fides_key> | ||
field: user.id | ||
direction: from |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
dataset: | ||
- fides_key: <instance_fides_key> | ||
name: Statsig Enterprise Dataset | ||
description: A sample dataset representing the Statsig Enterprise connector for Fides | ||
collections: | ||
- name: user | ||
fields: | ||
- name: id | ||
data_categories: [system.operations] | ||
fidesops_meta: | ||
primary_key: True | ||
data_type: string |
33 changes: 33 additions & 0 deletions
33
...api/service/saas_request/override_implementations/statsig_enterprise_request_overrides.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
from typing import Any, Dict, List | ||
|
||
from fides.api.graph.traversal import TraversalNode | ||
from fides.api.models.policy import Policy | ||
from fides.api.models.privacy_request import PrivacyRequest | ||
from fides.api.service.connectors.saas.authenticated_client import AuthenticatedClient | ||
from fides.api.service.saas_request.saas_request_override_factory import ( | ||
SaaSRequestType, | ||
register, | ||
) | ||
from fides.api.util.collection_util import Row | ||
|
||
|
||
@register("statsig_enterprise_user_read", [SaaSRequestType.READ]) | ||
def statsig_enterprise_user_read( | ||
client: AuthenticatedClient, | ||
node: TraversalNode, | ||
policy: Policy, | ||
privacy_request: PrivacyRequest, | ||
input_data: Dict[str, List[Any]], | ||
secrets: Dict[str, Any], | ||
) -> List[Row]: | ||
""" | ||
Convert the statsig_user_ids from the input_data into rows. We do this because erasure | ||
requests only receive input data that the access request received, or data returned from | ||
the access request. Erasure requests can't specify data in other datasets as dependencies. | ||
""" | ||
|
||
statsig_user_ids = input_data.get("user_id", []) | ||
results = [] | ||
for statsig_user_id in statsig_user_ids: | ||
results.append({"id": statsig_user_id}) | ||
return results |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
from typing import Any, Dict | ||
|
||
import pydash | ||
import pytest | ||
|
||
from tests.ops.integration_tests.saas.connector_runner import ( | ||
ConnectorRunner, | ||
generate_random_email, | ||
) | ||
from tests.ops.test_helpers.vault_client import get_secrets | ||
|
||
secrets = get_secrets("statsig_enterprise") | ||
|
||
|
||
@pytest.fixture(scope="session") | ||
def statsig_enterprise_secrets(saas_config) -> Dict[str, Any]: | ||
return { | ||
"domain": pydash.get(saas_config, "statsig_enterprise.domain") | ||
or secrets["domain"], | ||
"server_secret_key": pydash.get( | ||
saas_config, "statsig_enterprise.server_secret_key" | ||
) | ||
or secrets["server_secret_key"], | ||
} | ||
|
||
|
||
@pytest.fixture | ||
def statsig_enterprise_erasure_identity_email() -> str: | ||
return generate_random_email() | ||
|
||
|
||
@pytest.fixture | ||
def statsig_enterprise_erasure_external_references() -> Dict[str, Any]: | ||
return {"statsig_user_id": "123"} | ||
|
||
|
||
@pytest.fixture | ||
def statsig_enterprise_runner( | ||
db, | ||
cache, | ||
statsig_enterprise_secrets, | ||
statsig_enterprise_erasure_external_references, | ||
) -> ConnectorRunner: | ||
return ConnectorRunner( | ||
db, | ||
cache, | ||
"statsig_enterprise", | ||
statsig_enterprise_secrets, | ||
erasure_external_references=statsig_enterprise_erasure_external_references, | ||
) |
28 changes: 28 additions & 0 deletions
28
tests/ops/integration_tests/saas/test_statsig_enterprise_task.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
import pytest | ||
|
||
from fides.api.models.policy import Policy | ||
from tests.ops.integration_tests.saas.connector_runner import ConnectorRunner | ||
|
||
|
||
@pytest.mark.integration_saas | ||
class TestStatsigEnterpriseConnector: | ||
def test_connection(self, statsig_enterprise_runner: ConnectorRunner): | ||
statsig_enterprise_runner.test_connection() | ||
|
||
@pytest.mark.skip("Enterprise account only") | ||
async def test_non_strict_erasure_request( | ||
self, | ||
statsig_enterprise_runner: ConnectorRunner, | ||
policy: Policy, | ||
erasure_policy_string_rewrite: Policy, | ||
statsig_enterprise_erasure_identity_email: str, | ||
): | ||
( | ||
_, | ||
erasure_results, | ||
) = await statsig_enterprise_runner.non_strict_erasure_request( | ||
access_policy=policy, | ||
erasure_policy=erasure_policy_string_rewrite, | ||
identities={"email": statsig_enterprise_erasure_identity_email}, | ||
) | ||
assert erasure_results == {"statsig_enterprise_instance:user": 1} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I happened to notice this user guide link was pointing to square instead of slack-enterprise.