Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set all external Snowflake tables explicitly to auto_refresh = true. #561

Closed
j3-signalroom opened this issue Dec 16, 2024 · 0 comments · Fixed by #562
Closed

Set all external Snowflake tables explicitly to auto_refresh = true. #561

j3-signalroom opened this issue Dec 16, 2024 · 0 comments · Fixed by #562
Assignees
Labels
refactor 🏗️ Refactor code or work product

Comments

@j3-signalroom
Copy link
Owner

j3-signalroom commented Dec 16, 2024

AUTO_REFRESH = TRUE | FALSE
Specifies whether Snowflake should enable triggering automatic refreshes of the external table metadata when new or updated data files are available in the named external stage specified in the [ WITH ] LOCATION = setting.

Note

Setting this parameter to TRUE is not supported by partitioned external tables when partitions are added manually by the object owner (i.e. when PARTITION_TYPE = USER_SPECIFIED).

Setting this parameter to TRUE is not supported for external tables that reference data files stored on an S3-compatible external stage. You must manually refresh the metadata by executing an ALTER EXTERNAL TABLE … REFRESH command.

You must configure an event notification for your storage location to notify Snowflake when new or updated data is available to read into the external table metadata. For more information, see the instructions for your cloud storage service:

Amazon S3
Refreshing external tables automatically for Amazon S3

Google Cloud Storage
Refreshing external tables automatically for Google Cloud Storage

Microsoft Azure
Refreshing external tables automatically for Azure Blob Storage

When an external table is created, its metadata is refreshed automatically once unless REFRESH_ON_CREATE = FALSE.

TRUE
Snowflake enables triggering automatic refreshes of the external table metadata.

FALSE
Snowflake does not enable triggering automatic refreshes of the external table metadata. You must manually refresh the external table metadata periodically using ALTER EXTERNAL TABLE … REFRESH to synchronize the metadata with the current list of files in the stage path.

Default: TRUE

@j3-signalroom j3-signalroom added the refactor 🏗️ Refactor code or work product label Dec 16, 2024
@j3-signalroom j3-signalroom self-assigned this Dec 16, 2024
@j3-signalroom j3-signalroom moved this to In Progress in J3 Public Projects Dec 16, 2024
@j3-signalroom j3-signalroom linked a pull request Dec 16, 2024 that will close this issue
@github-project-automation github-project-automation bot moved this from In Progress to Done in J3 Public Projects Dec 16, 2024
@j3-signalroom j3-signalroom changed the title Set all external Snowflake tables to auto_refresh = true. Set all external Snowflake tables explicitly to auto_refresh = true. Dec 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactor 🏗️ Refactor code or work product
Projects
Development

Successfully merging a pull request may close this issue.

1 participant