Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Memory Efficiency of process_results() Override #1053

Closed
3 tasks done
peterallenwebb opened this issue May 17, 2024 · 0 comments · Fixed by #1050
Closed
3 tasks done

Improve Memory Efficiency of process_results() Override #1053

peterallenwebb opened this issue May 17, 2024 · 0 comments · Fixed by #1050
Assignees

Comments

@peterallenwebb
Copy link
Contributor

peterallenwebb commented May 17, 2024

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt-snowflake functionality, rather than a Big Idea better suited to a discussion

Describe the feature

The current implementation of dbt-snowflake's process_results() override holds a full additional copy of query results in memory. For queries with large result sets, such as those used by get_catalog(), this can limit dbt's ability to operate in memory-constrained environments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant