feat: scan_csv
#1555
Merged
feat: scan_csv
#1555
+185
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
narwhals/functions.py
Outdated
Comment on lines
948
to
949
| This allows the query optimizer to push down predicates and projections | ||
| to the scan level, thereby potentially reducing memory overhead. |
There was a problem hiding this comment.
this is Polars-specific, perhaps we can remove it?
narwhals/functions.py
Outdated
| ... return ( | ||
| ... nw.scan_csv("file.csv", native_namespace=native_namespace) | ||
| ... .to_native() | ||
| ... .collect() |
There was a problem hiding this comment.
collect isn't guaranteed to be present on a native frame, can we keep it out of agnostic_scan_csv? you can just put collect as the end of the Polars example, and compute at the end of the dask example
either that, or put collect before to_native
narwhals/stable/v1/__init__.py
Outdated
Comment on lines
3396
to
3397
| This allows the query optimizer to push down predicates and projections | ||
| to the scan level, thereby potentially reducing memory overhead. |
narwhals/stable/v1/__init__.py
Outdated
| ... .collect() | ||
| ... ) | ||
|
|
||
| Then we can read the file by passing Polars or dask namespaces: |
There was a problem hiding this comment.
how about
by passing, for example, Polars or Dask namespaces
?
MarcoGorelli
approved these changes
Dec 10, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this? (check all applicable)
Related issues
Checklist
If you have comments or can explain your changes, please do so below