-
Notifications
You must be signed in to change notification settings - Fork 998
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Be less conservative about ALTREP #4697
Comments
Should we also be more aggressive about using R 3.5.0 is 2.5 years old so I think we're getting close to where incorporating it makes sense. |
Great talk from Gabe Becker here: I think a low hanging fruit could be to set (first) keys to KNOWN_INCREASING, that could e.g. let R by very fast in doing |
Unless there are NAs which R places at the end and not at the front. |
Actually no! one of the sortedness enum fit our case if I'm not mistaken: link to slides here: |
Well if we manage to support altrep vectors as columns then out-of-memory data.table should be easier targetable. |
is there a more convincing real example? |
out-of-memory data.table is very desired feature due to severity of consequences when you just don't have enough memory. Together with long vector support these were two main points on the roadmap for data.table new features that Matt was working on. As he said, long vector support is not that great deal as long as there is no out-of-memory data.table. |
Discovered via SO Q&A:
https://stackoverflow.com/a/63770450/3576984
Currently
j
expands allALTREP
s, which can lead to significant performance degradation:I don't know enough about
ALTREP
to say for sure which cases we can/can't avoid expansion, but hopefully we can do better than the above.(and actually I'm not 100% sure
ALTREP
is to blame here, but seems the likeliest candidate)The text was updated successfully, but these errors were encountered: