Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing repeat for PooledArray #67

Merged
merged 9 commits into from
May 17, 2021

Conversation

sl-solution
Copy link
Contributor

the repeat function for PooledArray

before

xx = PooledArray(["1","2"])
@btime repeat(xx, inner = 10^6)
  43.819 ms (12 allocations: 7.63 MiB)

after

@btime repeat(xx, inner = 10^6)
  3.078 ms (11 allocations: 7.63 MiB)

This makes the repeat of a PooledArray more efficient.
@@ -601,4 +601,7 @@ _perm(o::F, z::V) where {F, V} = Base.Order.Perm{F, V}(o, z)

Base.Order.Perm(o::Base.Order.ForwardOrdering, y::PooledArray) = _perm(o, fast_sortable(y))

Base.repeat(x::PooledArray, counts...) = PooledArray(RefArray(repeat(x.refs, counts...)), copy(x.invpool))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copying of pool and invpool is not needed and should be avoided. Please check the implementation of other operations to see how copy-on-write is implemented. Thank you!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also could you please add some tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for comments. Updated the function and add some tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah - now I see. Such ambiguities are super annoying (and that is why having a detailed test coverage is crucial 😄). Thank you!

Copy link
Member

@bkamins bkamins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Let us wait for the CI to pass and other reviewers.

@codecov
Copy link

codecov bot commented May 8, 2021

Codecov Report

Merging #67 (5600e9b) into main (080771a) will increase coverage by 0.25%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #67      +/-   ##
==========================================
+ Coverage   88.10%   88.36%   +0.25%     
==========================================
  Files           1        1              
  Lines         269      275       +6     
==========================================
+ Hits          237      243       +6     
  Misses         32       32              
Impacted Files Coverage Δ
src/PooledArrays.jl 88.36% <100.00%> (+0.25%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 080771a...5600e9b. Read the comment docs.

@@ -601,4 +601,14 @@ _perm(o::F, z::V) where {F, V} = Base.Order.Perm{F, V}(o, z)

Base.Order.Perm(o::Base.Order.ForwardOrdering, y::PooledArray) = _perm(o, fast_sortable(y))

function Base.repeat(x::PooledArray{T, R, N, RA}, counts...) where {T, R<:Integer, N, RA}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would assume that the {T, R, N, RA} part is not needed. Why have you added it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based on CI output, it seems it is needed for Julia 1.0.5?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the problem should be solved in another way. I updated it.

src/PooledArrays.jl Show resolved Hide resolved
test/runtests.jl Outdated Show resolved Hide resolved
test/runtests.jl Outdated
Comment on lines 478 to 481
pa1 = PooledArray(["one", "two"])
# check it didn't mess the other pa
@test pa3 == [1, 2, 3, 1, 2, 3]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't check much AFAICT: no significant operation has been performed since the last check.

Suggested change
pa1 = PooledArray(["one", "two"])
# check it didn't mess the other pa
@test pa3 == [1, 2, 3, 1, 2, 3]
pa1 = PooledArray(["one", "two"])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I remove it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's what the suggestion does?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I am lost. What is your suggestion on lines 478-480?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why GitHub is confused about it, but I just suggested dropping lines 479-481.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see,
GitHub doesn't allow to commit suggestion (the suggestion is invalid...)!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

sl-solution and others added 4 commits May 17, 2021 07:45
Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>
Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>
@nalimilan nalimilan merged commit 35ecfd1 into JuliaData:main May 17, 2021
@nalimilan
Copy link
Member

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants