Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iterutils: optimize chunked_iter #79

Merged
merged 1 commit into from
Jun 4, 2016

Conversation

immerrr
Copy link
Contributor

@immerrr immerrr commented Jun 3, 2016

Hello and thank you for such a nice library.

This PR optimizes iterutils.chunked_iter performance (2x on pypy, 5x on cpython).

Before

immerrr@mmrcomp:~/sources/boltons$ time ~/pypy-5.1.1-linux64/bin/pypy bench_chunked_iter.py

real    0m1.862s
user    0m1.824s
sys     0m0.020s
immerrr@mmrcomp:~/sources/boltons$ time /usr/bin/python2.7 bench_chunked_iter.py

real    0m12.275s
user    0m12.256s
sys     0m0.004s
immerrr@mmrcomp:~/sources/boltons$ time /usr/bin/python3.5 bench_chunked_iter.py

real    0m17.307s
user    0m17.288s
sys     0m0.008s

After

immerrr@mmrcomp:~/sources/boltons$ time ~/pypy-5.1.1-linux64/bin/pypy bench_chunked_iter.py

real    0m0.926s
user    0m0.868s
sys     0m0.044s
immerrr@mmrcomp:~/sources/boltons$ time /usr/bin/python2.7 bench_chunked_iter.py

real    0m2.361s
user    0m2.340s
sys     0m0.008s
immerrr@mmrcomp:~/sources/boltons$ time /usr/bin/python3.5 bench_chunked_iter.py

real    0m3.007s
user    0m2.988s
sys     0m0.008s

With bench_chunked_iter.py being:

from boltons.iterutils import chunked_iter
from collections import deque

try:
    xrange
except NameError:
    xrange = range


for i in xrange(100):
    deque(chunked_iter(xrange(1000000), 100), 0)

@mahmoud
Copy link
Owner

mahmoud commented Jun 4, 2016

Hard to argue with numbers like that, and shorter code! Thanks!

@mahmoud mahmoud merged commit 5dcdd4d into mahmoud:master Jun 4, 2016
@immerrr immerrr deleted the optimize-chunked-iter branch June 6, 2016 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants