Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance improvements using iterator aggregates #233

Merged
merged 1 commit into from
Feb 3, 2022

Conversation

drupol
Copy link
Collaborator

@drupol drupol commented Jan 23, 2022

This PR:

  • Replaces IterableIterator with IterableIteratorAggregate (4 occurences)
  • Changes the signature of a couple of operations.
  • Update all the annotations where an Iterator is needed while now an iterable is enough.
  • Use yield from - align everything
  • Get rid of ClosureIterator and replace it with ClosureIteratorAggregate

Based on the benchmarks I did in loophp/iterators, it seems to be definitely, and it make sense.
IteratorAggregates are usually faster because the state of the Iterator is not built in userland but directly handled by PHP internaly.

$ ./vendor/bin/phpbench run --profile=local --group=local --report=aggregate --stop-on-error tests/benchmarks/IterableIteratorAggregateBench.php tests/benchmarks/IterableIteratorBench.php
PHPBench (1.2.3) running benchmarks...
with configuration file: /home/pol/dev/git/iterators/phpbench.json
with PHP version 8.1.1, xdebug ❌, opcache ✔

\benchmarks\loophp\iterators\IterableIteratorAggregateBench

    bench # IterableIteratorAggregate.......I0 - Mo1.963ms (±0.00%)
    bench # IterableIteratorAggregate.......R1 I7 - Mo0.937ms (±1.97%)
    bench # IterableIteratorAggregate.......R1 I25 - Mo0.927ms (±1.60%)

\benchmarks\loophp\iterators\IterableIteratorBench

    bench # IterableIterator................R1 I0 - Mo2.188ms (±0.00%)
    bench # IterableIterator................R1 I6 - Mo1.765ms (±1.80%)
    bench # IterableIterator................R4 I49 - Mo1.741ms (±2.25%)

Subjects: 2, Assertions: 0, Failures: 0, Errors: 0
+--------------------------------+---------+---------------------------+------+-----+----------+---------+--------+
| benchmark                      | subject | set                       | revs | its | mem_peak | mode    | rstdev |
+--------------------------------+---------+---------------------------+------+-----+----------+---------+--------+
| IterableIteratorAggregateBench | bench   | IterableIteratorAggregate | 10   | 1   | 3.695mb  | 1.963ms | ±0.00% |
| IterableIteratorAggregateBench | bench   | IterableIteratorAggregate | 10   | 10  | 3.695mb  | 0.937ms | ±1.97% |
| IterableIteratorAggregateBench | bench   | IterableIteratorAggregate | 10   | 50  | 3.695mb  | 0.927ms | ±1.60% |
| IterableIteratorBench          | bench   | IterableIterator          | 10   | 1   | 3.695mb  | 2.188ms | ±0.00% |
| IterableIteratorBench          | bench   | IterableIterator          | 10   | 10  | 3.695mb  | 1.765ms | ±1.80% |
| IterableIteratorBench          | bench   | IterableIterator          | 10   | 50  | 3.695mb  | 1.741ms | ±2.25% |
+--------------------------------+---------+---------------------------+------+-----+----------+---------+--------+

@AlexandruGG WDYT of this? Are you ok with this?

@drupol drupol added the enhancement New feature or request label Jan 23, 2022
@drupol drupol self-assigned this Jan 23, 2022
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch 4 times, most recently from a55efd4 to 4b9d0e7 Compare January 23, 2022 19:32
@drupol drupol changed the title WIP - Performance improvements using iterator aggregates. Performance improvements using iterator aggregates Jan 23, 2022
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch 6 times, most recently from 6fffeaf to 6f89a4e Compare January 25, 2022 21:52
@AlexandruGG
Copy link
Collaborator

Looks good at a first glance, let me know when it's ready for review and I'll go through it more in-depth 👍

@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch 2 times, most recently from 626c023 to 0b06339 Compare January 26, 2022 18:57
@drupol
Copy link
Collaborator Author

drupol commented Jan 26, 2022

I think it's done.

This is something I should have done a long time ago.

@drupol drupol removed their assignment Jan 26, 2022
@drupol drupol requested a review from AlexandruGG January 26, 2022 18:59
@drupol drupol marked this pull request as ready for review January 26, 2022 18:59
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch from 0b06339 to 192734d Compare January 26, 2022 19:00
@drupol drupol enabled auto-merge (squash) January 26, 2022 19:13
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch from 192734d to 68ffdee Compare January 28, 2022 09:35
auto-merge was automatically disabled January 28, 2022 09:35

Pull Request is not mergeable

@drupol drupol enabled auto-merge (squash) January 28, 2022 09:35
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch from 68ffdee to d623bc4 Compare January 28, 2022 18:26
auto-merge was automatically disabled January 28, 2022 18:26

Pull Request is not mergeable

@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch from d623bc4 to 1fe15b5 Compare January 28, 2022 20:21
@drupol drupol enabled auto-merge (squash) January 28, 2022 20:25
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch from 1fe15b5 to 5c431b1 Compare January 29, 2022 20:29
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch from 3346313 to e72f268 Compare January 30, 2022 11:14
@drupol drupol marked this pull request as draft January 30, 2022 11:17
auto-merge was automatically disabled January 30, 2022 11:17

Pull request was converted to draft

@drupol drupol self-assigned this Jan 30, 2022
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch from e72f268 to bee8a2a Compare January 30, 2022 11:26
@drupol drupol marked this pull request as ready for review January 30, 2022 11:26
@drupol drupol removed their assignment Jan 30, 2022
@drupol drupol enabled auto-merge (squash) January 30, 2022 11:26
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch from bee8a2a to 1be166e Compare January 30, 2022 12:00
@drupol drupol disabled auto-merge January 30, 2022 12:44
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch from 1be166e to e2b5dcc Compare January 30, 2022 15:17
@drupol drupol enabled auto-merge (squash) January 30, 2022 15:20
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch 7 times, most recently from d775074 to 36f1062 Compare January 30, 2022 21:30
composer.json Outdated Show resolved Hide resolved
src/Contract/Operation/Duplicateable.php Outdated Show resolved Hide resolved
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch 3 times, most recently from 25ed276 to 83405dc Compare January 31, 2022 13:27
@drupol drupol force-pushed the optimize-by-using-iteratoraggregate branch from 83405dc to a07f7e2 Compare January 31, 2022 18:19
private $source;

/**
* @psalm-external-mutation-free
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is removing this intentional?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think we should even get rid of them everywhere. psalm doesn't handle them correctly anyway. I really wanted to avoid updating the baselines too much.

}

/**
* @pure
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed some of these @pure annotations are being removed, but not everywhere. I think just in src/Collection.php?

How come this is needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as the previous one. We are not the only ones... https://github.com/marcosh/lamphpda/blob/master/adr/2022-01-25-pure-callable.md

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as the previous one. We are not the only ones... https://github.com/marcosh/lamphpda/blob/master/adr/2022-01-25-pure-callable.md

I get this one, but in this project we didn't go that far to use pure-callable types 😬 .

I think the errors you saw in this project might be related to the iterators project not having things marked as @pure.

In any case it might be too much of a hassle to maintain this kind of stuff in PHP, in which case I think it's fine to remove them but we can do it for the whole codebase. You can do it in a separate PR though

@drupol drupol merged commit a69acec into master Feb 3, 2022
@drupol drupol deleted the optimize-by-using-iteratoraggregate branch February 3, 2022 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants