Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pinot connector: by default turn on push down top n queries #17582

Conversation

xiangfu0
Copy link
Contributor

@xiangfu0 xiangfu0 commented Apr 5, 2022

This config was off as Pinot version < 0.4 doesn't support aggregation group by order by, so the order by can not be pushdown. We need to retrieve many groups then let presto order it.

Since Pinot supports ORDER BY now, this clause can now be pushed down by default.

For example, for presto query
SELECT Origin, sum(AirTime) FROM airlineStats GROUP BY Origin ORDER BY Origin LIMIT 10
the generated Pinot queries to push down are:

When the config is turned off (pinot.pushdown-topn-broker-queries=false):

GeneratedPinotQuery=SELECT Origin, sum(AirTime) FROM airlineStats GROUP BY Origin LIMIT 1000

After the config is turned on (pinot.pushdown-topn-broker-queries=true):

GeneratedPinotQuery=SELECT Origin, sum(AirTime) FROM airlineStats GROUP BY Origin ORDER BY Origin LIMIT 10
== RELEASE NOTES ==

Pinot Changes
* Turn on config `pinot.pushdown-topn-broker-queries` by default to push down aggregation group by order by queries to improve query performance.

@xiangfu0 xiangfu0 force-pushed the pinot-connector-turn-on-default-pushdown-topn-broker-queries branch 2 times, most recently from 79df9e0 to ec68721 Compare April 6, 2022 00:53
@xiangfu0 xiangfu0 force-pushed the pinot-connector-turn-on-default-pushdown-topn-broker-queries branch from ec68721 to d63fb49 Compare April 6, 2022 01:04
@xiangfu0
Copy link
Contributor Author

xiangfu0 commented Apr 6, 2022

@highker

@highker highker merged commit b87e28f into prestodb:master Apr 6, 2022
@xiangfu0 xiangfu0 deleted the pinot-connector-turn-on-default-pushdown-topn-broker-queries branch April 6, 2022 03:51
@mshang816 mshang816 mentioned this pull request May 17, 2022
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants