Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize const grouping keys #17642

Merged
merged 1 commit into from
Nov 4, 2022

Conversation

mlyublena
Copy link
Contributor

Move constant grouping keys into a Project node above the group by, for example:

SELECT userid, 'constant val' AS extra, COUNT(1) as cnt from T GROUP BY 1, 2

into

SELECT userid,  'constant val' as extra, cnt from (SELECT userid, AS extra, COUNT(1) from T GROUP BY 1) s

Test plan

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Apr 13, 2022

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: mlyublena (612ae0390b34baf1fa04f7edb026ec85e58b84c7)

Copy link
Contributor

@kaikalur kaikalur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also squash the commits

@@ -899,6 +900,11 @@ public SystemSessionProperties(
"push limits to the outer side of an outer join",
featuresConfig.isPushLimitThroughOuterJoin(),
false),
booleanProperty(
OPTIMIZE_CONSTANT_GROUPING_KEYS,
"push limits to the outer side of an outer join",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix wording

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, will do, copy-paste error

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


Set<Map.Entry<VariableReferenceExpression, RowExpression>> childAssighnments = child.getAssignments().entrySet();
for (Map.Entry<VariableReferenceExpression, RowExpression> entry : childAssighnments) {
if (isConstant(entry.getValue()) && outputVariables.contains(entry.getKey())) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we have a isConst method in ExpressionUtils?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, thank you!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done: the utility function expects Expression, I added a wrapper below


private static final Pattern<AggregationNode> PATTERN =
aggregation()
.with(source().matching(project().capturedAs(CHILD)));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaikalur is there a way to check if a variable reference is a constant without hard-coding the type of child node? that would miss cases when the constant is introduced further down in the tree, or is the result of a filter. Not sure if we support that type of property/constraint derivation?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are always rewriting the plans through all the optimizations so maybe just doing this at the end (just before exchanges) will do it. And I think this is the way to capture the children.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the way to do that would be to have additional "pull up constant" rules for other node types. We can run all of them until the constant is as far up in the plan as it goes.

@highker highker requested a review from NikhilCollooru April 15, 2022 02:32
@mlyublena mlyublena force-pushed the optimize-const-grouping-keys branch from 138a040 to a0f280f Compare April 18, 2022 16:51
@Test
public void testGroupByWithConstants()
{
assertQuery("SELECT * FROM (SELECT regionkey, col, count(*) FROM (SELECT regionkey, 'bla' as col from nation) group by regionkey, col)");
Copy link
Contributor

@kaikalur kaikalur Apr 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to add the expected one as well - for example:

select 'blah', * from (select regiokey, count(*) from nation group by regionkey)

So it is more obvious.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

private static boolean isConstant(RowExpression expr)
{
if (isExpression(expr)) {
return ExpressionTreeUtils.isConstant(castToExpression(expr));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

static import isConstant

Our style is to use static imports unless it would not be obvious without the fully qualified name

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


private static final Pattern<AggregationNode> PATTERN =
aggregation()
.with(source().matching(project().capturedAs(CHILD)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the way to do that would be to have additional "pull up constant" rules for other node types. We can run all of them until the constant is as far up in the plan as it goes.

@Override
public Result apply(AggregationNode parent, Captures captures, Context context)
{
if (!isOptimizeConstantGroupingKeys(context.getSession())) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead override the method "isEnabled"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -277,6 +278,12 @@ public PlanOptimizers(
.add(new PruneRedundantProjectionAssignments())
.build());

IterativeOptimizer groupByWithConstantsOptimizer = new IterativeOptimizer(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't need a special variable since it only adds one rule

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -395,6 +402,8 @@ public PlanOptimizers(
new TransformCorrelatedSingleRowSubqueryToProject())),
new CheckSubqueryNodesAreRewritten());

builder.add(groupByWithConstantsOptimizer);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would actually tag this on to some other block of rules. It doesn't need to be off by itself. Also, make sure inlineProjections gets called after this (InlineProjections and RemoveRedundantIdentityProject).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

assignments.putAll(remainingVars);
return Result.ofPlanNode(
new ProjectNode(
parent.getSourceLocation(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this spacing isn't correct. the arguments here should be indented another level

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Map<VariableReferenceExpression, RowExpression> remainingVars =
outputVariables.stream()
.filter(var -> !constChildVars.containsKey(var))
.collect(Collectors.toMap(Function.identity(), Function.identity()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

static import Function.identity()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

parent.getId(),
child,
parent.getAggregations(),
AggregationNode.singleGroupingSet(newGroupingKeys),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the pattern you also need to check that the aggregation only has a single grouping set. Then you should add a test that shows the optimization is not applied if there is more than one grouping set

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done + added a new test

extends BaseRuleTest
{
@Test
public void test_not_applicable()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all names should be camel case so testNotApplicable(). Also, add a more descriptive name (even if it's kind of long), as we want to also test other not applicable cases, and it should be obvious from looking at the test name exactly what is being tested e.g. all the not applicable tests could be testXXXDoesNotFier()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

import static com.facebook.presto.sql.planner.assertions.PlanMatchPattern.singleGroupingSet;
import static com.facebook.presto.sql.planner.assertions.PlanMatchPattern.values;

public class TestGroupByWithConstants
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some additional cases to test

  1. does not fire with the flag disabled
  2. multiple constants
  3. if only some constants aren't in the aggregation output, they don't get projected, but other constants do
  4. Multiple grouping sets

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added 1,2&4
having issues with plan matching for # 3 (let's discuss offline)

@mlyublena mlyublena force-pushed the optimize-const-grouping-keys branch from 9fac2a4 to 82ec363 Compare April 19, 2022 23:20
public class PullConstantsAboveGroupBy
implements Rule<AggregationNode>
{
private static final Capture<ProjectNode> CHILD = Capture.newCapture();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we use source of an aggregation, I think it will be good to use that same terminology here instead of child so it's easier to understand.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

AggregationNode newAgg = new AggregationNode(
parent.getSourceLocation(),
parent.getId(),
child,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And you should rewrite the child to drop the constant assignment and outputs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not always correct, the constant variable may be used as an argument to one of the agg functions, for example

select avg(x) from (select 1 as x) group by x;

I think pruning unused columns should be done separately as it may show up in other cases as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can call PruneUnreferencedOutputs after this optimizer runs

Copy link
Contributor

@rschlussel rschlussel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comments, but looks great

}

@Test
public void test()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

give this a descriptive name e.g. testSingleConstColumn

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

.matches(
project(
ImmutableMap.of(
"CONST_COL1", PlanMatchPattern.expression("1"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

static import expression

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@mlyublena mlyublena force-pushed the optimize-const-grouping-keys branch 2 times, most recently from d060d0b to 612ae03 Compare April 21, 2022 21:18
@rschlussel
Copy link
Contributor

@mlyublena you still need to sign the CLA

@rschlussel
Copy link
Contributor

Sorry, noticed the commit message title is too long and I don't think I have permission to update your branch myself. We follow https://cbea.ms/git-commit/ for our commit guidelines.

Can you change the commit title to "Move constant grouping keys above the group by" and put "For example" on the next line?

@mlyublena mlyublena force-pushed the optimize-const-grouping-keys branch from 612ae03 to f02c07e Compare May 9, 2022 18:00
@mlyublena mlyublena requested a review from a team as a code owner May 9, 2022 18:00
@mlyublena mlyublena requested a review from presto-oss May 9, 2022 18:00
Copy link
Contributor

@yingsu00 yingsu00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mlyublena Could you please check if the query SELECT userid, 'constant val' as extra, cnt from (SELECT userid, AS extra, COUNT(1) from T GROUP BY 1) s in the commit message is correct? Did you mean this?

SELECT userid, 'constant val' as extra, cnt from (SELECT userid,  COUNT(1) as cnt from t GROUP BY 1) s;

@mlyublena mlyublena requested a review from a team as a code owner November 4, 2022 18:16
@@ -0,0 +1,48 @@
/*
Copy link
Contributor Author

@mlyublena mlyublena Nov 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a benchmark test @kaikalur:

INFO: Without optimization
Nov 04, 2022 11:13:36 AM com.facebook.airlift.log.Logger info
INFO: -- Loading system access control --
Nov 04, 2022 11:13:36 AM com.facebook.airlift.log.Logger info
INFO: -- Loaded system access control allow-all --
Nov 04, 2022 11:13:36 AM com.facebook.airlift.log.Logger info
INFO: -- Loading temp storage local --
Nov 04, 2022 11:13:36 AM com.facebook.airlift.log.Logger info
INFO: -- Loaded temp storage local --
Nov 04, 2022 11:13:36 AM com.facebook.airlift.log.Logger info
INFO: -- Loading temp storage local --
Nov 04, 2022 11:13:36 AM com.facebook.airlift.log.Logger info
INFO: -- Loaded temp storage local --
peak_memory:1049052,elapsed_millis:71,input_rows_per_second:352,output_rows_per_second:70,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:70884750,cpu_nanos:63690000,user_nanos:59689000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:88,input_rows_per_second:282,output_rows_per_second:56,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:88427792,cpu_nanos:82240000,user_nanos:76153000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:73,input_rows_per_second:342,output_rows_per_second:68,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:73053500,cpu_nanos:66569000,user_nanos:61794000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:90,input_rows_per_second:277,output_rows_per_second:55,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:89986583,cpu_nanos:83379000,user_nanos:76933000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:81,input_rows_per_second:307,output_rows_per_second:61,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:81414083,cpu_nanos:72700000,user_nanos:68129000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:75,input_rows_per_second:335,output_rows_per_second:67,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:74596792,cpu_nanos:70166000,user_nanos:65303000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:73,input_rows_per_second:341,output_rows_per_second:68,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:73201583,cpu_nanos:67967000,user_nanos:62847000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:96,input_rows_per_second:261,output_rows_per_second:52,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:95575666,cpu_nanos:85599000,user_nanos:78474000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:151,input_rows_per_second:165,output_rows_per_second:33,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:151015041,cpu_nanos:135654000,user_nanos:123756000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:153,input_rows_per_second:163,output_rows_per_second:32,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:153300500,cpu_nanos:124190000,user_nanos:114321000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:150,input_rows_per_second:166,output_rows_per_second:33,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:149765583,cpu_nanos:139011000,user_nanos:127773000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:121,input_rows_per_second:206,output_rows_per_second:41,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:121312292,cpu_nanos:113491000,user_nanos:105332000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:58,input_rows_per_second:428,output_rows_per_second:85,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:58363583,cpu_nanos:53699000,user_nanos:50201000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:67,input_rows_per_second:371,output_rows_per_second:74,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:67276167,cpu_nanos:62683000,user_nanos:58266000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:59,input_rows_per_second:421,output_rows_per_second:84,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:59347209,cpu_nanos:55699000,user_nanos:51709000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:51,input_rows_per_second:493,output_rows_per_second:98,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:50684583,cpu_nanos:47907000,user_nanos:45072000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:53,input_rows_per_second:475,output_rows_per_second:95,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:52540792,cpu_nanos:48898000,user_nanos:45745000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:77,input_rows_per_second:326,output_rows_per_second:65,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:76682500,cpu_nanos:69061000,user_nanos:63892000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:57,input_rows_per_second:441,output_rows_per_second:88,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:56605834,cpu_nanos:52881000,user_nanos:49281000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1049052,elapsed_millis:54,input_rows_per_second:459,output_rows_per_second:91,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:54454666,cpu_nanos:51225000,user_nanos:47822000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
      pull_constants_above_group_by ::   77.335 cpu ms ::   1MB peak memory :: in    25,      0B,     323/s,      0B/s :: out     5,    130B,      64/s,  1.64KB/s
Nov 04, 2022 11:13:45 AM com.facebook.airlift.log.Logger info
INFO: With optimization
Nov 04, 2022 11:13:45 AM com.facebook.airlift.log.Logger info
INFO: -- Loading system access control --
Nov 04, 2022 11:13:45 AM com.facebook.airlift.log.Logger info
INFO: -- Loaded system access control allow-all --
Nov 04, 2022 11:13:45 AM com.facebook.airlift.log.Logger info
INFO: -- Loading temp storage local --
Nov 04, 2022 11:13:45 AM com.facebook.airlift.log.Logger info
INFO: -- Loaded temp storage local --
Nov 04, 2022 11:13:45 AM com.facebook.airlift.log.Logger info
INFO: -- Loading temp storage local --
Nov 04, 2022 11:13:45 AM com.facebook.airlift.log.Logger info
INFO: -- Loaded temp storage local --
peak_memory:1495616,elapsed_millis:37,input_rows_per_second:684,output_rows_per_second:136,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:36547500,cpu_nanos:35853000,user_nanos:30955000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:42,input_rows_per_second:595,output_rows_per_second:119,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:42003750,cpu_nanos:40892000,user_nanos:34889000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:59,input_rows_per_second:421,output_rows_per_second:84,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:59294625,cpu_nanos:55689000,user_nanos:47761000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:48,input_rows_per_second:522,output_rows_per_second:104,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:47834833,cpu_nanos:45860000,user_nanos:39594000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:36,input_rows_per_second:693,output_rows_per_second:138,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:36051750,cpu_nanos:35401000,user_nanos:30276000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:108,input_rows_per_second:232,output_rows_per_second:46,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:107698834,cpu_nanos:67399000,user_nanos:60120000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:89,input_rows_per_second:279,output_rows_per_second:55,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:89431750,cpu_nanos:87399000,user_nanos:82486000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:36,input_rows_per_second:702,output_rows_per_second:140,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:35595250,cpu_nanos:34275000,user_nanos:32643000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:39,input_rows_per_second:642,output_rows_per_second:128,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:38891708,cpu_nanos:37020000,user_nanos:34996000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:34,input_rows_per_second:732,output_rows_per_second:146,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:34141125,cpu_nanos:32628000,user_nanos:30786000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:38,input_rows_per_second:653,output_rows_per_second:130,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:38242459,cpu_nanos:36824000,user_nanos:35107000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:48,input_rows_per_second:520,output_rows_per_second:104,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:48046416,cpu_nanos:44795000,user_nanos:41783000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:38,input_rows_per_second:664,output_rows_per_second:132,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:37624625,cpu_nanos:36067000,user_nanos:34233000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:29,input_rows_per_second:860,output_rows_per_second:172,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:29043125,cpu_nanos:28326000,user_nanos:27263000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:53,input_rows_per_second:470,output_rows_per_second:94,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:53106041,cpu_nanos:49474000,user_nanos:46099000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:45,input_rows_per_second:553,output_rows_per_second:110,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:45162917,cpu_nanos:42568000,user_nanos:39742000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:40,input_rows_per_second:619,output_rows_per_second:123,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:40335667,cpu_nanos:36132000,user_nanos:34354000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:43,input_rows_per_second:581,output_rows_per_second:116,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:43028208,cpu_nanos:40935000,user_nanos:38190000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:30,input_rows_per_second:822,output_rows_per_second:164,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:30398000,cpu_nanos:29613000,user_nanos:28504000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
peak_memory:1495616,elapsed_millis:29,input_rows_per_second:867,output_rows_per_second:173,input_megabytes:0,input_megabytes_per_second:0,wall_nanos:28825333,cpu_nanos:28177000,user_nanos:27255000,input_rows:25,input_bytes:0,output_rows:5,output_bytes:130
      pull_constants_above_group_by ::   42.266 cpu ms :: 1.43MB peak memory :: in    25,      0B,     591/s,      0B/s :: out     5,    130B,     118/s,     3KB/s

@kaikalur
Copy link
Contributor

kaikalur commented Nov 4, 2022

Please squash the commits into a single one

For example:

SELECT userid, 'constant val' AS extra, COUNT(1) as cnt from T GROUP BY 1, 2
into

SELECT userid,  'constant val' as extra, cnt from (SELECT userid, AS extra, COUNT(1) from T GROUP BY 1) s
@mlyublena mlyublena force-pushed the optimize-const-grouping-keys branch from 3cd42fe to 00b69cc Compare November 4, 2022 18:41
@kaikalur kaikalur merged commit 6ad58d8 into prestodb:master Nov 4, 2022
@wanglinsong wanglinsong mentioned this pull request Jan 12, 2023
30 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants