Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

make the memory tracing in HashAggExec more accurate #14705

Closed
qw4990 opened this issue Feb 10, 2020 · 7 comments · Fixed by #18257
Closed

make the memory tracing in HashAggExec more accurate #14705

qw4990 opened this issue Feb 10, 2020 · 7 comments · Fixed by #18257
Labels
epic/memory-management help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. sig/execution SIG execution type/enhancement The issue or PR belongs to an enhancement.

Comments

@qw4990
Copy link
Contributor

qw4990 commented Feb 10, 2020 •
edited
Loading

Feature Request

Is your feature request related to a problem? Please describe:

A memTracker is used to trace memory consumed by Chunks in HashAggExec, but in some cases(#14024), the aggPartialResultMapper used to store intermediate results consumes a lot of memory but we don't trace it now.

Please help us to make the memory tracing in HashAggExec more accurate by tracing all memory consumed by the aggPartialResultMapper.

If you are interested in this issue, welcome to join our SIG-EXEC channel to discuss it with us.
Thanks for your contribution:D
@qw4990 qw4990 added type/enhancement The issue or PR belongs to an enhancement. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. difficulty/medium labels Feb 10, 2020
@qw4990 qw4990 mentioned this issue Feb 10, 2020
Closed
55 tasks
@tangwz
Copy link
Contributor

tangwz commented Feb 12, 2020

let me fix it.

@qw4990
Copy link
Contributor Author

qw4990 commented Feb 12, 2020

let me fix it.

Nice, thanks very much. If you have any problem or question, please feel free to discuss with us :D

@SunRunAway
Copy link
Contributor

@tangwz Friendly ping~

@SunRunAway
Copy link
Contributor

SunRunAway commented Mar 10, 2020 •
edited
Loading

Another case:
https://asktug.com/t/topic/32980

The key of the aggregation map is a string, we should also track the memory usage when the group by column is wide.

@tangwz tangwz mentioned this issue May 6, 2020
Closed
@tangwz
Copy link
Contributor

tangwz commented May 6, 2020

@SunRunAway PTAL.

@SunRunAway
Copy link
Contributor

Duplicate of #14103

@SunRunAway SunRunAway marked this as a duplicate of #14103 May 13, 2020
@b41sh b41sh mentioned this issue Jun 29, 2020
Merged
@XuHuaiyu XuHuaiyu reopened this Jul 13, 2020
@b41sh b41sh mentioned this issue Aug 21, 2020
Closed
17 tasks
@qw4990
Copy link
Contributor Author

qw4990 commented Oct 15, 2020

Close since all tasks have been finished. Thanks @b41sh @mmyj and all other Contributors!!! 🎊

@qw4990 qw4990 closed this as completed Oct 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
epic/memory-management help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. sig/execution SIG execution type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
4 participants
@SunRunAway @qw4990 @tangwz @XuHuaiyu