Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Dashboard] Add memory graphs optimized for OOM debugging #48530

Merged
merged 3 commits into from
Nov 7, 2024
Merged

[Dashboard] Add memory graphs optimized for OOM debugging #48530

merged 3 commits into from
Nov 7, 2024

Conversation

mimiliaogo
Copy link
Contributor

@mimiliaogo mimiliaogo commented Nov 4, 2024 •
edited
Loading

Why are these changes needed?

The current graph shows memory usage of each node along side the MAX memory across the cluster.
For oom debugging, we care more about memory utilization (percentage) on each node.

image

Related issue number

Closes #47007

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

alanwguo
alanwguo reviewed Nov 4, 2024
View reviewed changes
Copy link
Contributor

@alanwguo alanwguo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! let's actually start by not separating out > 80% and < 80%.

@mimiliaogo
Copy link
Contributor Author

Thanks! let's actually start by not separating out > 80% and < 80%.

I've added a double thick line for node > 80%, do you still want this?

@mimiliaogo
Add memory graphs optimized for OOM debugging
be7542d 
Signed-off-by: Mimi Liao <mimiliao2000@gmail.com>
@mimiliaogo
remove fill and stack for memory graphs and highlight >80% node
cbdff0f 
Signed-off-by: Mimi Liao <mimiliao2000@gmail.com>
@mimiliaogo
remove 80% seperate and highlights
eaf5679 
Signed-off-by: Mimi Liao <mimiliao2000@gmail.com>
@mimiliaogo mimiliaogo requested a review from alanwguo November 6, 2024 20:53
alanwguo
alanwguo approved these changes Nov 6, 2024
View reviewed changes
Copy link
Contributor

@alanwguo alanwguo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@alanwguo alanwguo added the go add ONLY when ready to merge, run all tests label Nov 6, 2024
@rkooo567 rkooo567 merged commit 4ecb0af into ray-project:master Nov 7, 2024
6 checks passed
JP-sDEV pushed a commit to JP-sDEV/ray that referenced this pull request Nov 14, 2024
@mimiliaogo @JP-sDEV
[Dashboard] Add memory graphs optimized for OOM debugging (ray-projec…
314cde8 
…t#48530)

The current graph shows memory usage of each node along side the MAX memory across the cluster.
For oom debugging, we care more about memory utilization (percentage) on each node.

Signed-off-by: JP-sDEV <jon.pablo80@gmail.com>
mohitjain2504 pushed a commit to mohitjain2504/ray that referenced this pull request Nov 15, 2024
@mimiliaogo @mohitjain2504
[Dashboard] Add memory graphs optimized for OOM debugging (ray-projec…
09bf426 
…t#48530)

The current graph shows memory usage of each node along side the MAX memory across the cluster.
For oom debugging, we care more about memory utilization (percentage) on each node.

Signed-off-by: mohitjain2504 <mohit.jain@dream11.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alanwguo alanwguo alanwguo approved these changes

Assignees
No one assigned
Labels
go add ONLY when ready to merge, run all tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Dashboard] Add memory graphs optimized for OOM debugging
3 participants
@mimiliaogo @alanwguo @rkooo567