Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track CPU usage #21

Merged
merged 4 commits into from
Oct 25, 2019
Merged

Track CPU usage #21

merged 4 commits into from
Oct 25, 2019

Conversation

krinsman
Copy link
Contributor

@krinsman krinsman commented Jul 11, 2019

EDIT: This is one PR of two; this one adds CPU information.

At NERSC (where two of the three days of the Jupyter for HPC workshop in June was), we're interested in having nbresuse report more metrics besides memory usage.

Also, when running on a shared login node on Cori, we would like the (recommended) memory limit for each user to be the total available memory divided by the number of current users.

This PR would allow us to accomplish that by having nbresuse track more metrics (specifically other memory characteristics, CPU usage, and number of JupyterHub users), as well as allow people to define their own custom functions to calculate dynamic memory limits, whose import location can then be specified using the notebook config file.

(An example custom function for dynamically calculating a memory limit can be found here; the README links to a video demo showing it work in practice.)

It seems like these features might also be useful for other people, hence the reason for making this PR. Of course, the implementation is more than a little wonky, and any suggestions you have for improvement would be highly appreciated, even if merging into master ultimately isn't possible.

This seems to work based on the testing I have done inside of a Docker image, see here. I still need to set everything up on Cori to test it there, although I don't see in advance why it wouldn't also work.

------

Maybe @shreddd could say more about why these features would be useful at NERSC, if that information would be helpful or useful to you @yuvipanda.

@krinsman krinsman changed the title Track more metrics and allow users to calculate custom dynamic memory limits. Track CPU usage Sep 3, 2019
@yuvipanda yuvipanda merged commit 9b25fbb into jupyter-server:master Oct 25, 2019
davidbrochart pushed a commit to davidbrochart/jupyter-resource-usage that referenced this pull request Dec 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants