Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ALL] psutil.test() truncates %MEM too aggressively #1474

Closed
embray opened this issue Apr 4, 2019 · 2 comments
Closed

[ALL] psutil.test() truncates %MEM too aggressively #1474

embray opened this issue Apr 4, 2019 · 2 comments
Assignees
@giampaolo
Labels
bug tests

Comments

@embray
Copy link
Contributor

embray commented Apr 4, 2019

Bug description

The psutil.test() function calls round(p.info['memory_percent'], 1), and for anything where that returns 0 it just displays ?. This means the %MEM shown for any small process using less then 1/100th a % of total memory is ?.

For a system with reasonably large total physical memory (e.g. 32 GB) and many small processes, most processes are just displayed like:

USER         PID %MEM     VSZ     RSS TTY           START    TIME  COMMAND
...
           29900    ?    6208     832 ?             Mar27   00:00  bash
           30396    ?    6208     832 ?             Mar27   00:00  bash
           30436    ?    5440    4480 ?             Apr03   00:00  bash
           30464    ?    6400     832 ?             Mar27   00:00  bash
           30492    ?    6208    7232 ?             Apr03   00:00  bash
           30580    ?   11648   13440 ?             Apr03   02:43  vim
           31264    ?    6400    2112 ?             Mar25   00:01  bash
           31812    ?    6400    2176 ?             Mar26   00:05  bash
           31828    ?    6144    1536 ?             Mar27   00:00  bash
           32040    ?    9280     960 ?             Mar25   01:34  vim

Maybe it would be better to round to 2 decimal places so we can at least display 100th of a %, and for anything smaller display 0.0 instead of ? which I think more clearly communicates "very small".
@embray embray added the bug label Apr 4, 2019
@giampaolo
Copy link
Owner

giampaolo commented Apr 5, 2019 •
edited
Loading

Fixed. Now it looks like this:

$ python -c "import psutil; psutil.test()" 
USER          PID  %MEM      VSZ      RSS          TTY START    TIME  COMMAND
root            1   0.0   220.9M     6.5M              Mar27   09:08  systemd
root            2   0.0     0.0B     0.0B              Mar27   00:00  kthreadd
root            4   0.0     0.0B     0.0B              Mar27   00:00  kworker/0:0H
root            6   0.0     0.0B     0.0B              Mar27   00:00  mm_percpu_wq
root            7   0.0     0.0B     0.0B              Mar27   00:06  ksoftirqd/0
root            8   0.0     0.0B     0.0B              Mar27   03:19  rcu_sched
root            9   0.0     0.0B     0.0B              Mar27   00:00  rcu_bh
root           10   0.0     0.0B     0.0B              Mar27   00:00  migration/0
root           11   0.0     0.0B     0.0B              Mar27   00:00  watchdog/0
root           12   0.0     0.0B     0.0B              Mar27   00:00  cpuhp/0
root           13   0.0     0.0B     0.0B              Mar27   00:00  cpuhp/1
root           14   0.0     0.0B     0.0B              Mar27   00:00  watchdog/1
[...]
systemd-r   28918   0.0    69.9M     3.9M              Mar29   01:07  systemd-resolved
systemd-t   28923   0.0   142.7M   652.0K              Mar29   00:00  systemd-timesyncd
root        28927   0.0   215.2M    52.4M              Mar29   00:06  systemd-journald
root        31095   0.0    45.3M     3.2M              Mar29   00:01  systemd-udevd
root        31165   0.0     0.0B     0.0B              12:28   00:00  kworker/u16:2
root        31678   0.0     0.0B     0.0B              12:31   00:00  kworker/6:2
root        32188   0.0     0.0B     0.0B              Apr03   00:00  xfsalloc
root        32189   0.0     0.0B     0.0B              Apr03   00:00  xfs_mru_cache
root        32199   0.0     0.0B     0.0B              Apr03   00:00  jfsIO
root        32200   0.0     0.0B     0.0B              Apr03   00:00  jfsCommit
root        32202   0.0     0.0B     0.0B              Apr03   00:00  jfsCommit
root        32203   0.0     0.0B     0.0B              Apr03   00:00  jfsCommit
root        32204   0.0     0.0B     0.0B              Apr03   00:00  jfsCommit
root        32205   0.0     0.0B     0.0B              Apr03   00:00  jfsCommit
root        32206   0.0     0.0B     0.0B              Apr03   00:00  jfsCommit
root        32207   0.0     0.0B     0.0B              Apr03   00:00  jfsCommit
root        32208   0.0     0.0B     0.0B              Apr03   00:00  jfsCommit
root        32209   0.0     0.0B     0.0B              Apr03   00:00  jfsSync
root        32219   0.0     0.0B     0.0B              Apr03   00:04  kworker/u17:3
giampaolo   32553   0.0   792.1M    33.1M              Apr02   00:13  gnome-calculator

@embray
Copy link
Contributor Author

embray commented Apr 5, 2019

Thank you. This is somewhat better, but please see 275df44#r33052053

Anyways it's obviously nothing urgent; just something I noticed and thought I'd make note of while tooling around :)

nlevitt added a commit to nlevitt/psutil that referenced this issue Apr 9, 2019
@nlevitt
Merge remote-tracking branch 'origin/master' into pslisten
ab4f505 
* origin/master: (150 commits)
  Linux / CPU freq, fixes giampaolo#1481
  improve pmap.py script
  reuse ps.py script in psutil.test()
  move get_terminal_size() in _compat.py
  improve ps.py script
  improve ps.py script
  move bytes2human() into psutil._common and reused it from scripts dir
  fix windows failure re. py 2 vs. 3
  fix linux tests
  fix giampaolo#1474: fix formatting of psutil.tests() which mimicks 'ps aux' output
  give CREDITS for giampaolo#1480
  remove outdated tests
  Fix read access violation in psutil.cpu_count(logical=False) (giampaolo#1480)
  update doc
  update doc
  [Win] Process IO priority constants + high priority (giampaolo#1479 / giampaolo#1476)
  don't fail if there are not prev failed tests
  fix giampaolo#1478: add make command to re-run tests failed on last run
  [Win] return value is not properly handled for undocumented NT* Windows APIs. (giampaolo#1477)
  fix error on py 2.7 where OSError doesn't always have winerror attribute
  update HISTORY for giampaolo#1475
  properly check OSError.winerror
  refactor ionice() on Linux
  refactor ionice() on Linux
  ionice test refactoring
  giampaolo#1404: fix regression not returning CPUs > 9
  give CREDITS to Daniel Beer for giampaolo#1471
  Fix spurious exception when iterating processes on Solaris (giampaolo#1471)
  give CREDITS for giampaolo#1470
  Fix corner case when /etc/mtab doesn't exist and procfs=/proc (giampaolo#1470)
  update DEVNOTES
  Typo fixed (giampaolo#1469)
  giampaolo#1458: implement colors on Windows
  update HISTORY / CREDITS, fix some C warnings
  Make uptime type consitent to fix boot time error. (giampaolo#1225)
  fix giampaolo#1463: cpu_distribution.py script is broken
  update issue template
  issue giampaolo#1404 / linux / phys CPUs count
  Big docfix (giampaolo#1464)
  Big docfix (giampaolo#1464)
  Make tests invariant to LANG setting (giampaolo#1462)
  test runner: show errors on KeyboardInterrupt
  test runner refactoring (avoid code duplication)
  Coloured tests (giampaolo#1459)
  pre-release
  [Windows] calculate USS memory by using NtQueryVirtualMemory (giampaolo#1453)
  fix version highlighting in docs/index (giampaolo#1455)
  fix version highlighting in README (giampaolo#1454)
  run win specific tests twice as fast
  test refactoring
  test: avoid failing at import time
  mention how to run tests in INSTALL guide
  giampaolo#1448: fix Wine support due to missing rtlIpv6AddressToStringA
  update HISTORY
  Fix giampaolo#1329: [AIX] disable some functions based on availability in libperfstat (giampaolo#1349)
  bump up version, fix some doc issues
  fix ResourceWarning
  pre-release
  fix giampaolo#1447: we weren't use @wrap_exceptions around oneshot() (doh\!)
  update doc + change git hook location
  update doc
  make pre-release checks/install src dist in a venv
  add new make command to check tar.gz sanity
  move doc; rephrase it a bit
  add issue templates for 'bug' and 'enhancement' types
  remove issue template commited by accident
  Update issue templates
  giampaolo#1291: (BACKWARD-INCOMPATIBLE) remove memory_maps() on OSX
  Restore Win-7 support on GIT master (5.5.1 was OK) (giampaolo#1446)
  try to fix ntext.h
  restore previous def
  fix compiler warning
  fix compiler warning
  fix compiler warning
  fix compiler warning
  take defs from PH
  set proper  SYSTEM_PROCESS_INFORMATION struct from PH
  fix compilation warnings
  giampaolo#1398 / win / cmdline: call NtQueryInformationProcess twice, the first time to get the right buf size (ProcessHacker does this)
  update doc
  better print formatting for print scripts
  fix giampaolo#1442: use python3 as Makefile default
  appveyor: run print scripts after tests
  highlight top 6 slowest calls
  add printerr() and exit() to shared utils module
  add arg parser for ad script
  introduce a new scriptsutils.py private module shared by all internal utils + refactor print_access_speed.py script
  giampaolo#1291 / OSX: mark memory_maps() as deprecated and make it alwats raise AccessDenied
  OSX memory_maps() - add error handling
  add script for to benchmark API calls
  move access_denied script
  _assert_alive() refactor (linux)
  refactor
  fix NetBSD: Process.connections() may return incomplete results if using oneshot() giampaolo#1439
  add win tests related to send_signal(CTRL_C_EVENT) giampaolo#1227
  fix win test
  fix win tests
  fix backslash warnings
  refactor README a bit
  #fix 1438: do not return any parent() for PID 0 + update doc
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@giampaolo giampaolo

Labels
bug tests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@embray @giampaolo