Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High CPU usage on host #3539

Closed
3 tasks done
marek-obuchowicz opened this issue Feb 21, 2019 · 43 comments
Closed
3 tasks done

High CPU usage on host #3539

marek-obuchowicz opened this issue Feb 21, 2019 · 43 comments

Comments

@marek-obuchowicz
Copy link

  • I have tried with the latest version of my channel (Stable or Edge)
  • I have uploaded Diagnostics
  • I have tried to reset docker to factory defaults
  • Diagnostics ID: D8C4C1BC-B3EE-42E6-9875-74843E9A55E7/20190221083009

Expected behavior

Host OS should not be using so much CPU when docker is running.

Actual behavior

I have 6 containers running (posgtgresql, redis, elasticsearch and PHP applicatons). They are basically doing nothing - no data is flowing thru the system and web-application is not getting any requests. The CPU usage on the docker VM, as indicated by top command, is very small (1-3%), but on host OS the process com.docker.hyperkit is consuming up to 70% CPU, causing my machine to produce excess heat and slowing down "everything".

Information

The problem used to be happen since a few months, i can't surely say that it is related to any specific update.

  • macOS Version: 10.12.6
  • docker-for-mac version: 2.0.0.3 (31259), stable
  • hardware: 8vCPU (4-core intel i7), 16GB RAM, SSD drive
  • resources allocated to docker: 4vCPU, 3GB RAM, 2GB SWAP

The issue seems to be regarding docker-for-mac VM. If I use VirtualBox to start Linux, allocate same resources and start containers - load on host OS is barely visible, as expected.

No osxfs is used, no files are shared with the host OS.

@Housik
Copy link

Housik commented Feb 21, 2019

Same happening to me, almost zero cpu usage inside containers, but com.docker.hyperkit is constantly consuming between 30% to 50% CPU.

  • Docker Engine version 18.09.2
  • OS X version 10.14.3
  • Diagnostics ID: F3876880-AD64-4811-BC9E-1FF410B98410/20190221112519

@kammoh
Copy link

kammoh commented Feb 28, 2019

Same issue, but no container running at all! Happen even after clean reinstall (deleted all everything under ~/Library/Containers). com.docker.hyperkit is running with 18 threads and 240% CPU usage even after docker.app is closed.

  • Docker version 18.09.2, build 6247962
  • OS X version 10.14.3

@hemith
Copy link

hemith commented Mar 7, 2019

Same actual bahavior.

Information

  • old macbook pro(i7) -Docker Community Edition (v 17.12.0-ce-mac55 d62ef8d1b0)
    It's OK.

I bought new macbook pro recently.

  • new macbook pro(i9) - Docker Desktop Community (v 2.0.0.3 (31259) 8858db33c8)
    It's not OK.

I installed Latest Docker Desktop Community on my new macbook pro.
Initially, there was a very high CPU load. However, after restarting the docker and using it, the CPU load is not so high.

  • macOS Version: 10.14.3
  • docker-for-mac version: 2.0.0.3 (31259), stable
  • hardware: Intel Core i9 / 2.9 GHz / 6 cores , 32GB RAM, SSD drive
  • resources allocated to docker: 4vCPU, 3GB RAM, 2GB SWAP

@treedy
Copy link

treedy commented Apr 11, 2019

From a fresh install of Docker Desktop Community, I regularly observe about 4% CPU utilization of com.docker.hyperkit. When I enable K8s I notice a sustained 40-60% CPU utilization. Both figures are without any other non-system or non-k8s containers running.

macOS Version: 10.14.3
Docker Desktop Community version: 2.0.0.3 (31259), stable
hardware: Intel Core i7 / 2 GHz / 8 cores , 16GB RAM, SSD drive
resources allocated to docker: 4vCPU, 2GB RAM, 1GB SWAP
Kubernetes: v1.10.11

@marek-obuchowicz
Copy link
Author

Due to high CPU usage, we have evaluated other hypervisors. Quick benchmarks showed that hypervisor used by docker is slower (in terms of IO and CPU) than other alternatives, like virtualbox, parallels or vmware. We have switched to docker-machine and minikube with NFS, which resulted in lower CPU usage and much nicer IO (for both local images and shared filesystems). While this solution has more complicated setup than docker-for-mac, it also enables a big performance boost for daily docker usage (especially if combined with parallels/vmware fusion, but even virtualbox seems to be better choice than hyperkit).

@bgehman
Copy link

bgehman commented May 30, 2019

Ya'll sure this is a docker problem and not a Kubernetes problem?

Docker Whale --> Preferences --> Kubernetes --> Check "Show System Containers"

Then run docker stats and see what you can see...

@eplanet
Copy link

eplanet commented Jun 4, 2019

Actually disabling Kubernetes single-node cluster stopped the ridiculous 100% usage for me. Thanks for the tip!

@ajsharp
Copy link

ajsharp commented Jun 22, 2019

Anyone from Docker had a look at this? This is clearly an ongoing major problem.

cc @mikeparker @akimd

@dfang
Copy link

dfang commented Aug 1, 2019

com.docker.hyperkit always consumes a lot of cpu ...

@dragonfax
Copy link

Is this problem still being ignored? fluctuating 45-70% cpu usage, when i've got no containers and pods at all (only system containers). Tried this on 2 different macs. And clean installations of Docker for Mac.

@teolisitza
Copy link

Is this problem still being ignored? fluctuating 45-70% cpu usage, when i've got no containers and pods at all (only system containers). Tried this on 2 different macs. And clean installations of Docker for Mac.

Same same:

macOS Version: 10.14.4
Docker Desktop Community version: 2.1.0.1, stable
hardware: Intel Core i7 / 2 GHz / 8 cores , 16GB RAM, SSD drive
resources allocated to docker: 8vCPU, 6GB RAM, 1GB SWAP

@ashishsc
Copy link

Same experience here:
No containers running
OSX 10.14.4
Docker Desktop Community version 2.1.0.1
Hardware: MBP 2019, i7, 6 cores @ 2.6GHz, 32GB RAM

Cpu usage: 11%

@vdavari
Copy link

vdavari commented Aug 30, 2019

+1

@FlorinAndrei
Copy link

FlorinAndrei commented Sep 12, 2019

+1

Docker Desktop 2.1.0.2
macOS 10.14.6

com.docker.hyperkit is at 30 ... 50% CPU even with no containers and no images
Kubernetes is running (but, again, no containers have been created)

EDIT: Stopping Kubernetes made the issue disappear - CPU usage is back to normal.

But K should not eat CPU like this even when it's doing nothing. That's not normal.

@FlorinAndrei
Copy link

On Windows 10 Pro, same Docker Desktop version, I do not see high CPU usage when Kubernetes is enabled.

Something's rotten in the state of Mac.

@PhillippOhlandt
Copy link

Same issue. It's utterly annoying and nobody seems to care.

If I could, I would just remove this mess called Kubernetes from my system but the IT industry is blindly moving towards it just because someone said it's cool.

Waiting 10 - 20 seconds for a page to load in the browser while com.docker.hyperkit consume 100% of my CPU is indeed very cool.

@jakeonrails
Copy link

My laptop is burning my legs and eating my entire battery in just over 2 hours because of this. No activity but constant 40-50% CPU usage for com.docker.hyperkit

@umotif-aebling
Copy link

I'm seeing the same - but up to 680% of CPU on an i9 MacBook Pro. Battery draining rapidly while Docker is doing stuff, despite being connected to an 80W USB C power supply.

Kubernetes is disabled and the bear minimum of macOS directories are listed in Preferences > File Sharing.

@TrebuhD
Copy link

TrebuhD commented Oct 8, 2019

Super high CPU usage here as well, despite nothing significant running inside containers. My Macbook barely charges while Docker is running!

@sam0x17
Copy link

sam0x17 commented Oct 12, 2019

I've been seeing this as well on a 2019 macbook pro. Constant 5% cpu usage regardless of whether any containers are running.

@zdnk
Copy link

zdnk commented Oct 29, 2019

Having same issue, is there any update on the issue?

@Tom-paqtcom
Copy link

My CPU is over 100% alot. Nothing running or Kubernetes enabled....

@janjongboom
Copy link

janjongboom commented Nov 20, 2019

I'm seeing a similar issue since enabling Kubernetes (70-90% CPU usage on Macbook Pro 13" 2019 model). This is the output of my docker stats (no containers running other than the k8s ones):

CONTAINER ID        NAME                                                                                                                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
8964178d15b0        k8s_kubernetes-dashboard_kubernetes-dashboard-5f7b999d65-cp5zc_kube-system_fbfd4b89-0b5c-11ea-8476-025000000001_0   0.00%               17.91MiB / 3.855GiB   0.45%               0B / 0B             8.19kB / 0B         13
fd07d6bb57b7        k8s_POD_kubernetes-dashboard-5f7b999d65-cp5zc_kube-system_fbfd4b89-0b5c-11ea-8476-025000000001_0                    0.00%               272KiB / 3.855GiB     0.01%               0B / 0B             0B / 0B             1
50c7ce04f745        k8s_compose_compose-api-57ff65b8c7-bdj4w_docker_de56a98a-0b56-11ea-9ebb-e66bb9043d76_3                              0.13%               18.97MiB / 3.855GiB   0.48%               0B / 0B             3.69MB / 8.19kB     14
9ef840e10e7f        k8s_kube-proxy_kube-proxy-xsspt_kube-system_b2259e7e-0b56-11ea-9ebb-e66bb9043d76_2                                  0.20%               11.77MiB / 3.855GiB   0.30%               0B / 0B             30.8MB / 0B         12
ef9299261668        k8s_POD_kube-proxy-xsspt_kube-system_b2259e7e-0b56-11ea-9ebb-e66bb9043d76_2                                         0.00%               232KiB / 3.855GiB     0.01%               0B / 0B             0B / 0B             1
a43bbf085f2a        k8s_coredns_coredns-6dcc67dcbc-swplf_kube-system_b23909d4-0b56-11ea-9ebb-e66bb9043d76_2                             1.18%               8.883MiB / 170MiB     5.23%               0B / 0B             15.6MB / 12.3kB     15
789486c24108        k8s_coredns_coredns-6dcc67dcbc-p728d_kube-system_b2326db3-0b56-11ea-9ebb-e66bb9043d76_2                             0.81%               8.883MiB / 170MiB     5.23%               0B / 0B             15MB / 12.3kB       17
e4518796713f        k8s_compose_compose-6c67d745f6-9qb8d_docker_de6173f0-0b56-11ea-9ebb-e66bb9043d76_2                                  0.00%               8.562MiB / 3.855GiB   0.22%               0B / 0B             20.6MB / 8.19kB     14
32c04605af7b        k8s_POD_compose-6c67d745f6-9qb8d_docker_de6173f0-0b56-11ea-9ebb-e66bb9043d76_2                                      0.00%               228KiB / 3.855GiB     0.01%               0B / 0B             0B / 0B             1
2bf398eef49e        k8s_POD_coredns-6dcc67dcbc-swplf_kube-system_b23909d4-0b56-11ea-9ebb-e66bb9043d76_2                                 0.00%               240KiB / 3.855GiB     0.01%               0B / 0B             0B / 0B             1
52d78d919a11        k8s_POD_coredns-6dcc67dcbc-p728d_kube-system_b2326db3-0b56-11ea-9ebb-e66bb9043d76_2                                 0.00%               260KiB / 3.855GiB     0.01%               0B / 0B             0B / 0B             1
cc9227476b18        k8s_POD_compose-api-57ff65b8c7-bdj4w_docker_de56a98a-0b56-11ea-9ebb-e66bb9043d76_2                                  0.00%               244KiB / 3.855GiB     0.01%               0B / 0B             0B / 0B             1
7d009cbb0428        k8s_kube-scheduler_kube-scheduler-docker-desktop_kube-system_a4ab8fef1586dc510fabb4d25490cc80_2                     0.17%               11.62MiB / 3.855GiB   0.29%               0B / 0B             31.5MB / 0B         14
a4a065bded3c        k8s_kube-controller-manager_kube-controller-manager-docker-desktop_kube-system_4d7f45923abda25d42f820adbd5ddfe8_2   5.37%               46.16MiB / 3.855GiB   1.17%               0B / 0B             67.4MB / 0B         13
601377738fbb        k8s_kube-apiserver_kube-apiserver-docker-desktop_kube-system_ee523ec2318a3aaecc0ee0c0b0a8bd2a_2                     5.44%               249.2MiB / 3.855GiB   6.31%               0B / 0B             78.3MB / 0B         16
5999ff785b69        k8s_etcd_etcd-docker-desktop_kube-system_3773efb8e009876ddfa2c10173dba95e_2                                         2.92%               40.38MiB / 3.855GiB   1.02%               0B / 0B             37.8MB / 435MB      17
52303bd67f7a        k8s_POD_kube-scheduler-docker-desktop_kube-system_a4ab8fef1586dc510fabb4d25490cc80_2                                0.00%               236KiB / 3.855GiB     0.01%               0B / 0B             0B / 0B             1
9f549f54bed1        k8s_POD_kube-controller-manager-docker-desktop_kube-system_4d7f45923abda25d42f820adbd5ddfe8_2                       0.00%               268KiB / 3.855GiB     0.01%               0B / 0B             0B / 0B             1
5db15b0db0b5        k8s_POD_kube-apiserver-docker-desktop_kube-system_ee523ec2318a3aaecc0ee0c0b0a8bd2a_2                                0.00%               264KiB / 3.855GiB     0.01%               0B / 0B             0B / 0B             1
65e81c331383        k8s_POD_etcd-docker-desktop_kube-system_3773efb8e009876ddfa2c10173dba95e_2                                          0.00%               244KiB / 3.855GiB     0.01%               0B / 0B             729kB / 0B          1

I'm not sure if the CPU% here is the total utilization over all cores, as it adds up to ~17%.

@ericholscher
Copy link

ericholscher commented Nov 26, 2019

Same issue here, around 100% CPU usage on the host, but nothing showing on the running containers. Seems to only happen when I'm running containers that have Python processes in them, instead of just postgres/redis services. So perhaps something interesting there.

@mvaitkus
Copy link

Same here, no containers running, but hyperkit CPU usage was around 35-45%. Dropped to around 3% when I disabled local kubernetes cluster.

@sam0x17
Copy link

sam0x17 commented Dec 11, 2019

I now just disabled docker unless I am actively using it :(

@jasonbronson
Copy link

So here is what I found for my setup. I know some folks are saying no containers etc cause issues.
I removed almost all my mappings for shared folders except 1.
I set my cpus to 8
ram to 15gb
swap to 1gb
no kubernetes running
I made sure that spot light was not indexing the docker.raw file under preferences
I have filevault turned off under system preferences

my docker-compose file is using cached for handling the mounted volume.
I'm running the following containers
portainer
openbridge/ob_php-fpm
openbridge/nginx
redis
mysql

I was getting a very high 500% cpu use.
IO on the mac didn't seem even remotely running much
very odd....

I ran docker stats from command line and it showed around 45% on the php container which made no sense since I was not running anything.
I logged into the container and ran "top"
I noticed this
https://gyazo.com/e70e3150bc6cdb6f9da5697480220817
The container was running a bunch of find commands every X cycles with https://mmonit.com/monit/documentation/monit.html

So I copied the find chown command it's using and ran it in the container by hand.
Turns out it's trying to run chown on every single file inside of node_modules and vendor directories for php which is hundreds of thousands.

I increased the cycle checks in the /etc/monit.d/ folder to 300000 and restarted monit program in the container.
The cpu went from 500% to 21% and the fan shutdown like it was the end of a work day.

What I believe is happening is that the docker program is not really translating io back out to mac osx for whatever reason. Any heavy container io activity will spike the cpu like a mad man.
Docker stats really doesn't translate the details of what is really happening in the container either.

I recommend if you have high cpu run top and figure out what is running and verify you expected it to run.
Also keep in mind none of the find commands I had running were spiking cpu in the containers they just happened to be running io commands.
I would recommend maybe even looking at iotop as well inside the containers you run to get a better idea.

Hope this helps someone.

my new cpu usage
https://gyazo.com/b3aa2f47053da52af83a752e8169e4a3

@sam0x17
Copy link

sam0x17 commented Jan 7, 2020

Dumped my mac for an XPS w/ubuntu budgie 19.10, problem went away xD

@ozburo
Copy link

ozburo commented Jan 11, 2020

Dumped my mac for an XPS w/ubuntu budgie 19.10, problem went away xD

Right behind ya! 👍

@olivierodo
Copy link

olivierodo commented Jan 11, 2020

It doesn't sound that it's gonna be resolved quite soon...

My interim solution to limit that craziness is to manually control the damage. Then i'm using my own VM and install docker in it.
(virtualbox -> debian server -> docker)
Went from 200% to 16%...
Good luck to all

@douglascorrea
Copy link

Same happen here with Macbook Pro 16" - and docker-for-mac 2.2.0, I'll be trying alternatives for using docker containers, this is probably not good for the MacBook lifetime

@170
Copy link

170 commented Feb 5, 2020

@douglascorrea
Did you do Reset to factory defaults?
It's fixed on my MacBook tentatively.

@douglascorrea
Copy link

@170 I've tried but didn't worked, moved to use default apache and install postresql via brew in my machine

@cabezayunke
Copy link

Same problem here.
CPU for docker hyperkit between 40% and 70% with kubernetes enabled. Down to 9% when disabled.

Docker engine 19.03.8
Mac OS 10.15.3
Docker desktop 2.2.0.4
kubernetes 1.15.5

@m3talsmith
Copy link

Just switched back to a Mac from Windows and Linux. Was wondering why my fan was screaming for over an hour. Checked the activity monitor to see docker Desktop had my CPU pegged around 140%. Quit Docker and all is nice and quiet now. My rust compilations don't even spin the fan up that much! Considering this was all a fresh install and I hadn't even used Docker yet, and looking at this thread, I'd say the mac version has some serious problems. I'm going to go grab podman and buildah while you sort this out Docker.

@ozburo
Copy link

ozburo commented Jun 10, 2020

@m3talsmith pls report back your findings if you find a decent alternative for running Docker based containers on a Mac! It's been this way since forever 😄

@djs55
Copy link
Contributor

djs55 commented Jul 30, 2020

Hi -- a quick progress update on the Mac CPU issue: It seems to be a mixture of several bugs including

  1. excessively large queues of events in the filesharing implementation -- there is a proposed fix for this but it's not in any current release build
  2. a clash with osquery over the hyperkit serial console: this is fixed in Edge but not in Stable
  3. a further problem with the hyperkit serial console: there is a proposed fix for this but it's not in any current release build

(and possibly more, since the symptoms are all very similar it means they're hard to tell apart)

There is a development build with all the current known fixes (for 1, 2 and 3): https://desktop-stage.docker.com/mac/edge/47050/Docker.dmg (Edited to make this a notarized build) . If you have a chance to try it, please do and let me know how it goes. Unfortunately the build is signed but not notarized so you will need to open it in finder, right click and click on "Open" to bypass the notarization check. I'm very interested to hear both about any bugs in this build and about any remaining CPU issues. Thanks!

@Sutharsan
Copy link

@djs55, Docker Desktop CPU usage dropped from 100+% to less than 5% when containers are (mostly) idle. Nice work.

@teunzwart
Copy link

I also see a large drop in CPU usage, from 150+% to 10-20%. Thanks!

@anton-kazlouski
Copy link

@djs55 works for me! CPU usage dropped from 120+% to 10-20%.
Got another minor problem with one volume, but if was easy to fix.

@ikwattro
Copy link

ikwattro commented Aug 8, 2020

@djs55 Works for me, dropped from 100+% to 8%. Great work and thank you !

@stephen-turner
Copy link
Contributor

I'm going to dupe this against the main CPU ticket, #3499, so that we can consolidate the conversation in one place. I've just posted a new build there that you can try.

@docker-robott
Copy link
Collaborator

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked

@docker docker locked and limited conversation to collaborators Sep 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests