Built-in DNS server extremely slow for large responses #4430

dziemba · 2020-04-07T17:00:29Z

I have tried with the latest version of my channel (Stable or Edge)
I have uploaded Diagnostics
Diagnostics ID: 747C7FF1-4351-4543-B9E8-2B79CD4183A2/20200407164330

Expected behavior

When running DNS queries through docker's built-in DNS server, I expect similar performance (response times) compared to using external DNS servers directly.

Context

While the following reproduction case is an artificial example, it is a quite common case to have these huge DNS responses, especially when working with K8s ingress etc. I'm filing this issue because we essentially can't use Docker for Mac at our company because it breaks our DNS resolution (by timing out a lot).

Using an external DNS server directly is not an option for us since we also need DNS resolution for hostnames in the current docker network (for integration tests etc.)

Actual behavior

When executing DNS queries that yield large responses, it takes an extremely long time to return results. The results look correct though. This only happens on Docker for Mac. Running the same queries on linux-native docker does not show the same issue.

Information

macOS Version: 10.15.4
Docker Desktop Version: 2.2.0.5

Steps to reproduce the behavior

To reproduce create a docker container with dig installed:

docker run -ti debian bash
apt-get update && apt-get install -y dnsutils

Then run DNS queries that yield large responses in that container:

time dig hugedns.test.dziemba.net  # ~5s
time dig hugedns.test.dziemba.net @ns-73-a.gandi.net  # ~0.2s

If you run queries that involve a lot of CNAMES, the performance gets even worse:

time dig cname5.hugedns.test.dziemba.net  # ~18s
time dig cname5.hugedns.test.dziemba.net @ns-73-a.gandi.net  # ~0.2s

As you can see, this only happens when docker's DNS server is used. When querying the external DNS server directly, performance is not impacted. When run on linux directly (not Docker for Mac), docker's built-in DNS server is also as fast as the external one.

Please let me know if you need more details.

The text was updated successfully, but these errors were encountered:

dziemba · 2020-04-17T08:39:55Z

Hey @djs55, maybe you can help out here? 🙂
You were already very helpful with a related-but-different problem a while back: #2160

docker-robott · 2020-07-16T01:00:11Z

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

dziemba · 2020-07-16T12:41:24Z

/remove-lifecycle stale

thaJeztah · 2020-07-16T13:37:54Z

@djs55 any idea where the bottleneck is? I can confirm that this is slow on Docker for Mac, but fast on a native Linux machine.

dziemba · 2020-08-31T07:54:37Z

Any update on this? 🙂 @thaJeztah @djs55

thaJeztah · 2020-08-31T09:24:04Z

No update yet

docker-robott · 2020-11-29T01:00:07Z

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

thaJeztah · 2020-11-30T21:11:56Z

/remove-lifecycle stale

docker-robott · 2021-02-28T01:00:13Z

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

djs55 · 2021-02-28T15:27:26Z

/remove-lifecycle stale

thaJeztah · 2021-02-28T16:08:46Z

/lifecycle frozen

jjqq2013 · 2021-03-19T06:30:10Z

I've been suffering this issue for half a year.

Even for a normal size dns query, it is extremely slow, 30 seconds!

root@98f777f20ca9:/# cat /etc/resolv.conf
# DNS requests are forwarded to the host. DHCP DNS options are ignored.
nameserver 192.168.65.5

root@98f777f20ca9:/# time nslookup -v a.b.c
Server:		192.168.65.5
Address:	192.168.65.5#53

Non-authoritative answer:
Name:	a.b.c
Address: 100.79.28.134
;; connection timed out; no servers could be reached


real	0m30.013s
user	0m0.005s
sys	0m0.008s

A ping command takes 15 seconds to resolve the dns name, see the strace result, it shows that the slow sys call happens in

sendto
recvfrom

The docker info is:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc., v0.5.0)

Server:
 Containers: 147
  Running: 5
  Paused: 0
  Stopped: 142
 Images: 43
 Server Version: 20.10.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc version: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.19.121-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 1.941GiB
 Name: docker-desktop
 ID: IGA3:HGMF:LFF4:QJIL:DKRS:FQ7D:OG6Y:W5AN:G7KE:QJQI:CRPH:XLO7
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 52
  Goroutines: 65
  System Time: 2021-03-19T06:31:49.480891297Z
  EventsListeners: 4
 HTTP Proxy: gateway.docker.internal:3128
 HTTPS Proxy: gateway.docker.internal:3129
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

arsenius-clbs · 2023-04-24T23:50:19Z

what's the status of this?

maxpatiiuk · 2024-07-25T01:49:37Z

This started happening for us in 2021 when we started migrating to m1 macs
The issue is not present on intel macs

docker info

Client:
 Version:    24.0.6
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.5
    Path:     /Users/maxpatiiuk/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.22.0-desktop.2
    Path:     /Users/maxpatiiuk/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/maxpatiiuk/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/maxpatiiuk/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.8
    Path:     /Users/maxpatiiuk/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/maxpatiiuk/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/maxpatiiuk/.docker/cli-plugins/docker-scan
  scout: Docker Scout (Docker Inc.)
    Version:  v1.0.7
    Path:     /Users/maxpatiiuk/.docker/cli-plugins/docker-scout

Server:
 Containers: 9
  Running: 8
  Paused: 0
  Stopped: 1
 Images: 32
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
 runc version: v1.1.8-0-g82f18fe
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.4.16-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 7.667GiB
 Name: docker-desktop
 ID: 2653f9f0-5660-4b56-8c59-4e486c26a70b
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

more details in this issue: specify/specify7#2574 (comment)

docker-robott added the version/2.2.0.5 label Apr 7, 2020

docker-robott added the lifecycle/stale label Jul 16, 2020

docker-robott removed the lifecycle/stale label Jul 16, 2020

thaJeztah mentioned this issue Aug 31, 2020

Mac: handle large DNS result sets moby/vpnkit#474

Open

docker-robott added the lifecycle/stale label Nov 29, 2020

docker-robott removed the lifecycle/stale label Nov 30, 2020

Rauell mentioned this issue Jan 25, 2021

DNS Resolution Hangs for 5-15 Seconds on Docker Desktop 3.1.0 docker/for-win#10212

Open

5 tasks

docker-robott added the lifecycle/stale label Feb 28, 2021

docker-robott removed the lifecycle/stale label Feb 28, 2021

docker-robott added the lifecycle/frozen label Feb 28, 2021

jjqq2013 mentioned this issue Mar 29, 2021

ping -4 works but ping does not work. -> vpnkit dns: extremely slow, with internal unexpected error from dnssd: Timeout (-65568) #5516

Closed

3 tasks

maxpatiiuk mentioned this issue Jul 25, 2024

Optimize Dockerized Specify 7 for M1 macs specify/specify7#2574

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Built-in DNS server extremely slow for large responses #4430

Built-in DNS server extremely slow for large responses #4430

dziemba commented Apr 7, 2020

dziemba commented Apr 17, 2020

docker-robott commented Jul 16, 2020

dziemba commented Jul 16, 2020

thaJeztah commented Jul 16, 2020

dziemba commented Aug 31, 2020

thaJeztah commented Aug 31, 2020

docker-robott commented Nov 29, 2020

thaJeztah commented Nov 30, 2020

docker-robott commented Feb 28, 2021

djs55 commented Feb 28, 2021

thaJeztah commented Feb 28, 2021

jjqq2013 commented Mar 19, 2021 •

edited

Loading

arsenius-clbs commented Apr 24, 2023

maxpatiiuk commented Jul 25, 2024

Built-in DNS server extremely slow for large responses #4430

Built-in DNS server extremely slow for large responses #4430

Comments

dziemba commented Apr 7, 2020

Expected behavior

Context

Actual behavior

Information

Steps to reproduce the behavior

dziemba commented Apr 17, 2020

docker-robott commented Jul 16, 2020

dziemba commented Jul 16, 2020

thaJeztah commented Jul 16, 2020

dziemba commented Aug 31, 2020

thaJeztah commented Aug 31, 2020

docker-robott commented Nov 29, 2020

thaJeztah commented Nov 30, 2020

docker-robott commented Feb 28, 2021

djs55 commented Feb 28, 2021

thaJeztah commented Feb 28, 2021

jjqq2013 commented Mar 19, 2021 • edited Loading

arsenius-clbs commented Apr 24, 2023

maxpatiiuk commented Jul 25, 2024

jjqq2013 commented Mar 19, 2021 •

edited

Loading