qemu with socket_vmnet not working on macos m1

[csantanapr]

What Happened?

Trying the new feature to avoid having minikube tunnel

Using macbook m1

Followed the instructions https://minikube.sigs.k8s.io/docs/drivers/qemu/
Install vmnet

git clone https://github.com/lima-vm/socket_vmnet.git && cd socket_vmnet
sudo make PREFIX=/opt/socket_vmnet install

Had problems dhcp_leases file not found then try workaround

make uninstall

reboot

make install

Then try again

minikube delete --all
minikube start --driver=qemu --network=socket_vmnet

Made sure no weird DNS in host /etc/resolv.conf

Attach the log file

Error

😄  minikube v1.28.0 on Darwin 12.6.1 (arm64)
✨  Using the qemu2 (experimental) driver based on user configuration
👍  Starting control plane node minikube in cluster minikube
🔥  Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
🔥  Deleting "minikube" in qemu2 ...
🤦  StartHost failed, but will try again: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: could not find an IP address for 26:38:e8:5a:fc:7c
🔥  Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
😿  Failed to start qemu2 VM. Running "minikube delete" may fix it: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: could not find an IP address for 72:47:f9:c2:48:b8

❌  Exiting due to GUEST_PROVISION: Failed to start host: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: could not find an IP address for 72:47:f9:c2:48:b8

log.txt

Operating System

macOS (Default)

Driver

QEMU

[mprimeaux]

@csantanapr Thanks for opening the issue.

I've been using Minikube 1.28.0 today on my Mac Studio (M1 Ultra) with the --network socket_vmnet option and it has been working without any noticeable issue.

I am curious as to your local network configuration in that the logs appear to indicate there are no more DHCP leases available. Would you mind reviewing this section and let me know if this helps?

Would you mind posting the output of minikube config view so I can investigate a bit further?

[spowelljr]

@medyagh had this issue, after he uninstalled, rebooted, and reinstalled it worked for him so that's where the documented workaround came from. If we discover another workaround we should document it as well.

[csantanapr]

@mprimeaux @spowelljr

I tried all workarounds, including all the ones mentioned in the doc like uninstalling and rebooting still doesn't work for me 😢

[mprimeaux]

I tried all workarounds, including all the ones mentioned in the doc like uninstalling and rebooting still doesn't work for me 😢

@csantanapr Well, that's certainly less than ideal 😞

Would you mind posting the output of minikube config view and also trying the following?

git clone https://github.com/lima-vm/socket_vmnet.git && cd socket_vmnet
sudo make PREFIX=/opt/socket_vmnet uninstall
sudo make PREFIX=/opt/socket_vmnet install BRIDGED=en0

I'm curious if bridged mode changes anything. Note, you might need to change the value en0 if you're only on WiFi or have multiple ethernet adapters.

[csantanapr]

Hi @mprimeaux I'm using both wifi and ethernet, your saying to configure it with ethernet interface?

[csantanapr]

Tried again uninstall and install with BRIDGED=en0

🤦  StartHost failed, but will try again: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: open /var/db/dhcpd_leases: no such file or directory
🔥  Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
😿  Failed to start qemu2 VM. Running "minikube delete" may fix it: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: open /var/db/dhcpd_leases: no such file or directory

❌  Exiting due to GUEST_PROVISION: Failed to start host: creating host: create: creating: IP address never found in dhcp leases file: failed to get IP address: open /var/db/dhcpd_leases: no such file or directory

The minikube config view is empty

[mprimeaux]

Hi @csantanapr. Thanks for trying that option. I suspected you'd end up with the same failure. I'll debug a bit on my end.

[causton81]

Hello, I see this too on OSX Ventura. I watched the leases file while starting minikube and the MAC address it wants only shows up after minikube gives up on polling the leases file. I suspect something is keeping the file locked and preventing the DHCP daemon from writing the assignment?

watch grep 'a6:fc:8b:db:0:a0' /var/db/dhcpd_leases

[spowelljr]

Forgot to link it, but I opened an issue in socket_vmnet

lima-vm/socket_vmnet#18

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[aelsnz]

Just raising here, but could it be that the issue is with bootpd not starting/running properly for ipv4 requests. I have seen this issue and when digging more into the details it does seem as if socket_vmnet is working fine, but the DHCP address never gets assigned. I had things working, then updated to latest macOS 13.x (13.3 and then 13.3.1) and in both cases I noticed that it appears bootpd is not assigning the DHCP address. While testing with enable/disable ipv6 and restarting mac things seem to start working again without any changes to socket_vmnet. - lima-vm/socket_vmnet#18 (comment)

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[relivvius]

It still does not work for me also, is there any update?

[mprimeaux]

Please try the following:

sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/libexec/bootpd
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --unblock /usr/libexec/bootpd

This has worked consistently for me and others. I'll update this comment with the related issue where this was identified. I believe @spowelljr may find it faster.

UPDATE: Here is the related issue in socket_vmnet.

[iwinux]

@mprimeaux sorry to bring up an old issue: are these 2 commands the same as adding bootpd to the allow list in Settings Panel "Network -> Firewall"?

[mprimeaux]

@iwinux I believe this is the case as I do see bootp in the list of firewall entries configured to "Allow incoming connections".