Ingress not working on Mac M1 host machine

[NixBiks]

What Happened?

I have ingress addon enable in my minikube cluster. I add my host names to /etc/hosts, e.g. myapp.local. The following command just hangs instead of responding as expected

curl myapp.local

I think this might be related to this issue where minikube tunnel hangs (as it does for me as well - usually I don't need to use tunneling with ingress addon though)

Attach the log file

Let me know which kind of logs is helpful

Operating System

macOS (Default)

Driver

Docker

[RA489]

/kind support

[kmahelona]

I've got the exact same problem. With docker we had to do things like domain-name.local:8090 and adding the port was annoying. With minikube and hyperkit driver the port shenanigans went away! But now with Apple M1 Pro and only having the docker driver, I suspect the very reason why we had to do the 8090 thing is the same reason why this issue exists.

Also for my specific case, I'm using the resolver approach as documented and while scutil --dns reports my domains as, e.g.

resolver #10
  domain   : ipython.corporalocal.nz
  nameserver[0] : 192.168.49.2
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000002 (Reachable)
  order    : 1

If I ping ipython.corporalocal.nz I get ping: cannot resolve corporalocal.io: Unknown host. I thought the issue was the resolver so I added 192.168.49.2 corporalocal.nz to my /etc/hosts. If I ping corporalocal.nz I do get the initial PING corporalocal.nz (192.168.49.2): 56 data bytes before the request timeouts (I assume since no "pong" is running).

FWIW tunneling does work for me, but again that's not an ideal way for us. I think the real problem here is docker and weather this can be fixed in minikube 🤷🏽‍♂️ .

[NixBiks]

But did you actually get it working on your M1 @kmahelona ? I can't reach my minikube ingress services ufortunalely - haven't figured out any workaround

[sharifelgamal]

minikube tunnel is required for ingress to work on the docker/podman drivers, and that's currently the only driver supported on M1 (VM drivers are in the works). Right now, minikube tunnel will hang when run in another terminal, but that does not mean it's not working, we just don't output anything.

[klaases]

hi @mr-bjerre, were you able to review the comments above regarding the tunnel command? There won't necessarily be any output in the same terminal where the tunnel command running, but the tunnel is working.

[NixBiks]

@klaases we decided to move away from minikube instead since we can orchestrate locally without. So I never tried actually - sorry

[kosta1221]

Hi, I can confirm that running minikube tunnel works for me on m1 with the docker driver. Keep in mind that your etc/hosts file needs to map to 127.0.0.1, instead of the output of minikube ip or kubectl get ingress - this is an important gotcha.

[cjellick]

Weird follow up question on this: all the docs regarding minikube tunnel talk about it tunnelling for services of type LoadBalancer, but I don't have any of those. I installed the nginx ingress add on and it created a nodeport service and I need minikube tunnel to expose my ingresses.

So is minikube tunnel doing something for the nginx-ingress nodeport serivce? Is this a special case or does it do something for all nodePort service?

[germanflighel]

Dropping by to let you know what worked for us:

• enabling the Ingress addon for minikube minikube addons enable ingress
• minikube tunnel was needed. Do this in another terminal since it needs to be up. You should see logs indicating that the tunnel is working for your exposed service
• Though it only worked with LoadBalancer services. Using either NodePort or the default service type, would not do the trick. I think for most use cases when runnning locally, the service type should not really matter.
• Mapping our domain to 127.0.0.1 instead of the output of minikube ip

With these, we were able to access our cluster from outside using Ingress!

[dskatz]

The steps outlined here by several people do indeed work. However there is an important limitation moving away from hyperkit and toward the docker driver. hyperkit essentially allowed multiple ingress controllers to be running within minikube and exposed as distinct IP addresses by minikube tunnel. However, on docker this is not possible since it runs on localhost address space.

I'd like to see support for hyperkit or another driver that allows routes into the container IP space.

[klaases]

Dropping by to let you know what worked for us:

• enabling the Ingress addon for minikube minikube addons enable ingress
• minikube tunnel was needed. Do this in another terminal since it needs to be up. You should see logs indicating that the tunnel is working for your exposed service
• Though it only worked with LoadBalancer services. Using either NodePort or the default service type, would not do the trick. I think for most use cases when runnning locally, the service type should not really matter.
• Mapping our domain to 127.0.0.1 instead of the output of minikube ip

With these, we were able to access our cluster from outside using Ingress!

Glad to see the community stepping into work through resolutions!

If needed, feel free to re-open the issue by commenting with /reopen.

Thank you for sharing your experience!

[Leo-Mrl]

/reopen

[k8s-ci-robot]

@Leo-Mrl: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Leo-Mrl]

Hello everyone,

I would like to reopen this issue or get any information you may have about how to solve it @klaases.

I am currently taking an intro course to Kubernetes; I am experienced developer, proficient with Docker, but new to Kubernetes.

I have a namespace.yaml and an app.yaml file. I am able to start my minikube cluster and deploy my pod without any trouble.

I am trying to access my app through 3 different ways:

• By creating a Kubectl-portward tunnel: using the command kubectl port-forward service/ingress-nginx-controller -n ingress-nginx 8080:80; this works great!
• By creating a tunnel with minikube tunnel: using minikube tunnel and then retrieving IP address of the Ingress with kubectl get ingress podinfo; this does not work.
• By directly accessing the cluster IP address: using minikube ip; this does not work either, and returns the same ip address as the one exposed in the previous step.

To summarize, I am able to connect to my app deployed in the cluster by making use of port forwarding, but not by using the minikube tunnel command, which seems like a hurdle in one's path to learning Kubernetes.

I also tried the steps described by @germanflighel, but it did not solve my issue unfortunately.

Any advice or solution 🙏 ?

Thank you very much, I will remain available if needed.

[abduljabbar3492]

Hi, I can confirm that running minikube tunnel works for me on m1 with the docker driver. Keep in mind that your etc/hosts file needs to map to 127.0.0.1, instead of the output of minikube ip or kubectl get ingress - this is an important gotcha.

Thank you so much.
After spending hours it works in my Mac M1

[harshraj22]

@germanflighel not able to access ingress. Can you maybe share a small yaml file with ingress and clusterIp defined which is accessible from host machine for your mac ?

[louisthaboogie]

this solution works but I don't think you are using the ingress. When you create a loadbalancer service and use minikube tunnel it sends the traffic to the service directly. It should work without the ingress. I think Ingress are broken with minikube on m1s

With these, we were able to access our cluster from outside using Ingress!

[Leo-Mrl]

Update on my side (my apologies for not updating earlier): in a nutshell, Mac M1 runs minikube's cluster in a Docker container inside the Docker for Mac VM.

Ideally, we would want to use minikube start --vm with Virtualbox (or another VM provider) as backend, but Mac M1 do not offer the required drivers.

Therefore: port-forwarding should work natively on Mac M1, but tunneling is a bit more complex to set up. Ideally, if you require the use of tunneling, you want to run your app on another platform (such a Linux).

[sushant032]

I tried almost all the things mentioned in this thread but nothing worked out. I had to change the driver to QEMU and that worked seamlessly.
Steps that I followed: https://medium.com/@sushantkumarsinha22/kubernetes-setting-up-ingress-on-apple-silicon-mac-m1-5fb6bddcb838