Enable arm again #3852
Merged
Enable arm again #3852
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Issue building the new image: |
This was referenced Mar 31, 2019
|
Waiting for feedback in jaegertracing/jaeger-client-cpp#151 |
This was referenced May 24, 2019
|
Looking forward to seeing this 👍 Note: you have some rebase conflicts showing up? |
Codecov Report
@@ Coverage Diff @@
## master #3852 +/- ##
========================================
Coverage ? 57.9%
========================================
Files ? 87
Lines ? 6544
Branches ? 0
========================================
Hits ? 3789
Misses ? 2324
Partials ? 431
Continue to review full report at Codecov.
|
|
Still getting build errors https://gist.github.com/aledbf/62eee376ec5859bb32f97489c9242ded |
|
@alexellis thank you for asking the state of this PR. The error is located in the jaeger tracing. Right now I am testing a conditional build, omitting this plugin for arm. If this works I can re-enable arm with the caveat that the opentracing feature (for jaeger) not be available. |
ec82c3a to
bf0a814
Compare
|
ok, this is working now. I am building the final version of the image now.
@alexellis I will post the link of a docker image of the ingress controller so you can test it |
I want to add a new route to the ingress controller (Go) to dump a JSON object with stats of the ingress controller:
Then the user can upload to a new google form (not sure if supports uploads) and then decide if we can do some cleanup using two or three releases for deprecation and removal |
|
I would love to test this. I'm currently installing and operating Nginx through the helm chart. Can you provide a helm command that I can use? |
|
@alexellis something like helm install \
--name nginx-ingress stable/nginx-ingress \
--namespace arm \
--set rbac.create=true \
--set controller.service.type=NodePort \kubectl --namespace arm set image deployment/nginx-ingress-controller \
nginx-ingress-controller=quay.io/kubernetes-ingress-controller/nginx-ingress-controller-arm:dev |
|
@aledbf apologies if this error is unrelated but I gave this a go (your exact commands) and the pod output was this (raspberry pi 3b): |
|
@MattJeanes no, this is an issue with the helm chart if you run a k8s cluster > v1.14.0 related to #4127 To fix this please run the next patch command to add the new roles compatible with the new API. kubectl patch --namespace ingress-nginx role nginx-ingress-role --type='json' -p='[{"op": "add", "path": "/rules/-", "value": {"apiGroups": ["networking.k8s.io"],"resources": ["ingresses"],"verbs": ["get","list","watch"]}}]'
kubectl patch --namespace ingress-nginx role nginx-ingress-role --type='json' -p='[{"op": "add", "path": "/rules/-", "value": {"apiGroups": ["networking.k8s.io"],"resources": ["ingresses/status"],"verbs": ["update"]}}]'After running that command delete the pod. The new one will have the new permissions |
|
Had to modify those commands slightly but still no dice. Not sure if your link was correct? Here are exact commands used: Might be worth noting that the arm:0.20.0 build starts ok. Happy to take this to another issue if you want, had a search around couldn't find any other documentation/issues/etc around this. Some more hopefully useful output: Latest pod logs: |
|
@MattJeanes please also execute The previous command only patches the role in the ingress-nginx namespace |
|
Thank you very much, that's fixed it. Just had to make a tiny tweak to your commands as below. Hopefully this can help others if they run into this problem too. |
|
This is exciting 🎉 What is the final set of commands to run? Quick question for @MattJeanes: last time I checked, tiller wasn't available for armhf - are you templating the YAML then applying it, or do you have a working tiller Docker image on your RPi too? Alex |
|
@alexellis if you use my commands used from my previous two comments you should get it working. Note this is only for Kubernetes v1.14 and above otherwise you don't need to run all the patch commands. I used https://github.com/jessestuart/tiller-multiarch to install tiller on arm. |
|
I'll note that it was also necessary to change the default-backend image to arm too: kubectl --namespace nginx-ingress set image deployment/nginx-ingress-default-backend \
nginx-ingress-default-backend=gcr.io/google_containers/defaultbackend-arm:1.5 |
|
@MattJeanes I am facing this "does not have any active Endpoint" .. how do i resolve this ? |
|
@alokhom did you also follow the command above sent by @mylesagray about changing the default backend? The error seems to be saying that for the default backend service there are no active pods ready, likely caused by them not being the arm versions and failing to start up |
|
Do we have an update on ingress-nginx on arm? I have the same issue where I cannot install ingress-nginx in K3S running on RPi 4 64bit. Thanks |
|
@sotiris84 what do you mean? Edit: if you are using helm to install the ingress controller please check #4876 |
|
@sotiris84 I just added a comment in the k3s issue you opened about this issue |
|
@aledbf do I miss something? At least for 0.27.0 the images are not for arm64: In addition to that I checked this by extracting the image to check the Besides of that (looking at the filesystem of the container) I see that the image contains header files (opentracing, msgpack, jaegertracing), timezones, terminfo database, many of Lua files. I guess it's size could be further reduced by removing most of that? |
|
@bwolf thank you for finding this issue. Please download the images again.
Please no. The ingress controller runs as user, something that avoids the installation of packages. Also, the lua files are required by the ingress controller. About the size, have you seen we reduce the size to the half in this release? |
|
Even with the right compilation, docker returns a different platform |
|
Best to use the manifest query command in docker instead. See docker manifest --help https://docs.docker.com/engine/reference/commandline/manifest/ |
Thank you very much for the reply. That was before I figure out I needed to add the -arm64 prefix in the image. Now all is good thanks. However, it would be nice if it could detect the cpu architecture and choose the correct image. |
Thank you very much. Much appreciated! |
|
@sotiris84 the issue here is related to the lack of support of V2 manifests in quay.io. We already have a PR to fix this in the project #4271 |
|
Can some kind soul explain to me how to install this? I am following the nginx official instructions but now have to apply the deployment YAML. What image do I give for arm? |
|
Yes, try k3sup, "k3sup app install nginx-ingress" on any Kubernetes Cluster. You can also install with helm but https://k3sup.dev automates that including on arm. |
|
@alexellis I can use this on a non k3sup cluster? |
|
I did just say that. Read the link please. |
|
I'm sorry. I need to wake up! Again, apologies. I will RTFM. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Which issue this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close that issue when PR gets merged): fixes #Special notes for your reviewer: