[Kubernetes]: The kube server could be used as http-proxy for docker #7469
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…plicitly. If enabled, it expects docker to have a http-proxy with an IP (likely a local/private IP) If docker do have http-proxy, it creates an IPTable entry to direct any traffic to that IP to the configured Kubernetes master. Undo the IPTables when kube server is removed.
|
This pull request introduces 4 alerts when merging bf9fc28 into ccc7bd1 - view on LGTM.com new alerts:
|
W/o this fix, it throws exception
isabelmsft
reviewed
May 5, 2021
isabelmsft
reviewed
May 5, 2021
isabelmsft
reviewed
May 5, 2021
isabelmsft
reviewed
May 5, 2021
isabelmsft
reviewed
May 5, 2021
No logical code changes.
isabelmsft
reviewed
May 6, 2021
isabelmsft
previously approved these changes
May 7, 2021
lguohan
reviewed
May 7, 2021
|
Looking at a different way of meeting this requirement. Hence closing. |
|
This pull request introduces 1 alert when merging 6753a0f into a557dbd - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging ce0f672 into a557dbd - view on LGTM.com new alerts:
|
Dropped unused import
isabelmsft
previously approved these changes
Jun 5, 2021
syslog spewing errors, not able to test properly. will approve pending further testing
K8s control plane had issue with 1.21.1, which is now fixed, hence back to 1.21.1
isabelmsft
approved these changes
Jun 15, 2021
qiluo-msft
pushed a commit
that referenced
this pull request
Jun 17, 2021
…7469) Why I did it The SONiC switches get their docker images from local repo, populated during install with container images pre-built into SONiC FW. With the introduction of kubernetes, new docker images available in remote repo could be deployed. This requires dockerd to be able to pull images from remote repo. Depending on the Switch network domain & config, it may or may not be able to reach the remote repo. In the case where remote repo is unreachable, we could potentially make Kubernetes server to also act as http-proxy. How I did it When admin explicitly enables, the kubernetes-server could be configured as docker-proxy. But any update to docker-proxy has to be via service-conf file environment variable, implying a "service restart docker" is required. But restart of dockerd is vey expensive, as it would restarts all dockers, including database docker. To avoid dockerd restart, pre-configure an http_proxy using an unused IP. When k8s server is enabled to act as http-proxy, an IP table entry would be created to direct all traffic to the configured-unused-proxy-ip to the kubernetes-master IP. This way any update to Kubernetes master config would be just manipulating IPTables, which will be transparent to all modules, until dockerd needs to download from remote repo. How to verify it Configure a switch such that image repo is unreachable Pre-configure dockerd with http_proxy.conf using an unused IP (e.g. 172.16.1.1) Update ctrmgrd.service to invoke ctrmgrd.py with "-p" option. Configure a k8s server, and deploy an image for feature with set_owner="kube" Check if switch could successfully download the image or not.
lguohan
pushed a commit
that referenced
this pull request
Jun 30, 2021
renukamanavalan
added a commit
that referenced
this pull request
Jul 16, 2021
renukamanavalan
added a commit
to renukamanavalan/sonic-buildimage
that referenced
this pull request
Jul 20, 2021
…roxy for docker (sonic-net#7469)" (sonic-net#8023)" (sonic-net#8158) This reverts commit 7236fa9. Restore original PR sonic-net#7469
carl-nokia
pushed a commit
to carl-nokia/sonic-buildimage
that referenced
this pull request
Aug 7, 2021
…onic-net#7469) Why I did it The SONiC switches get their docker images from local repo, populated during install with container images pre-built into SONiC FW. With the introduction of kubernetes, new docker images available in remote repo could be deployed. This requires dockerd to be able to pull images from remote repo. Depending on the Switch network domain & config, it may or may not be able to reach the remote repo. In the case where remote repo is unreachable, we could potentially make Kubernetes server to also act as http-proxy. How I did it When admin explicitly enables, the kubernetes-server could be configured as docker-proxy. But any update to docker-proxy has to be via service-conf file environment variable, implying a "service restart docker" is required. But restart of dockerd is vey expensive, as it would restarts all dockers, including database docker. To avoid dockerd restart, pre-configure an http_proxy using an unused IP. When k8s server is enabled to act as http-proxy, an IP table entry would be created to direct all traffic to the configured-unused-proxy-ip to the kubernetes-master IP. This way any update to Kubernetes master config would be just manipulating IPTables, which will be transparent to all modules, until dockerd needs to download from remote repo. How to verify it Configure a switch such that image repo is unreachable Pre-configure dockerd with http_proxy.conf using an unused IP (e.g. 172.16.1.1) Update ctrmgrd.service to invoke ctrmgrd.py with "-p" option. Configure a k8s server, and deploy an image for feature with set_owner="kube" Check if switch could successfully download the image or not.
carl-nokia
pushed a commit
to carl-nokia/sonic-buildimage
that referenced
this pull request
Aug 7, 2021
… docker (sonic-net#7469)" (sonic-net#8023) This change causes nightly test to fail due to the fake proxy IP is not reachable. Reverts sonic-net#7469 This reverts commit f7ed82f.
carl-nokia
pushed a commit
to carl-nokia/sonic-buildimage
that referenced
this pull request
Aug 7, 2021
…roxy for docker (sonic-net#7469)" (sonic-net#8023)" (sonic-net#8158) This reverts commit 7236fa9. Restore original PR sonic-net#7469
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why I did it
The SONiC switches get their docker images from local repo, populated during install with container images pre-built into SONiC FW. With the introduction of kubernetes, new docker images available in remote repo could be deployed. This requires dockerd to be able to pull images from remote repo.
Depending on the Switch network domain & config, it may or may not be able to reach the remote repo. In the case where remote repo is unreachable, we could potentially make Kubernetes server to also act as http-proxy.
How I did it
When admin explicitly enables, the kubernetes-server could be configured as docker-proxy. But any update to docker-proxy has to be via service-conf file environment variable, implying a "service restart docker" is required. But restart of dockerd is vey expensive, as it would restarts all dockers, including database docker.
To avoid dockerd restart, pre-configure an http_proxy using an unused IP. When k8s server is enabled to act as http-proxy, an IP table entry would be created to direct all traffic to the configured-unused-proxy-ip to the kubernetes-master IP. This way any update to Kubernetes master config would be just manipulating IPTables, which will be transparent to all modules, until dockerd needs to download from remote repo.
How to verify it
Configure a switch such that image repo is unreachable
Pre-configure dockerd with http_proxy.conf using an unused IP (e.g. 172.16.1.1)
Update ctrmgrd.service to invoke ctrmgrd.py with "-p" option.
Configure a k8s server, and deploy an image for feature with set_owner="kube"
Check if switch could successfully download the image or not.
Which release branch to backport (provide reason below if selected)
Description for the changelog
A picture of a cute animal (not mandatory but encouraged)