Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make proxy optional for k8s_metadata_decorator #239

Merged
merged 4 commits into from
Jan 4, 2021
Merged

Conversation

jsirianni
Copy link
Member

@jsirianni jsirianni commented Dec 31, 2020

Description of Changes

Added allow_proxy option (default: false) for enabling or disabling the use of a proxy for the k8s_metadata_decorator operator.

Generally, you would not want to use a proxy for connections to the Kubernetes API, however, the Kubernetes client will use any proxy set at HTTP_PROXY or HTTPS_PROXY unless the proxy function is set.

This is a problem because the proxy is likely not inside the cluster, therefore it cannot forward requests to the internal Kubernetes APIs.

supporting docs:

More context here: resolves #238

Please check that the PR fulfills these requirements

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • Add a changelog entry (for non-trivial bug fixes / features)
  • CI passes

Joseph Sirianni added 2 commits December 31, 2020 12:42
disabling the use of a proxy for this operator. Generally, you would
not want to use a proxy for connections to the Kubernetes API.

The Kubernetes client, by default, will use any proxy set at HTTP_PROXY or
HTTPS_PROXY unless the proxy function is set.

supporting docs:
- https://godoc.org/k8s.io/client-go/rest#Config
- https://golang.org/pkg/net/http/#ProxyFromEnvironment

resolves #238
@djaglowski
Copy link
Member

Log Files Logs / Second CPU Avg (%) CPU Avg Δ (%) Memory Avg (MB) Memory Avg Δ (MB)
1 1000 1.8448719 +0.18967581 121.58648 -3.8917007
1 5000 6.138044 +0.2241211 129.77681 -0.9392395
1 10000 12.362334 +0.27586842 138.61368 +1.2118683
1 50000 59.44743 +2.1536484 175.36786 -3.8544006
1 100000 117.1406 +0.4727249 231.18993 +10.140228
10 100 2.3104236 +0.13792849 127.77573 +2.0765076
10 500 7.069168 +0.17242289 133.65773 -1.1739044
10 1000 13.638361 +0.22426796 142.39682 +2.9362793
10 5000 62.156887 -2.276188 179.47037 +2.6087036
10 10000 125.76164 +2.097229 227.02074 +13.126343

@codecov
Copy link

codecov bot commented Dec 31, 2020

Codecov Report

Merging #239 (7c3d494) into master (92e9403) will decrease coverage by 0.09%.
The diff coverage is 50.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #239      +/-   ##
==========================================
- Coverage   71.36%   71.27%   -0.09%     
==========================================
  Files         101      101              
  Lines        5538     5542       +4     
==========================================
- Hits         3952     3950       -2     
- Misses       1165     1166       +1     
- Partials      421      426       +5     
Impacted Files Coverage Δ
.../transformer/k8smetadata/k8s_metadata_decorator.go 31.01% <50.00%> (+0.49%) ⬆️
operator/flusher/flusher.go 84.00% <0.00%> (-4.00%) ⬇️
operator/builtin/input/file/file.go 73.97% <0.00%> (-2.05%) ⬇️
operator/builtin/output/forward/forward.go 56.52% <0.00%> (+1.45%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92e9403...7c3d494. Read the comment docs.

Copy link
Member

@djaglowski djaglowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just one comment on documentation.

docs/operators/k8s_metadata_decorator.md Outdated Show resolved Hide resolved
@djaglowski
Copy link
Member

Log Files Logs / Second CPU Avg (%) CPU Avg Δ (%) Memory Avg (MB) Memory Avg Δ (MB)
1 1000 1.6379507 -0.017245412 124.02829 -1.4498901
1 5000 6.0862975 +0.17237473 130.85829 +0.14224243
1 10000 12.586557 +0.50009155 137.52505 +0.12324524
1 50000 58.795914 +1.5021324 175.5846 -3.6376648
1 100000 111.692535 -4.975342 226.59671 +5.5470123
10 100 2.1380153 -0.034479856 128.78664 +3.0874176
10 500 7.017459 +0.12071371 136.75821 +1.9265747
10 1000 13.345241 -0.068852425 141.99866 +2.5381165
10 5000 63.984367 -0.44870758 187.48317 +10.621506
10 10000 125.63978 +1.9753647 227.57098 +13.67659

@djaglowski
Copy link
Member

Log Files Logs / Second CPU Avg (%) CPU Avg Δ (%) Memory Avg (MB) Memory Avg Δ (MB)
1 1000 1.5689942 -0.086201906 119.70353 -5.7746506
1 5000 6.4485035 +0.5345807 130.12715 -0.5888977
1 10000 12.190102 +0.10363579 137.09941 -0.30239868
1 50000 57.708138 +0.41435623 174.39143 -4.830826
1 100000 115.40625 -1.2616272 227.1087 +6.0590057
10 100 2.2587876 +0.086292505 125.9767 +0.27748108
10 500 6.758783 -0.13796234 134.99785 +0.16621399
10 1000 13.22446 -0.18963337 145.18292 +5.7223816
10 5000 62.83741 -1.595665 181.25162 +4.3899536
10 10000 128.74446 +5.0800476 213.39102 -0.5033722

@jsirianni jsirianni merged commit 0a43d10 into master Jan 4, 2021
@jsirianni jsirianni deleted the k8smetadata-proxy branch January 4, 2021 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

k8s_metadata_decorator operator breaks when using a proxy
2 participants