Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Using k0s in Podman on Fedora 34 Silverblue #1195

Closed
FilBot3 opened this issue Oct 15, 2021 · 8 comments
Closed

Using k0s in Podman on Fedora 34 Silverblue #1195

FilBot3 opened this issue Oct 15, 2021 · 8 comments
Assignees
@makhov
Labels
bug Something isn't working needs-triage

Comments

@FilBot3
Copy link

FilBot3 commented Oct 15, 2021

The issue or request

I am trying to run k0s in Podman on Fedora 34 Silverblue in an attempt to see if it would even work. I copied and pasted commands 1 and 3 because 2 was optional from the k0s in Docker docs page. The Control Plane seems to come up, but isn't showing any resources. It seems to be in a continual crash and restart loop. Has anyone else tried this or had luck with it?

The details

Version

➜  ~ podman exec k0s kubectl version  

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:09:25Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2+k0s", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-10-05T17:58:39Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}

Platform
Which platform did you run k0s on?

➜  ~ cat /etc/os-release 
NAME=Fedora
VERSION="34.20211013.0 (Silverblue)"
ID=fedora
VERSION_ID=34
VERSION_CODENAME=""
PLATFORM_ID="platform:f34"
PRETTY_NAME="Fedora 34.20211013.0 (Silverblue)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:34"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora-silverblue/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=34
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=34
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Silverblue"
VARIANT_ID=silverblue
OSTREE_VERSION='34.20211013.0'
➜  ~ uname -a
Linux work-silverblue 5.14.11-200.fc34.x86_64 #1 SMP Sun Oct 10 14:39:31 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
➜  ~ podman version
Version:      3.4.0
API Version:  3.4.0
Go Version:   go1.16.8
Built:        Thu Sep 30 14:40:21 2021
OS/Arch:      linux/amd64

What happened?
A clear and concise description of what the bug is.

When using Podman to run a single node k0s instance, the kubectl get nodes command returns no resources found.

How To Reproduce
How can we reproduce this issue? (as minimally and as precisely as possible)

➜  ~ podman run -d --name k0s --hostname k0s --privileged -v /var/lib/k0s -p 6443:6443 docker.io/k0sproject/k0s:latest
Trying to pull docker.io/k0sproject/k0s:latest...
Getting image source signatures
Copying blob 2ad3ac81256a done  
Copying blob 617214daaa84 done  
Copying blob 4e9f2cdf4387 done  
Copying blob 5e5a3fa2bef8 done  
Copying blob e46241d59715 done  
Copying config 2ebc8d64d5 done  
Writing manifest to image destination
Storing signatures
96bce16c21118d3f2b8c76adc7ee68e9bc90eb9273b7271887f7aa39c125e7e6

➜  ~ podman exec k0s kubectl get nodes
No resources found

➜  ~ podman exec k0s kubectl version  

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:09:25Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2+k0s", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-10-05T17:58:39Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}

➜  ~ podman exec k0s which kubectl  
/usr/local/bin/kubectl

➜  ~ which kubectl # Local
~/.local/bin/kubectl

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots & Logs
If applicable, add screenshots to help explain your problem.
Also add any output from kubectl if applicable:

I expected to see a table output of at least a single node which is performing the control plane and kubelet function.

Additional context
Add any other context about the problem here.

Podman logs of k0s

➜  ~ podman logs k0s
time="2021-10-15 22:12:14" level=info msg="no config file given, using defaults"
time="2021-10-15 22:12:14" level=info msg="using api address: 10.0.2.100"
time="2021-10-15 22:12:14" level=info msg="using listen port: 6443"
time="2021-10-15 22:12:14" level=info msg="using sans: [10.0.2.100]"
time="2021-10-15 22:12:14" level=info msg="DNS address: 10.96.0.10"
time="2021-10-15 22:12:14" level=info msg="Using storage backend etcd"
time="2021-10-15 22:12:14" level=info msg="initializing Certificates\n"
2021/10/15 22:12:14 [INFO] generate received request
2021/10/15 22:12:14 [INFO] received CSR
2021/10/15 22:12:14 [INFO] generating key: rsa-2048
2021/10/15 22:12:14 [INFO] encoded CSR
2021/10/15 22:12:14 [INFO] signed certificate with serial number 317225729857011839347128928790303712532967401477
2021/10/15 22:12:14 [INFO] generate received request
2021/10/15 22:12:14 [INFO] received CSR
2021/10/15 22:12:14 [INFO] generating key: rsa-2048
2021/10/15 22:12:14 [INFO] generate received request
2021/10/15 22:12:14 [INFO] received CSR
2021/10/15 22:12:14 [INFO] generating key: rsa-2048
2021/10/15 22:12:14 [INFO] generate received request
2021/10/15 22:12:14 [INFO] received CSR
2021/10/15 22:12:14 [INFO] generating key: rsa-2048
2021/10/15 22:12:14 [INFO] generate received request
2021/10/15 22:12:14 [INFO] received CSR
2021/10/15 22:12:14 [INFO] generating key: rsa-2048
2021/10/15 22:12:14 [INFO] generate received request
2021/10/15 22:12:14 [INFO] received CSR
2021/10/15 22:12:14 [INFO] generating key: rsa-2048
2021/10/15 22:12:14 [INFO] generate received request
2021/10/15 22:12:14 [INFO] received CSR
2021/10/15 22:12:14 [INFO] generating key: rsa-2048
2021/10/15 22:12:14 [INFO] generate received request
2021/10/15 22:12:14 [INFO] received CSR
2021/10/15 22:12:14 [INFO] generating key: rsa-2048
2021/10/15 22:12:14 [INFO] generate received request
2021/10/15 22:12:14 [INFO] received CSR
2021/10/15 22:12:14 [INFO] generating key: rsa-2048
2021/10/15 22:12:14 [INFO] encoded CSR
2021/10/15 22:12:14 [INFO] encoded CSR
2021/10/15 22:12:14 [INFO] signed certificate with serial number 337483343134460078200949745178730287746213460518
2021/10/15 22:12:14 [INFO] signed certificate with serial number 687611608232915338688573143556524923079539283172
2021/10/15 22:12:14 [INFO] generate received request
2021/10/15 22:12:14 [INFO] received CSR
2021/10/15 22:12:14 [INFO] generating key: rsa-2048
2021/10/15 22:12:14 [INFO] encoded CSR
2021/10/15 22:12:14 [INFO] signed certificate with serial number 460887382359498560648203578896171924323984927661
2021/10/15 22:12:14 [INFO] encoded CSR
2021/10/15 22:12:14 [INFO] signed certificate with serial number 721665791322024725956400881448722799571183217582
2021/10/15 22:12:14 [INFO] encoded CSR
2021/10/15 22:12:14 [INFO] signed certificate with serial number 291288219078607069400897159299588127474951128850
2021/10/15 22:12:14 [INFO] encoded CSR
2021/10/15 22:12:14 [INFO] signed certificate with serial number 60669938754798521134339623100417981911384099940
2021/10/15 22:12:14 [INFO] encoded CSR
2021/10/15 22:12:14 [INFO] signed certificate with serial number 481765686867269215634992721040740449284346803396
2021/10/15 22:12:14 [INFO] encoded CSR
2021/10/15 22:12:14 [INFO] signed certificate with serial number 501396252662111224587049172859652040341584372151
2021/10/15 22:12:14 [INFO] encoded CSR
2021/10/15 22:12:14 [INFO] signed certificate with serial number 2299292494428530742702855144378745482025446899
time="2021-10-15 22:12:14" level=info msg="initializing Etcd\n"
time="2021-10-15 22:12:14" level=info msg="initializing APIServer\n"
time="2021-10-15 22:12:14" level=info msg="initializing Konnectivity\n"
time="2021-10-15 22:12:14" level=info msg="initializing Scheduler\n"
time="2021-10-15 22:12:14" level=info msg="initializing Manager\n"
time="2021-10-15 22:12:14" level=info msg="initializing DummyLeaderElector\n"
time="2021-10-15 22:12:14" level=info msg="initializing Manager\n"
time="2021-10-15 22:12:14" level=info msg="initializing K0SControlAPI\n"
time="2021-10-15 22:12:14" level=info msg="initializing Component\n"
time="2021-10-15 22:12:14" level=info msg="initializing CSRApprover\n"
time="2021-10-15 22:12:14" level=info msg="initializing Status\n"
time="2021-10-15 22:12:14" level=warning msg="running kube-controller-manager as root: user: unknown user kube-apiserver"
time="2021-10-15 22:12:14" level=warning msg="running kube-apiserver as root: user: unknown user kube-apiserver"
time="2021-10-15 22:12:14" level=info msg="Staging /var/lib/k0s/bin/kube-apiserver"
time="2021-10-15 22:12:14" level=info msg="Staging /var/lib/k0s/bin/kube-controller-manager"
time="2021-10-15 22:12:14" level=info msg="kube client has been init" component=telemetry
time="2021-10-15 22:12:14" level=info msg="segment client has been init" component=telemetry
time="2021-10-15 22:12:14" level=warning msg="running kube-scheduler as root: user: unknown user kube-scheduler"
time="2021-10-15 22:12:14" level=info msg="Staging /var/lib/k0s/bin/kube-scheduler"
time="2021-10-15 22:12:14" level=warning msg="running etcd as root: user: unknown user etcd"
time="2021-10-15 22:12:14" level=warning msg="running konnectivity as root: user: unknown user konnectivity-server"
time="2021-10-15 22:12:14" level=info msg="Staging /var/lib/k0s/bin/etcd"
time="2021-10-15 22:12:14" level=info msg="Listening address /run/k0s/status.sock" component=status
time="2021-10-15 22:12:14" level=info msg="Staging /var/lib/k0s/bin/konnectivity-server"
time="2021-10-15 22:12:15" level=info msg="starting Certificates"
time="2021-10-15 22:12:15" level=info msg="starting Etcd"
time="2021-10-15 22:12:15" level=info msg="Starting etcd"
2021/10/15 22:12:15 [INFO] generate received request
2021/10/15 22:12:15 [INFO] received CSR
2021/10/15 22:12:15 [INFO] generating key: rsa-2048
2021/10/15 22:12:15 [INFO] encoded CSR
2021/10/15 22:12:15 [INFO] signed certificate with serial number 118179885905237296833559983680104488674271504327
2021/10/15 22:12:15 [INFO] generate received request
2021/10/15 22:12:15 [INFO] received CSR
2021/10/15 22:12:15 [INFO] generating key: rsa-2048
2021/10/15 22:12:15 [INFO] generate received request
2021/10/15 22:12:15 [INFO] received CSR
2021/10/15 22:12:15 [INFO] generating key: rsa-2048
2021/10/15 22:12:15 [INFO] generate received request
2021/10/15 22:12:15 [INFO] received CSR
2021/10/15 22:12:15 [INFO] generating key: rsa-2048
2021/10/15 22:12:15 [INFO] encoded CSR
2021/10/15 22:12:15 [INFO] signed certificate with serial number 139288546236620271063841240175795799550108859032
2021/10/15 22:12:15 [INFO] encoded CSR
2021/10/15 22:12:15 [INFO] signed certificate with serial number 244063360960889189012537955354966536276063773836
2021/10/15 22:12:15 [INFO] encoded CSR
2021/10/15 22:12:15 [INFO] signed certificate with serial number 500715861403553904435277705766415678973270703713
time="2021-10-15 22:12:15" level=info msg="starting etcd with args: map[--advertise-client-urls:https://127.0.0.1:2379 --auth-token:jwt,pub-key=/var/lib/k0s/pki/etcd/jwt.pub,priv-key=/var/lib/k0s/pki/etcd/jwt.key,sign-method=RS512,ttl=10m --cert-file:/var/lib/k0s/pki/etcd/server.crt --client-cert-auth:true --data-dir:/var/lib/k0s/etcd --enable-pprof:false --initial-advertise-peer-urls:https://10.0.2.100:2380 --key-file:/var/lib/k0s/pki/etcd/server.key --listen-client-urls:https://127.0.0.1:2379 --listen-peer-urls:https://10.0.2.100:2380 --log-level:info --name:k0s --peer-cert-file:/var/lib/k0s/pki/etcd/peer.crt --peer-client-cert-auth:true --peer-key-file:/var/lib/k0s/pki/etcd/peer.key --peer-trusted-ca-file:/var/lib/k0s/pki/etcd/ca.crt --trusted-ca-file:/var/lib/k0s/pki/etcd/ca.crt]"
time="2021-10-15 22:12:15" level=info msg="Starting to supervise" component=etcd
time="2021-10-15 22:12:15" level=info msg="Started successfully, go nuts" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.609Z\",\"caller\":\"etcdmain/etcd.go:72\",\"msg\":\"Running: \",\"args\":[\"/var/lib/k0s/bin/etcd\",\"--name=k0s\",\"--trusted-ca-file=/var/lib/k0s/pki/etcd/ca.crt\",\"--listen-peer-urls=https://10.0.2.100:2380\",\"--key-file=/var/lib/k0s/pki/etcd/server.key\",\"--peer-trusted-ca-file=/var/lib/k0s/pki/etcd/ca.crt\",\"--listen-client-urls=https://127.0.0.1:2379\",\"--peer-client-cert-auth=true\",\"--data-dir=/var/lib/k0s/etcd\",\"--peer-cert-file=/var/lib/k0s/pki/etcd/peer.crt\",\"--log-level=info\",\"--auth-token=jwt,pub-key=/var/lib/k0s/pki/etcd/jwt.pub,priv-key=/var/lib/k0s/pki/etcd/jwt.key,sign-method=RS512,ttl=10m\",\"--advertise-client-urls=https://127.0.0.1:2379\",\"--cert-file=/var/lib/k0s/pki/etcd/server.crt\",\"--peer-key-file=/var/lib/k0s/pki/etcd/peer.key\",\"--client-cert-auth=true\",\"--enable-pprof=false\",\"--initial-advertise-peer-urls=https://10.0.2.100:2380\"]}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.609Z\",\"caller\":\"embed/etcd.go:131\",\"msg\":\"configuring peer listeners\",\"listen-peer-urls\":[\"https://10.0.2.100:2380\"]}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.609Z\",\"caller\":\"embed/etcd.go:478\",\"msg\":\"starting with peer TLS\",\"tls-info\":\"cert = /var/lib/k0s/pki/etcd/peer.crt, key = /var/lib/k0s/pki/etcd/peer.key, client-cert=, client-key=, trusted-ca = /var/lib/k0s/pki/etcd/ca.crt, client-cert-auth = true, crl-file = \",\"cipher-suites\":[]}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.609Z\",\"caller\":\"embed/etcd.go:139\",\"msg\":\"configuring client listeners\",\"listen-client-urls\":[\"https://127.0.0.1:2379\"]}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.609Z\",\"caller\":\"embed/etcd.go:307\",\"msg\":\"starting an etcd server\",\"etcd-version\":\"3.5.0\",\"git-sha\":\"946a5a6\",\"go-version\":\"go1.16.8\",\"go-os\":\"linux\",\"go-arch\":\"amd64\",\"max-cpu-set\":8,\"max-cpu-available\":8,\"member-initialized\":false,\"name\":\"k0s\",\"data-dir\":\"/var/lib/k0s/etcd\",\"wal-dir\":\"\",\"wal-dir-dedicated\":\"\",\"member-dir\":\"/var/lib/k0s/etcd/member\",\"force-new-cluster\":false,\"heartbeat-interval\":\"100ms\",\"election-timeout\":\"1s\",\"initial-election-tick-advance\":true,\"snapshot-count\":100000,\"snapshot-catchup-entries\":5000,\"initial-advertise-peer-urls\":[\"https://10.0.2.100:2380\"],\"listen-peer-urls\":[\"https://10.0.2.100:2380\"],\"advertise-client-urls\":[\"https://127.0.0.1:2379\"],\"listen-client-urls\":[\"https://127.0.0.1:2379\"],\"listen-metrics-urls\":[],\"cors\":[\"*\"],\"host-whitelist\":[\"*\"],\"initial-cluster\":\"k0s=https://10.0.2.100:2380\",\"initial-cluster-state\":\"new\",\"initial-cluster-token\":\"etcd-cluster\",\"quota-size-bytes\":2147483648,\"pre-vote\":true,\"initial-corrupt-check\":false,\"corrupt-check-time-interval\":\"0s\",\"auto-compaction-mode\":\"periodic\",\"auto-compaction-retention\":\"0s\",\"auto-compaction-interval\":\"0s\",\"discovery-url\":\"\",\"discovery-proxy\":\"\",\"downgrade-check-interval\":\"5s\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.613Z\",\"caller\":\"etcdserver/backend.go:81\",\"msg\":\"opened backend db\",\"path\":\"/var/lib/k0s/etcd/member/snap/db\",\"took\":\"3.85097ms\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.618Z\",\"caller\":\"etcdserver/raft.go:448\",\"msg\":\"starting local member\",\"local-member-id\":\"40aeedb3ea706971\",\"cluster-id\":\"8ddb5e0761f442f7\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.618Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 switched to configuration voters=()\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.618Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 became follower at term 0\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.618Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"newRaft 40aeedb3ea706971 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.618Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 became follower at term 1\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.618Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 switched to configuration voters=(4660924021363206513)\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.625Z\",\"caller\":\"mvcc/kvstore.go:415\",\"msg\":\"kvstore restored\",\"current-rev\":1}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.628Z\",\"caller\":\"etcdserver/quota.go:94\",\"msg\":\"enabled backend quota with default value\",\"quota-name\":\"v3-applier\",\"quota-size-bytes\":2147483648,\"quota-size\":\"2.1 GB\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.632Z\",\"caller\":\"etcdserver/server.go:843\",\"msg\":\"starting etcd server\",\"local-member-id\":\"40aeedb3ea706971\",\"local-server-version\":\"3.5.0\",\"cluster-version\":\"to_be_decided\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.633Z\",\"caller\":\"etcdserver/server.go:728\",\"msg\":\"started as single-node; fast-forwarding election ticks\",\"local-member-id\":\"40aeedb3ea706971\",\"forward-ticks\":9,\"forward-duration\":\"900ms\",\"election-ticks\":10,\"election-timeout\":\"1s\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.633Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 switched to configuration voters=(4660924021363206513)\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.633Z\",\"caller\":\"membership/cluster.go:393\",\"msg\":\"added member\",\"cluster-id\":\"8ddb5e0761f442f7\",\"local-member-id\":\"40aeedb3ea706971\",\"added-peer-id\":\"40aeedb3ea706971\",\"added-peer-peer-urls\":[\"https://10.0.2.100:2380\"]}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.634Z\",\"caller\":\"embed/etcd.go:687\",\"msg\":\"starting with client TLS\",\"tls-info\":\"cert = /var/lib/k0s/pki/etcd/server.crt, key = /var/lib/k0s/pki/etcd/server.key, client-cert=, client-key=, trusted-ca = /var/lib/k0s/pki/etcd/ca.crt, client-cert-auth = true, crl-file = \",\"cipher-suites\":[]}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.634Z\",\"caller\":\"embed/etcd.go:276\",\"msg\":\"now serving peer/client/metrics\",\"local-member-id\":\"40aeedb3ea706971\",\"initial-advertise-peer-urls\":[\"https://10.0.2.100:2380\"],\"listen-peer-urls\":[\"https://10.0.2.100:2380\"],\"advertise-client-urls\":[\"https://127.0.0.1:2379\"],\"listen-client-urls\":[\"https://127.0.0.1:2379\"],\"listen-metrics-urls\":[]}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.634Z\",\"caller\":\"embed/etcd.go:580\",\"msg\":\"serving peer traffic\",\"address\":\"10.0.2.100:2380\"}" component=etcd
time="2021-10-15 22:12:15" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:15.634Z\",\"caller\":\"embed/etcd.go:552\",\"msg\":\"cmux::serve\",\"address\":\"10.0.2.100:2380\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.318Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 is starting a new election at term 1\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.318Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 became pre-candidate at term 1\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.318Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 received MsgPreVoteResp from 40aeedb3ea706971 at term 1\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.318Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 became candidate at term 2\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.319Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 received MsgVoteResp from 40aeedb3ea706971 at term 2\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.319Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"40aeedb3ea706971 became leader at term 2\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.319Z\",\"logger\":\"raft\",\"caller\":\"etcdserver/zap_raft.go:77\",\"msg\":\"raft.node: 40aeedb3ea706971 elected leader 40aeedb3ea706971 at term 2\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.320Z\",\"caller\":\"etcdserver/server.go:2476\",\"msg\":\"setting up initial cluster version using v2 API\",\"cluster-version\":\"3.5\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.321Z\",\"caller\":\"etcdserver/server.go:2027\",\"msg\":\"published local member to cluster through raft\",\"local-member-id\":\"40aeedb3ea706971\",\"local-member-attributes\":\"{Name:k0s ClientURLs:[https://127.0.0.1:2379]}\",\"request-path\":\"/0/members/40aeedb3ea706971/attributes\",\"cluster-id\":\"8ddb5e0761f442f7\",\"publish-timeout\":\"7s\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.321Z\",\"caller\":\"embed/serve.go:98\",\"msg\":\"ready to serve client requests\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.321Z\",\"caller\":\"etcdmain/main.go:47\",\"msg\":\"notifying init daemon\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.322Z\",\"caller\":\"etcdmain/main.go:53\",\"msg\":\"successfully notified init daemon\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.325Z\",\"caller\":\"membership/cluster.go:531\",\"msg\":\"set initial cluster version\",\"cluster-id\":\"8ddb5e0761f442f7\",\"local-member-id\":\"40aeedb3ea706971\",\"cluster-version\":\"3.5\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.325Z\",\"caller\":\"api/capability.go:75\",\"msg\":\"enabled capabilities for version\",\"cluster-version\":\"3.5\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.325Z\",\"caller\":\"etcdserver/server.go:2500\",\"msg\":\"cluster version is updated\",\"cluster-version\":\"3.5\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="{\"level\":\"info\",\"ts\":\"2021-10-15T22:12:16.327Z\",\"caller\":\"embed/serve.go:188\",\"msg\":\"serving client traffic securely\",\"address\":\"127.0.0.1:2379\"}" component=etcd
time="2021-10-15 22:12:16" level=info msg="starting APIServer"
time="2021-10-15 22:12:16" level=info msg="Starting kube-apiserver"
time="2021-10-15 22:12:16" level=info msg="Starting to supervise" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="Started successfully, go nuts" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="Flag --insecure-port has been deprecated, This flag has no effect now and will be removed in v1.24." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.379776      38 flags.go:59] FLAG: --add-dir-header=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.379867      38 flags.go:59] FLAG: --address=\"127.0.0.1\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.379906      38 flags.go:59] FLAG: --admission-control=\"[]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.379950      38 flags.go:59] FLAG: --admission-control-config-file=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.379978      38 flags.go:59] FLAG: --advertise-address=\"10.0.2.100\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380001      38 flags.go:59] FLAG: --allow-metric-labels=\"[]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380037      38 flags.go:59] FLAG: --allow-privileged=\"true\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380069      38 flags.go:59] FLAG: --alsologtostderr=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380090      38 flags.go:59] FLAG: --anonymous-auth=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380121      38 flags.go:59] FLAG: --api-audiences=\"[https://kubernetes.default.svc,system:konnectivity-server]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380152      38 flags.go:59] FLAG: --apiserver-count=\"1\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380175      38 flags.go:59] FLAG: --audit-log-batch-buffer-size=\"10000\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380206      38 flags.go:59] FLAG: --audit-log-batch-max-size=\"1\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380232      38 flags.go:59] FLAG: --audit-log-batch-max-wait=\"0s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380255      38 flags.go:59] FLAG: --audit-log-batch-throttle-burst=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380284      38 flags.go:59] FLAG: --audit-log-batch-throttle-enable=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380322      38 flags.go:59] FLAG: --audit-log-batch-throttle-qps=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380348      38 flags.go:59] FLAG: --audit-log-compress=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380371      38 flags.go:59] FLAG: --audit-log-format=\"json\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380376      38 flags.go:59] FLAG: --audit-log-maxage=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380379      38 flags.go:59] FLAG: --audit-log-maxbackup=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380381      38 flags.go:59] FLAG: --audit-log-maxsize=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380383      38 flags.go:59] FLAG: --audit-log-mode=\"blocking\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380385      38 flags.go:59] FLAG: --audit-log-path=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380397      38 flags.go:59] FLAG: --audit-log-truncate-enabled=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380399      38 flags.go:59] FLAG: --audit-log-truncate-max-batch-size=\"10485760\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380403      38 flags.go:59] FLAG: --audit-log-truncate-max-event-size=\"102400\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380405      38 flags.go:59] FLAG: --audit-log-version=\"audit.k8s.io/v1\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380408      38 flags.go:59] FLAG: --audit-policy-file=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380410      38 flags.go:59] FLAG: --audit-webhook-batch-buffer-size=\"10000\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380412      38 flags.go:59] FLAG: --audit-webhook-batch-initial-backoff=\"10s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380414      38 flags.go:59] FLAG: --audit-webhook-batch-max-size=\"400\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380416      38 flags.go:59] FLAG: --audit-webhook-batch-max-wait=\"30s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380418      38 flags.go:59] FLAG: --audit-webhook-batch-throttle-burst=\"15\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380421      38 flags.go:59] FLAG: --audit-webhook-batch-throttle-enable=\"true\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380423      38 flags.go:59] FLAG: --audit-webhook-batch-throttle-qps=\"10\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380425      38 flags.go:59] FLAG: --audit-webhook-config-file=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380428      38 flags.go:59] FLAG: --audit-webhook-initial-backoff=\"10s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380430      38 flags.go:59] FLAG: --audit-webhook-mode=\"batch\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380433      38 flags.go:59] FLAG: --audit-webhook-truncate-enabled=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380435      38 flags.go:59] FLAG: --audit-webhook-truncate-max-batch-size=\"10485760\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380437      38 flags.go:59] FLAG: --audit-webhook-truncate-max-event-size=\"102400\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380439      38 flags.go:59] FLAG: --audit-webhook-version=\"audit.k8s.io/v1\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380441      38 flags.go:59] FLAG: --authentication-token-webhook-cache-ttl=\"2m0s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380443      38 flags.go:59] FLAG: --authentication-token-webhook-config-file=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380445      38 flags.go:59] FLAG: --authentication-token-webhook-version=\"v1beta1\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380447      38 flags.go:59] FLAG: --authorization-mode=\"[Node,RBAC]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380451      38 flags.go:59] FLAG: --authorization-policy-file=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380453      38 flags.go:59] FLAG: --authorization-webhook-cache-authorized-ttl=\"5m0s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380455      38 flags.go:59] FLAG: --authorization-webhook-cache-unauthorized-ttl=\"30s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380457      38 flags.go:59] FLAG: --authorization-webhook-config-file=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380459      38 flags.go:59] FLAG: --authorization-webhook-version=\"v1beta1\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380477      38 flags.go:59] FLAG: --bind-address=\"0.0.0.0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380489      38 flags.go:59] FLAG: --cert-dir=\"/var/run/kubernetes\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380492      38 flags.go:59] FLAG: --client-ca-file=\"/var/lib/k0s/pki/ca.crt\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380495      38 flags.go:59] FLAG: --cloud-config=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380496      38 flags.go:59] FLAG: --cloud-provider=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380498      38 flags.go:59] FLAG: --contention-profiling=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380500      38 flags.go:59] FLAG: --cors-allowed-origins=\"[]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380504      38 flags.go:59] FLAG: --default-not-ready-toleration-seconds=\"300\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380506      38 flags.go:59] FLAG: --default-unreachable-toleration-seconds=\"300\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380508      38 flags.go:59] FLAG: --default-watch-cache-size=\"100\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380511      38 flags.go:59] FLAG: --delete-collection-workers=\"1\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380513      38 flags.go:59] FLAG: --deserialization-cache-size=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380515      38 flags.go:59] FLAG: --disable-admission-plugins=\"[]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380518      38 flags.go:59] FLAG: --disabled-metrics=\"[]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380521      38 flags.go:59] FLAG: --egress-selector-config-file=\"/var/lib/k0s/konnectivity.conf\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380526      38 flags.go:59] FLAG: --enable-admission-plugins=\"[NodeRestriction,PodSecurityPolicy]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380530      38 flags.go:59] FLAG: --enable-aggregator-routing=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380532      38 flags.go:59] FLAG: --enable-bootstrap-token-auth=\"true\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380552      38 flags.go:59] FLAG: --enable-garbage-collector=\"true\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380561      38 flags.go:59] FLAG: --enable-logs-handler=\"true\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380569      38 flags.go:59] FLAG: --enable-priority-and-fairness=\"true\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380576      38 flags.go:59] FLAG: --enable-swagger-ui=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380583      38 flags.go:59] FLAG: --encryption-provider-config=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380590      38 flags.go:59] FLAG: --endpoint-reconciler-type=\"lease\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380598      38 flags.go:59] FLAG: --etcd-cafile=\"/var/lib/k0s/pki/etcd/ca.crt\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380601      38 flags.go:59] FLAG: --etcd-certfile=\"/var/lib/k0s/pki/apiserver-etcd-client.crt\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380604      38 flags.go:59] FLAG: --etcd-compaction-interval=\"5m0s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380606      38 flags.go:59] FLAG: --etcd-count-metric-poll-period=\"1m0s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380611      38 flags.go:59] FLAG: --etcd-db-metric-poll-interval=\"30s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380613      38 flags.go:59] FLAG: --etcd-healthcheck-timeout=\"2s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380615      38 flags.go:59] FLAG: --etcd-keyfile=\"/var/lib/k0s/pki/apiserver-etcd-client.key\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380632      38 flags.go:59] FLAG: --etcd-prefix=\"/registry\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380637      38 flags.go:59] FLAG: --etcd-servers=\"[https://127.0.0.1:2379]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380641      38 flags.go:59] FLAG: --etcd-servers-overrides=\"[]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380647      38 flags.go:59] FLAG: --event-ttl=\"1h0m0s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380649      38 flags.go:59] FLAG: --experimental-encryption-provider-config=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380653      38 flags.go:59] FLAG: --experimental-logging-sanitization=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380655      38 flags.go:59] FLAG: --external-hostname=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380658      38 flags.go:59] FLAG: --feature-gates=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380674      38 flags.go:59] FLAG: --goaway-chance=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380679      38 flags.go:59] FLAG: --help=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380705      38 flags.go:59] FLAG: --http2-max-streams-per-connection=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380714      38 flags.go:59] FLAG: --identity-lease-duration-seconds=\"3600\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380722      38 flags.go:59] FLAG: --identity-lease-renew-interval-seconds=\"10\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380729      38 flags.go:59] FLAG: --insecure-bind-address=\"127.0.0.1\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380737      38 flags.go:59] FLAG: --insecure-port=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380745      38 flags.go:59] FLAG: --kubelet-certificate-authority=\"/var/lib/k0s/pki/ca.crt\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380766      38 flags.go:59] FLAG: --kubelet-client-certificate=\"/var/lib/k0s/pki/apiserver-kubelet-client.crt\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380772      38 flags.go:59] FLAG: --kubelet-client-key=\"/var/lib/k0s/pki/apiserver-kubelet-client.key\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380774      38 flags.go:59] FLAG: --kubelet-port=\"10250\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380778      38 flags.go:59] FLAG: --kubelet-preferred-address-types=\"[InternalIP,ExternalIP,Hostname]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380783      38 flags.go:59] FLAG: --kubelet-read-only-port=\"10255\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380785      38 flags.go:59] FLAG: --kubelet-timeout=\"5s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380788      38 flags.go:59] FLAG: --kubernetes-service-node-port=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380790      38 flags.go:59] FLAG: --lease-reuse-duration-seconds=\"60\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380793      38 flags.go:59] FLAG: --livez-grace-period=\"0s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380805      38 flags.go:59] FLAG: --log-backtrace-at=\":0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380823      38 flags.go:59] FLAG: --log-dir=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380840      38 flags.go:59] FLAG: --log-file=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380856      38 flags.go:59] FLAG: --log-file-max-size=\"1800\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380864      38 flags.go:59] FLAG: --log-flush-frequency=\"5s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380871      38 flags.go:59] FLAG: --logging-format=\"text\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380879      38 flags.go:59] FLAG: --logtostderr=\"true\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380881      38 flags.go:59] FLAG: --master-service-namespace=\"default\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380885      38 flags.go:59] FLAG: --max-connection-bytes-per-sec=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380901      38 flags.go:59] FLAG: --max-mutating-requests-inflight=\"200\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380903      38 flags.go:59] FLAG: --max-requests-inflight=\"400\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380919      38 flags.go:59] FLAG: --min-request-timeout=\"1800\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380929      38 flags.go:59] FLAG: --oidc-ca-file=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380934      38 flags.go:59] FLAG: --oidc-client-id=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380942      38 flags.go:59] FLAG: --oidc-groups-claim=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380945      38 flags.go:59] FLAG: --oidc-groups-prefix=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380947      38 flags.go:59] FLAG: --oidc-issuer-url=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380949      38 flags.go:59] FLAG: --oidc-required-claim=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380955      38 flags.go:59] FLAG: --oidc-signing-algs=\"[RS256]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380960      38 flags.go:59] FLAG: --oidc-username-claim=\"sub\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380964      38 flags.go:59] FLAG: --oidc-username-prefix=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380967      38 flags.go:59] FLAG: --one-output=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380976      38 flags.go:59] FLAG: --permit-address-sharing=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380991      38 flags.go:59] FLAG: --permit-port-sharing=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.380999      38 flags.go:59] FLAG: --port=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381008      38 flags.go:59] FLAG: --profiling=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381015      38 flags.go:59] FLAG: --proxy-client-cert-file=\"/var/lib/k0s/pki/front-proxy-client.crt\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381022      38 flags.go:59] FLAG: --proxy-client-key-file=\"/var/lib/k0s/pki/front-proxy-client.key\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381030      38 flags.go:59] FLAG: --request-timeout=\"1m0s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381080      38 flags.go:59] FLAG: --requestheader-allowed-names=\"[front-proxy-client]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381087      38 flags.go:59] FLAG: --requestheader-client-ca-file=\"/var/lib/k0s/pki/front-proxy-ca.crt\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381090      38 flags.go:59] FLAG: --requestheader-extra-headers-prefix=\"[X-Remote-Extra-]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381107      38 flags.go:59] FLAG: --requestheader-group-headers=\"[X-Remote-Group]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381110      38 flags.go:59] FLAG: --requestheader-username-headers=\"[X-Remote-User]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381113      38 flags.go:59] FLAG: --runtime-config=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381117      38 flags.go:59] FLAG: --secure-port=\"6443\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381119      38 flags.go:59] FLAG: --service-account-api-audiences=\"[https://kubernetes.default.svc,system:konnectivity-server]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381122      38 flags.go:59] FLAG: --service-account-extend-token-expiration=\"true\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381125      38 flags.go:59] FLAG: --service-account-issuer=\"[https://kubernetes.default.svc]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381129      38 flags.go:59] FLAG: --service-account-jwks-uri=\"https://kubernetes.default.svc/openid/v1/jwks\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381131      38 flags.go:59] FLAG: --service-account-key-file=\"[/var/lib/k0s/pki/sa.pub]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381133      38 flags.go:59] FLAG: --service-account-lookup=\"true\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381135      38 flags.go:59] FLAG: --service-account-max-token-expiration=\"0s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381137      38 flags.go:59] FLAG: --service-account-signing-key-file=\"/var/lib/k0s/pki/sa.key\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381139      38 flags.go:59] FLAG: --service-cluster-ip-range=\"10.96.0.0/12\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381141      38 flags.go:59] FLAG: --service-node-port-range=\"30000-32767\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381145      38 flags.go:59] FLAG: --show-hidden-metrics-for-version=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381147      38 flags.go:59] FLAG: --shutdown-delay-duration=\"0s\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381149      38 flags.go:59] FLAG: --skip-headers=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381151      38 flags.go:59] FLAG: --skip-log-headers=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381153      38 flags.go:59] FLAG: --stderrthreshold=\"2\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381167      38 flags.go:59] FLAG: --storage-backend=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381176      38 flags.go:59] FLAG: --storage-media-type=\"application/vnd.kubernetes.protobuf\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381179      38 flags.go:59] FLAG: --strict-transport-security-directives=\"[]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381182      38 flags.go:59] FLAG: --target-ram-mb=\"0\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381184      38 flags.go:59] FLAG: --tls-cert-file=\"/var/lib/k0s/pki/server.crt\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381186      38 flags.go:59] FLAG: --tls-cipher-suites=\"[TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381193      38 flags.go:59] FLAG: --tls-min-version=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381195      38 flags.go:59] FLAG: --tls-private-key-file=\"/var/lib/k0s/pki/server.key\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381197      38 flags.go:59] FLAG: --tls-sni-cert-key=\"[]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381200      38 flags.go:59] FLAG: --token-auth-file=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381202      38 flags.go:59] FLAG: --tracing-config-file=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381204      38 flags.go:59] FLAG: --v=\"1\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381206      38 flags.go:59] FLAG: --version=\"false\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381210      38 flags.go:59] FLAG: --vmodule=\"\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381212      38 flags.go:59] FLAG: --watch-cache=\"true\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381214      38 flags.go:59] FLAG: --watch-cache-sizes=\"[]\"" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381259      38 server.go:553] external host was not specified, using 10.0.2.100" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.381585      38 server.go:161] Version: v1.22.2+k0s" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.767632      38 shared_informer.go:240] Waiting for caches to sync for node_authorizer" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.768245      38 plugins.go:158] Loaded 13 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,PodSecurityPolicy,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.768256      38 plugins.go:161] Loaded 12 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurityPolicy,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.768918      38 plugins.go:158] Loaded 13 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,PodSecurityPolicy,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.768928      38 plugins.go:161] Loaded 12 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurityPolicy,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.782143      38 cacher.go:406] cacher (*apiextensions.CustomResourceDefinition): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="W1015 22:12:16.792487      38 genericapiserver.go:455] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.793043      38 instance.go:278] Using reconciler: lease" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.807143      38 cacher.go:406] cacher (*core.PodTemplate): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.817438      38 cacher.go:406] cacher (*core.LimitRange): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.821644      38 cacher.go:406] cacher (*core.ResourceQuota): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.826283      38 cacher.go:406] cacher (*core.Secret): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.830680      38 cacher.go:406] cacher (*core.PersistentVolume): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.834582      38 cacher.go:406] cacher (*core.PersistentVolumeClaim): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.839371      38 cacher.go:406] cacher (*core.ConfigMap): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.843614      38 cacher.go:406] cacher (*core.Namespace): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.847761      38 cacher.go:406] cacher (*core.Endpoints): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.852112      38 cacher.go:406] cacher (*core.Node): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.856860      38 cacher.go:406] cacher (*core.Pod): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.861562      38 cacher.go:406] cacher (*core.ServiceAccount): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.873991      38 cacher.go:406] cacher (*core.ReplicationController): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.877331      38 rest.go:130] the default service ipfamily for this cluster is: IPv4" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.878078      38 cacher.go:406] cacher (*core.Service): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.923537      38 instance.go:565] Skipping disabled API group \"internal.apiserver.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.923580      38 deleted_kinds.go:174] Removing resource tokenreviews.v1beta1.authentication.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.923591      38 deleted_kinds.go:185] Removing version v1beta1.authentication.k8s.io because it is time to stop serving it because it has no resources per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.923594      38 instance.go:586] Enabling API group \"authentication.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.923621      38 deleted_kinds.go:174] Removing resource subjectaccessreviews.v1beta1.authorization.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.923624      38 deleted_kinds.go:174] Removing resource selfsubjectaccessreviews.v1beta1.authorization.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.923626      38 deleted_kinds.go:174] Removing resource localsubjectaccessreviews.v1beta1.authorization.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.923628      38 deleted_kinds.go:174] Removing resource selfsubjectrulesreviews.v1beta1.authorization.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.923652      38 deleted_kinds.go:185] Removing version v1beta1.authorization.k8s.io because it is time to stop serving it because it has no resources per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.923657      38 instance.go:586] Enabling API group \"authorization.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.928725      38 cacher.go:406] cacher (*autoscaling.HorizontalPodAutoscaler): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.933571      38 cacher.go:406] cacher (*autoscaling.HorizontalPodAutoscaler): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.939287      38 instance.go:586] Enabling API group \"autoscaling\"." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.939637      38 cacher.go:406] cacher (*autoscaling.HorizontalPodAutoscaler): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.944373      38 cacher.go:406] cacher (*batch.Job): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.948390      38 cacher.go:406] cacher (*batch.CronJob): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.952724      38 instance.go:586] Enabling API group \"batch\"." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.953182      38 cacher.go:406] cacher (*batch.CronJob): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.957992      38 cacher.go:406] cacher (*certificates.CertificateSigningRequest): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.961704      38 deleted_kinds.go:174] Removing resource certificatesigningrequests.v1beta1.certificates.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.961716      38 deleted_kinds.go:174] Removing resource certificatesigningrequests/status.v1beta1.certificates.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.961719      38 deleted_kinds.go:174] Removing resource certificatesigningrequests/approval.v1beta1.certificates.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.961727      38 deleted_kinds.go:185] Removing version v1beta1.certificates.k8s.io because it is time to stop serving it because it has no resources per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.961730      38 instance.go:586] Enabling API group \"certificates.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.962108      38 cacher.go:406] cacher (*certificates.CertificateSigningRequest): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.966248      38 cacher.go:406] cacher (*coordination.Lease): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.970326      38 deleted_kinds.go:174] Removing resource leases.v1beta1.coordination.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.970338      38 deleted_kinds.go:185] Removing version v1beta1.coordination.k8s.io because it is time to stop serving it because it has no resources per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.970342      38 instance.go:586] Enabling API group \"coordination.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.970725      38 cacher.go:406] cacher (*coordination.Lease): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.975131      38 cacher.go:406] cacher (*discovery.EndpointSlice): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.979078      38 instance.go:586] Enabling API group \"discovery.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.979468      38 cacher.go:406] cacher (*discovery.EndpointSlice): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.983158      38 deleted_kinds.go:174] Removing resource ingresses.v1beta1.extensions because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.983167      38 deleted_kinds.go:174] Removing resource ingresses/status.v1beta1.extensions because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.983171      38 deleted_kinds.go:185] Removing version v1beta1.extensions because it is time to stop serving it because it has no resources per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.983174      38 instance.go:582] Removing API group extensions because it is time to stop serving it because it has no versions per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.983566      38 cacher.go:406] cacher (*networking.Ingress): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.987686      38 cacher.go:406] cacher (*networking.NetworkPolicy): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.992164      38 cacher.go:406] cacher (*networking.Ingress): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.996183      38 cacher.go:406] cacher (*networking.IngressClass): initialized" component=kube-apiserver
time="2021-10-15 22:12:16" level=info msg="I1015 22:12:16.999931      38 cacher.go:406] cacher (*networking.Ingress): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.003952      38 deleted_kinds.go:174] Removing resource ingressclasses.v1beta1.networking.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.003962      38 deleted_kinds.go:174] Removing resource ingresses.v1beta1.networking.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.003965      38 deleted_kinds.go:174] Removing resource ingresses/status.v1beta1.networking.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.003968      38 deleted_kinds.go:185] Removing version v1beta1.networking.k8s.io because it is time to stop serving it because it has no resources per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.003971      38 instance.go:586] Enabling API group \"networking.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.004743      38 cacher.go:406] cacher (*networking.IngressClass): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.008798      38 cacher.go:406] cacher (*node.RuntimeClass): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.012352      38 instance.go:586] Enabling API group \"node.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.012858      38 cacher.go:406] cacher (*node.RuntimeClass): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.017049      38 cacher.go:406] cacher (*policy.PodDisruptionBudget): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.021897      38 cacher.go:406] cacher (*policy.PodSecurityPolicy): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.025453      38 instance.go:586] Enabling API group \"policy\"." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.025870      38 cacher.go:406] cacher (*policy.PodDisruptionBudget): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.029987      38 cacher.go:406] cacher (*rbac.Role): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.034252      38 cacher.go:406] cacher (*rbac.RoleBinding): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.039052      38 cacher.go:406] cacher (*rbac.ClusterRole): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.064225      38 cacher.go:406] cacher (*rbac.ClusterRoleBinding): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.077418      38 cacher.go:406] cacher (*rbac.Role): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.093251      38 cacher.go:406] cacher (*rbac.RoleBinding): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.102980      38 cacher.go:406] cacher (*rbac.ClusterRole): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.110328      38 deleted_kinds.go:174] Removing resource roles.v1beta1.rbac.authorization.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.110347      38 deleted_kinds.go:174] Removing resource rolebindings.v1beta1.rbac.authorization.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.110351      38 deleted_kinds.go:174] Removing resource clusterroles.v1beta1.rbac.authorization.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.110355      38 deleted_kinds.go:174] Removing resource clusterrolebindings.v1beta1.rbac.authorization.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.110377      38 deleted_kinds.go:185] Removing version v1beta1.rbac.authorization.k8s.io because it is time to stop serving it because it has no resources per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.110384      38 instance.go:586] Enabling API group \"rbac.authorization.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.111066      38 cacher.go:406] cacher (*rbac.ClusterRoleBinding): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.121270      38 cacher.go:406] cacher (*scheduling.PriorityClass): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.125901      38 deleted_kinds.go:174] Removing resource priorityclasses.v1beta1.scheduling.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.125920      38 deleted_kinds.go:185] Removing version v1beta1.scheduling.k8s.io because it is time to stop serving it because it has no resources per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.125926      38 instance.go:586] Enabling API group \"scheduling.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.126540      38 cacher.go:406] cacher (*scheduling.PriorityClass): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.132806      38 cacher.go:406] cacher (*storage.StorageClass): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.145937      38 cacher.go:406] cacher (*storage.VolumeAttachment): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.150147      38 cacher.go:406] cacher (*storage.CSINode): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.155521      38 cacher.go:406] cacher (*storage.CSIDriver): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.159696      38 cacher.go:406] cacher (*storage.CSIStorageCapacity): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.164988      38 cacher.go:406] cacher (*storage.StorageClass): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.169205      38 cacher.go:406] cacher (*storage.VolumeAttachment): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.173346      38 cacher.go:406] cacher (*storage.CSINode): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.177198      38 deleted_kinds.go:174] Removing resource csinodes.v1beta1.storage.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.177245      38 deleted_kinds.go:174] Removing resource csidrivers.v1beta1.storage.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.177255      38 deleted_kinds.go:174] Removing resource storageclasses.v1beta1.storage.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.177261      38 deleted_kinds.go:174] Removing resource volumeattachments.v1beta1.storage.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.177271      38 instance.go:586] Enabling API group \"storage.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.177772      38 cacher.go:406] cacher (*storage.CSIDriver): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.185444      38 cacher.go:406] cacher (*flowcontrol.FlowSchema): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.190703      38 instance.go:586] Enabling API group \"flowcontrol.apiserver.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.191085      38 cacher.go:406] cacher (*flowcontrol.PriorityLevelConfiguration): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.195845      38 cacher.go:406] cacher (*apps.Deployment): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.199884      38 cacher.go:406] cacher (*apps.StatefulSet): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.204402      38 cacher.go:406] cacher (*apps.DaemonSet): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.208714      38 cacher.go:406] cacher (*apps.ReplicaSet): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.212936      38 instance.go:586] Enabling API group \"apps\"." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.213408      38 cacher.go:406] cacher (*apps.ControllerRevision): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.218173      38 cacher.go:406] cacher (*admissionregistration.ValidatingWebhookConfiguration): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.222042      38 cacher.go:406] cacher (*admissionregistration.MutatingWebhookConfiguration): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.226646      38 cacher.go:406] cacher (*admissionregistration.ValidatingWebhookConfiguration): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.230496      38 deleted_kinds.go:174] Removing resource validatingwebhookconfigurations.v1beta1.admissionregistration.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.230506      38 deleted_kinds.go:174] Removing resource mutatingwebhookconfigurations.v1beta1.admissionregistration.k8s.io because it is time to stop serving it per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.230509      38 deleted_kinds.go:185] Removing version v1beta1.admissionregistration.k8s.io because it is time to stop serving it because it has no resources per APILifecycle." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.230513      38 instance.go:586] Enabling API group \"admissionregistration.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.230940      38 cacher.go:406] cacher (*admissionregistration.MutatingWebhookConfiguration): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.238905      38 instance.go:586] Enabling API group \"events.k8s.io\"." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.297804      38 genericapiserver.go:455] Skipping API authentication.k8s.io/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.298981      38 genericapiserver.go:455] Skipping API authorization.k8s.io/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.307897      38 genericapiserver.go:455] Skipping API certificates.k8s.io/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.309164      38 genericapiserver.go:455] Skipping API coordination.k8s.io/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.313660      38 genericapiserver.go:455] Skipping API networking.k8s.io/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.316014      38 genericapiserver.go:455] Skipping API node.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.321183      38 genericapiserver.go:455] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.321196      38 genericapiserver.go:455] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.322397      38 genericapiserver.go:455] Skipping API scheduling.k8s.io/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.322408      38 genericapiserver.go:455] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.325788      38 genericapiserver.go:455] Skipping API storage.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.327534      38 genericapiserver.go:455] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.331270      38 genericapiserver.go:455] Skipping API apps/v1beta2 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.331281      38 genericapiserver.go:455] Skipping API apps/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.332833      38 genericapiserver.go:455] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.335913      38 plugins.go:158] Loaded 13 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,PodSecurityPolicy,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.335922      38 plugins.go:161] Loaded 12 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurityPolicy,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota." component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="I1015 22:12:17.341576      38 cacher.go:406] cacher (*apiregistration.APIService): initialized" component=kube-apiserver
time="2021-10-15 22:12:17" level=info msg="W1015 22:12:17.351142      38 genericapiserver.go:455] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources." component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.522610      38 genericapiserver.go:419] [graceful-termination] ShutdownTimeout=1m0s" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.522733      38 dynamic_cafile_content.go:155] \"Starting controller\" name=\"request-header::/var/lib/k0s/pki/front-proxy-ca.crt\"" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.522787      38 dynamic_cafile_content.go:155] \"Starting controller\" name=\"client-ca-bundle::/var/lib/k0s/pki/ca.crt\"" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523034      38 dynamic_serving_content.go:129] \"Starting controller\" name=\"serving-cert::/var/lib/k0s/pki/server.crt::/var/lib/k0s/pki/server.key\"" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523205      38 secure_serving.go:266] Serving securely on [::]:6443" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523228      38 genericapiserver.go:378] [graceful-termination] waiting for shutdown to be initiated" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523239      38 tlsconfig.go:240] \"Starting DynamicServingCertificateController\"" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523301      38 apiservice_controller.go:97] Starting APIServiceRegistrationController" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523309      38 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523364      38 customresource_discovery_controller.go:209] Starting DiscoveryController" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523379      38 autoregister_controller.go:141] Starting autoregister controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523382      38 cache.go:32] Waiting for caches to sync for autoregister controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523390      38 apf_controller.go:299] Starting API Priority and Fairness config controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523588      38 establishing_controller.go:76] Starting EstablishingController" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523724      38 dynamic_serving_content.go:129] \"Starting controller\" name=\"aggregator-proxy-cert::/var/lib/k0s/pki/front-proxy-client.crt::/var/lib/k0s/pki/front-proxy-client.key\"" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523763      38 controller.go:83] Starting OpenAPI AggregationController" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523795      38 controller.go:85] Starting OpenAPI controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.523805      38 naming_controller.go:291] Starting NamingConditionController" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.524314      38 available_controller.go:491] Starting AvailableConditionController" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.524323      38 cache.go:32] Waiting for caches to sync for AvailableConditionController controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.524654      38 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.524666      38 shared_informer.go:240] Waiting for caches to sync for cluster_authentication_trust_controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.524737      38 dynamic_cafile_content.go:155] \"Starting controller\" name=\"client-ca-bundle::/var/lib/k0s/pki/ca.crt\"" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.524758      38 dynamic_cafile_content.go:155] \"Starting controller\" name=\"request-header::/var/lib/k0s/pki/front-proxy-ca.crt\"" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.525394      38 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.525404      38 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.525412      38 crd_finalizer.go:266] Starting CRDFinalizer" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.525420      38 crdregistration_controller.go:111] Starting crd-autoregister controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.525423      38 shared_informer.go:240] Waiting for caches to sync for crd-autoregister" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="E1015 22:12:18.532501      38 controller.go:152] Unable to remove old endpoints from kubernetes service: StorageError: key not found, Code: 1, Key: /registry/masterleases/10.0.2.100, ResourceVersion: 0, AdditionalErrorMsg: " component=kube-apiserver
time="2021-10-15 22:12:18" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:18" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.543957      38 controller.go:611] quota admission added evaluator for: namespaces" component=kube-apiserver
time="2021-10-15 22:12:18" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.567732      38 shared_informer.go:247] Caches are synced for node_authorizer " component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.623498      38 apf_controller.go:304] Running API Priority and Fairness config worker" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.623522      38 cache.go:39] Caches are synced for autoregister controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.623547      38 cache.go:39] Caches are synced for APIServiceRegistrationController controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.624345      38 cache.go:39] Caches are synced for AvailableConditionController controller" component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.624767      38 shared_informer.go:247] Caches are synced for cluster_authentication_trust_controller " component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.625451      38 shared_informer.go:247] Caches are synced for crd-autoregister " component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.627858      38 cacher.go:799] cacher (*apiregistration.APIService): 1 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.627872      38 cacher.go:799] cacher (*apiregistration.APIService): 2 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.630030      38 cacher.go:799] cacher (*apiregistration.APIService): 3 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.630092      38 cacher.go:799] cacher (*apiregistration.APIService): 4 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.630132      38 cacher.go:799] cacher (*apiregistration.APIService): 5 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:18" level=info msg="I1015 22:12:18.630165      38 cacher.go:799] cacher (*apiregistration.APIService): 6 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:18" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:18" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:18" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:18" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=info msg="I1015 22:12:19.524083      38 controller.go:132] OpenAPI AggregationController: action for item : Nothing (removed from the queue)." component=kube-apiserver
time="2021-10-15 22:12:19" level=info msg="I1015 22:12:19.524103      38 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue)." component=kube-apiserver
time="2021-10-15 22:12:19" level=info msg="I1015 22:12:19.541258      38 storage_scheduling.go:132] created PriorityClass system-node-critical with value 2000001000" component=kube-apiserver
time="2021-10-15 22:12:19" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=info msg="I1015 22:12:19.635457      38 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000" component=kube-apiserver
time="2021-10-15 22:12:19" level=info msg="I1015 22:12:19.635547      38 storage_scheduling.go:148] all system priority classes are created successfully or already exist." component=kube-apiserver
time="2021-10-15 22:12:19" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=error msg="health-check: APIServer might be down: expected 200 for api server ready check, got 500"
time="2021-10-15 22:12:19" level=info msg="I1015 22:12:19.990148      38 controller.go:611] quota admission added evaluator for: roles.rbac.authorization.k8s.io" component=kube-apiserver
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.009373      38 controller.go:611] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io" component=kube-apiserver
time="2021-10-15 22:12:20" level=info msg="starting Konnectivity"
time="2021-10-15 22:12:20" level=info msg="Starting to supervise" component=konnectivity
time="2021-10-15 22:12:20" level=info msg="Started successfully, go nuts" component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101483      54 options.go:107] ServerCert set to \"\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101517      54 options.go:108] ServerKey set to \"\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101521      54 options.go:109] ServerCACert set to \"\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101525      54 options.go:110] ClusterCert set to \"/var/lib/k0s/pki/server.crt\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101531      54 options.go:111] ClusterKey set to \"/var/lib/k0s/pki/server.key\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101535      54 options.go:112] ClusterCACert set to \"\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101539      54 options.go:113] Mode set to \"grpc\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101547      54 options.go:114] UDSName set to \"/run/k0s/konnectivity-server/konnectivity-server.sock\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101554      54 options.go:115] DeleteUDSFile set to false." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101558      54 options.go:116] Server port set to 0." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101562      54 options.go:117] Agent port set to 8132." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101566      54 options.go:118] Admin port set to 8133." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101570      54 options.go:119] Health port set to 8092." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101574      54 options.go:120] Keepalive time set to 1h0m0s." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101583      54 options.go:121] Frontend keepalive time set to 1h0m0s." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101588      54 options.go:122] EnableProfiling set to false." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101595      54 options.go:123] EnableContentionProfiling set to false." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101600      54 options.go:124] ServerID set to 5cad1cd34b9adf80c26de2ebd64a958cf816113dc8155fa0235812918076fbe7." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101604      54 options.go:125] ServerCount set to 1." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101609      54 options.go:126] AgentNamespace set to \"kube-system\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101613      54 options.go:127] AgentServiceAccount set to \"konnectivity-agent\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101619      54 options.go:128] AuthenticationAudience set to \"system:konnectivity-server\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101624      54 options.go:129] KubeconfigPath set to \"/var/lib/k0s/pki/konnectivity.conf\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101629      54 options.go:130] KubeconfigQPS set to 0.000000." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101635      54 options.go:131] KubeconfigBurst set to 0." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.101643      54 options.go:132] ProxyStrategies set to \"default\"." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.104185      54 server.go:90] Starting frontend server for client connections." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.105743      54 server.go:102] Starting agent server for tunnel connections." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.106145      54 server.go:107] Starting admin server for debug connections." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.106218      54 server.go:112] Starting health server for healthchecks." component=konnectivity
time="2021-10-15 22:12:20" level=info msg="W1015 22:12:20.158196      38 lease.go:233] Resetting endpoints for master service \"kubernetes\" to [10.0.2.100]" component=kube-apiserver
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.158857      38 controller.go:611] quota admission added evaluator for: endpoints" component=kube-apiserver
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.161348      38 controller.go:611] quota admission added evaluator for: endpointslices.discovery.k8s.io" component=kube-apiserver
time="2021-10-15 22:12:20" level=info msg="starting Scheduler"
time="2021-10-15 22:12:20" level=info msg="Starting kube-scheduler"
time="2021-10-15 22:12:20" level=info msg="Starting to supervise" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="Started successfully, go nuts" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192018      64 flags.go:59] FLAG: --add-dir-header=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192039      64 flags.go:59] FLAG: --address=\"0.0.0.0\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192043      64 flags.go:59] FLAG: --allow-metric-labels=\"[]\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192051      64 flags.go:59] FLAG: --alsologtostderr=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192054      64 flags.go:59] FLAG: --authentication-kubeconfig=\"/var/lib/k0s/pki/scheduler.conf\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192056      64 flags.go:59] FLAG: --authentication-skip-lookup=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192065      64 flags.go:59] FLAG: --authentication-token-webhook-cache-ttl=\"10s\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192068      64 flags.go:59] FLAG: --authentication-tolerate-lookup-failure=\"true\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192069      64 flags.go:59] FLAG: --authorization-always-allow-paths=\"[/healthz,/readyz,/livez]\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192075      64 flags.go:59] FLAG: --authorization-kubeconfig=\"/var/lib/k0s/pki/scheduler.conf\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192077      64 flags.go:59] FLAG: --authorization-webhook-cache-authorized-ttl=\"10s\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192079      64 flags.go:59] FLAG: --authorization-webhook-cache-unauthorized-ttl=\"10s\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192081      64 flags.go:59] FLAG: --bind-address=\"127.0.0.1\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192083      64 flags.go:59] FLAG: --cert-dir=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192085      64 flags.go:59] FLAG: --client-ca-file=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192086      64 flags.go:59] FLAG: --config=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192088      64 flags.go:59] FLAG: --contention-profiling=\"true\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192089      64 flags.go:59] FLAG: --disabled-metrics=\"[]\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192093      64 flags.go:59] FLAG: --experimental-logging-sanitization=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192095      64 flags.go:59] FLAG: --feature-gates=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192098      64 flags.go:59] FLAG: --help=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192100      64 flags.go:59] FLAG: --http2-max-streams-per-connection=\"0\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192102      64 flags.go:59] FLAG: --kube-api-burst=\"100\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192104      64 flags.go:59] FLAG: --kube-api-content-type=\"application/vnd.kubernetes.protobuf\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192106      64 flags.go:59] FLAG: --kube-api-qps=\"50\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192109      64 flags.go:59] FLAG: --kubeconfig=\"/var/lib/k0s/pki/scheduler.conf\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192111      64 flags.go:59] FLAG: --leader-elect=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192112      64 flags.go:59] FLAG: --leader-elect-lease-duration=\"15s\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192114      64 flags.go:59] FLAG: --leader-elect-renew-deadline=\"10s\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192116      64 flags.go:59] FLAG: --leader-elect-resource-lock=\"leases\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192117      64 flags.go:59] FLAG: --leader-elect-resource-name=\"kube-scheduler\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192119      64 flags.go:59] FLAG: --leader-elect-resource-namespace=\"kube-system\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192121      64 flags.go:59] FLAG: --leader-elect-retry-period=\"2s\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192122      64 flags.go:59] FLAG: --lock-object-name=\"kube-scheduler\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192124      64 flags.go:59] FLAG: --lock-object-namespace=\"kube-system\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192126      64 flags.go:59] FLAG: --log-backtrace-at=\":0\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192129      64 flags.go:59] FLAG: --log-dir=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192131      64 flags.go:59] FLAG: --log-file=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192133      64 flags.go:59] FLAG: --log-file-max-size=\"1800\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192134      64 flags.go:59] FLAG: --log-flush-frequency=\"5s\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192136      64 flags.go:59] FLAG: --logging-format=\"text\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192138      64 flags.go:59] FLAG: --logtostderr=\"true\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192140      64 flags.go:59] FLAG: --master=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192141      64 flags.go:59] FLAG: --one-output=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192143      64 flags.go:59] FLAG: --permit-address-sharing=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192144      64 flags.go:59] FLAG: --permit-port-sharing=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192146      64 flags.go:59] FLAG: --policy-config-file=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192276      64 flags.go:59] FLAG: --policy-configmap=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192284      64 flags.go:59] FLAG: --policy-configmap-namespace=\"kube-system\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192286      64 flags.go:59] FLAG: --port=\"10251\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192288      64 flags.go:59] FLAG: --profiling=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192291      64 flags.go:59] FLAG: --requestheader-allowed-names=\"[]\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192303      64 flags.go:59] FLAG: --requestheader-client-ca-file=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192306      64 flags.go:59] FLAG: --requestheader-extra-headers-prefix=\"[x-remote-extra-]\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192331      64 flags.go:59] FLAG: --requestheader-group-headers=\"[x-remote-group]\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192335      64 flags.go:59] FLAG: --requestheader-username-headers=\"[x-remote-user]\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192344      64 flags.go:59] FLAG: --secure-port=\"10259\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192346      64 flags.go:59] FLAG: --show-hidden-metrics-for-version=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192348      64 flags.go:59] FLAG: --skip-headers=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192352      64 flags.go:59] FLAG: --skip-log-headers=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192354      64 flags.go:59] FLAG: --stderrthreshold=\"2\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192357      64 flags.go:59] FLAG: --tls-cert-file=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192361      64 flags.go:59] FLAG: --tls-cipher-suites=\"[]\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192365      64 flags.go:59] FLAG: --tls-min-version=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192367      64 flags.go:59] FLAG: --tls-private-key-file=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192371      64 flags.go:59] FLAG: --tls-sni-cert-key=\"[]\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192376      64 flags.go:59] FLAG: --use-legacy-policy-config=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192378      64 flags.go:59] FLAG: --v=\"1\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192384      64 flags.go:59] FLAG: --version=\"false\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192393      64 flags.go:59] FLAG: --vmodule=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.192395      64 flags.go:59] FLAG: --write-config-to=\"\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="starting Manager"
time="2021-10-15 22:12:20" level=info msg="Starting kube-controller-manager"
time="2021-10-15 22:12:20" level=info msg="Starting to supervise" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="Started successfully, go nuts" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299136      75 flags.go:59] FLAG: --add-dir-header=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299170      75 flags.go:59] FLAG: --address=\"127.0.0.1\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299182      75 flags.go:59] FLAG: --allocate-node-cidrs=\"true\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299194      75 flags.go:59] FLAG: --allow-metric-labels=\"[]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299199      75 flags.go:59] FLAG: --allow-untagged-cloud=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299202      75 flags.go:59] FLAG: --alsologtostderr=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299205      75 flags.go:59] FLAG: --attach-detach-reconcile-sync-period=\"1m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299209      75 flags.go:59] FLAG: --authentication-kubeconfig=\"/var/lib/k0s/pki/ccm.conf\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299213      75 flags.go:59] FLAG: --authentication-skip-lookup=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299216      75 flags.go:59] FLAG: --authentication-token-webhook-cache-ttl=\"10s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299219      75 flags.go:59] FLAG: --authentication-tolerate-lookup-failure=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299222      75 flags.go:59] FLAG: --authorization-always-allow-paths=\"[/healthz,/readyz,/livez]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299229      75 flags.go:59] FLAG: --authorization-kubeconfig=\"/var/lib/k0s/pki/ccm.conf\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299233      75 flags.go:59] FLAG: --authorization-webhook-cache-authorized-ttl=\"10s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299236      75 flags.go:59] FLAG: --authorization-webhook-cache-unauthorized-ttl=\"10s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299238      75 flags.go:59] FLAG: --bind-address=\"127.0.0.1\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299356      75 flags.go:59] FLAG: --cert-dir=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299371      75 flags.go:59] FLAG: --cidr-allocator-type=\"RangeAllocator\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299375      75 flags.go:59] FLAG: --client-ca-file=\"/var/lib/k0s/pki/ca.crt\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299394      75 flags.go:59] FLAG: --cloud-config=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299454      75 flags.go:59] FLAG: --cloud-provider=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299466      75 flags.go:59] FLAG: --cluster-cidr=\"10.244.0.0/16\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299470      75 flags.go:59] FLAG: --cluster-name=\"k0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299472      75 flags.go:59] FLAG: --cluster-signing-cert-file=\"/var/lib/k0s/pki/ca.crt\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299489      75 flags.go:59] FLAG: --cluster-signing-duration=\"8760h0m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299494      75 flags.go:59] FLAG: --cluster-signing-key-file=\"/var/lib/k0s/pki/ca.key\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299653      75 flags.go:59] FLAG: --cluster-signing-kube-apiserver-client-cert-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299779      75 flags.go:59] FLAG: --cluster-signing-kube-apiserver-client-key-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299789      75 flags.go:59] FLAG: --cluster-signing-kubelet-client-cert-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299792      75 flags.go:59] FLAG: --cluster-signing-kubelet-client-key-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299795      75 flags.go:59] FLAG: --cluster-signing-kubelet-serving-cert-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299798      75 flags.go:59] FLAG: --cluster-signing-kubelet-serving-key-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299800      75 flags.go:59] FLAG: --cluster-signing-legacy-unknown-cert-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299803      75 flags.go:59] FLAG: --cluster-signing-legacy-unknown-key-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299851      75 flags.go:59] FLAG: --concurrent-deployment-syncs=\"5\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299863      75 flags.go:59] FLAG: --concurrent-endpoint-syncs=\"5\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299866      75 flags.go:59] FLAG: --concurrent-gc-syncs=\"20\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299869      75 flags.go:59] FLAG: --concurrent-namespace-syncs=\"10\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299906      75 flags.go:59] FLAG: --concurrent-rc-syncs=\"5\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299912      75 flags.go:59] FLAG: --concurrent-replicaset-syncs=\"5\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299947      75 flags.go:59] FLAG: --concurrent-resource-quota-syncs=\"5\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299954      75 flags.go:59] FLAG: --concurrent-service-endpoint-syncs=\"5\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299982      75 flags.go:59] FLAG: --concurrent-service-syncs=\"1\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299987      75 flags.go:59] FLAG: --concurrent-serviceaccount-token-syncs=\"5\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.299990      75 flags.go:59] FLAG: --concurrent-statefulset-syncs=\"5\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300015      75 flags.go:59] FLAG: --concurrent-ttl-after-finished-syncs=\"5\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300040      75 flags.go:59] FLAG: --configure-cloud-routes=\"true\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300045      75 flags.go:59] FLAG: --contention-profiling=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300048      75 flags.go:59] FLAG: --controller-start-interval=\"0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300074      75 flags.go:59] FLAG: --controllers=\"[*,bootstrapsigner,tokencleaner]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300107      75 flags.go:59] FLAG: --deleting-pods-burst=\"0\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300134      75 flags.go:59] FLAG: --deleting-pods-qps=\"0.1\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300166      75 flags.go:59] FLAG: --deployment-controller-sync-period=\"30s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300192      75 flags.go:59] FLAG: --disable-attach-detach-reconcile-sync=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300219      75 flags.go:59] FLAG: --disabled-metrics=\"[]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300247      75 flags.go:59] FLAG: --enable-dynamic-provisioning=\"true\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300278      75 flags.go:59] FLAG: --enable-garbage-collector=\"true\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300284      75 flags.go:59] FLAG: --enable-hostpath-provisioner=\"true\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300315      75 flags.go:59] FLAG: --enable-leader-migration=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300322      75 flags.go:59] FLAG: --enable-taint-manager=\"true\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300325      75 flags.go:59] FLAG: --endpoint-updates-batch-period=\"0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300367      75 flags.go:59] FLAG: --endpointslice-updates-batch-period=\"0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300471      75 flags.go:59] FLAG: --experimental-cluster-signing-duration=\"8760h0m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300481      75 flags.go:59] FLAG: --experimental-logging-sanitization=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300485      75 flags.go:59] FLAG: --external-cloud-volume-plugin=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300488      75 flags.go:59] FLAG: --feature-gates=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300532      75 flags.go:59] FLAG: --flex-volume-plugin-dir=\"/usr/libexec/kubernetes/kubelet-plugins/volume/exec/\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300537      75 flags.go:59] FLAG: --help=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300575      75 flags.go:59] FLAG: --horizontal-pod-autoscaler-cpu-initialization-period=\"5m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300584      75 flags.go:59] FLAG: --horizontal-pod-autoscaler-downscale-delay=\"5m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300588      75 flags.go:59] FLAG: --horizontal-pod-autoscaler-downscale-stabilization=\"5m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300590      75 flags.go:59] FLAG: --horizontal-pod-autoscaler-initial-readiness-delay=\"30s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300593      75 flags.go:59] FLAG: --horizontal-pod-autoscaler-sync-period=\"15s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300633      75 flags.go:59] FLAG: --horizontal-pod-autoscaler-tolerance=\"0.1\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300640      75 flags.go:59] FLAG: --horizontal-pod-autoscaler-upscale-delay=\"3m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300676      75 flags.go:59] FLAG: --http2-max-streams-per-connection=\"0\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300687      75 flags.go:59] FLAG: --kube-api-burst=\"30\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300691      75 flags.go:59] FLAG: --kube-api-content-type=\"application/vnd.kubernetes.protobuf\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300694      75 flags.go:59] FLAG: --kube-api-qps=\"20\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300743      75 flags.go:59] FLAG: --kubeconfig=\"/var/lib/k0s/pki/ccm.conf\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300747      75 flags.go:59] FLAG: --large-cluster-size-threshold=\"50\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300760      75 flags.go:59] FLAG: --leader-elect=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300802      75 flags.go:59] FLAG: --leader-elect-lease-duration=\"15s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300807      75 flags.go:59] FLAG: --leader-elect-renew-deadline=\"10s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300843      75 flags.go:59] FLAG: --leader-elect-resource-lock=\"leases\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300853      75 flags.go:59] FLAG: --leader-elect-resource-name=\"kube-controller-manager\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300856      75 flags.go:59] FLAG: --leader-elect-resource-namespace=\"kube-system\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300859      75 flags.go:59] FLAG: --leader-elect-retry-period=\"2s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300892      75 flags.go:59] FLAG: --leader-migration-config=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300904      75 flags.go:59] FLAG: --log-backtrace-at=\":0\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300936      75 flags.go:59] FLAG: --log-dir=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300946      75 flags.go:59] FLAG: --log-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300950      75 flags.go:59] FLAG: --log-file-max-size=\"1800\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300953      75 flags.go:59] FLAG: --log-flush-frequency=\"5s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300995      75 flags.go:59] FLAG: --logging-format=\"text\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.300999      75 flags.go:59] FLAG: --logtostderr=\"true\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301002      75 flags.go:59] FLAG: --master=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301043      75 flags.go:59] FLAG: --max-endpoints-per-slice=\"100\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301052      75 flags.go:59] FLAG: --min-resync-period=\"12h0m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301056      75 flags.go:59] FLAG: --mirroring-concurrent-service-endpoint-syncs=\"5\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301058      75 flags.go:59] FLAG: --mirroring-endpointslice-updates-batch-period=\"0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301104      75 flags.go:59] FLAG: --mirroring-max-endpoints-per-subset=\"1000\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301115      75 flags.go:59] FLAG: --namespace-sync-period=\"5m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301119      75 flags.go:59] FLAG: --node-cidr-mask-size=\"24\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301122      75 flags.go:59] FLAG: --node-cidr-mask-size-ipv4=\"0\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301164      75 flags.go:59] FLAG: --node-cidr-mask-size-ipv6=\"0\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301169      75 flags.go:59] FLAG: --node-eviction-rate=\"0.1\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301173      75 flags.go:59] FLAG: --node-monitor-grace-period=\"40s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301214      75 flags.go:59] FLAG: --node-monitor-period=\"5s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301218      75 flags.go:59] FLAG: --node-startup-grace-period=\"1m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301255      75 flags.go:59] FLAG: --node-sync-period=\"0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301263      75 flags.go:59] FLAG: --one-output=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301266      75 flags.go:59] FLAG: --permit-address-sharing=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301268      75 flags.go:59] FLAG: --permit-port-sharing=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301269      75 flags.go:59] FLAG: --pod-eviction-timeout=\"5m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301271      75 flags.go:59] FLAG: --port=\"0\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301273      75 flags.go:59] FLAG: --profiling=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301303      75 flags.go:59] FLAG: --pv-recycler-increment-timeout-nfs=\"30\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301309      75 flags.go:59] FLAG: --pv-recycler-minimum-timeout-hostpath=\"60\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301311      75 flags.go:59] FLAG: --pv-recycler-minimum-timeout-nfs=\"300\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301313      75 flags.go:59] FLAG: --pv-recycler-pod-template-filepath-hostpath=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301315      75 flags.go:59] FLAG: --pv-recycler-pod-template-filepath-nfs=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301316      75 flags.go:59] FLAG: --pv-recycler-timeout-increment-hostpath=\"30\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301317      75 flags.go:59] FLAG: --pvclaimbinder-sync-period=\"15s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301350      75 flags.go:59] FLAG: --register-retry-count=\"10\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301356      75 flags.go:59] FLAG: --requestheader-allowed-names=\"[]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301360      75 flags.go:59] FLAG: --requestheader-client-ca-file=\"/var/lib/k0s/pki/front-proxy-ca.crt\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301364      75 flags.go:59] FLAG: --requestheader-extra-headers-prefix=\"[x-remote-extra-]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301391      75 flags.go:59] FLAG: --requestheader-group-headers=\"[x-remote-group]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301397      75 flags.go:59] FLAG: --requestheader-username-headers=\"[x-remote-user]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301400      75 flags.go:59] FLAG: --resource-quota-sync-period=\"5m0s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301402      75 flags.go:59] FLAG: --root-ca-file=\"/var/lib/k0s/pki/ca.crt\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301404      75 flags.go:59] FLAG: --route-reconciliation-period=\"10s\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301406      75 flags.go:59] FLAG: --secondary-node-eviction-rate=\"0.01\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301434      75 flags.go:59] FLAG: --secure-port=\"10257\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301441      75 flags.go:59] FLAG: --service-account-private-key-file=\"/var/lib/k0s/pki/sa.key\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301443      75 flags.go:59] FLAG: --service-cluster-ip-range=\"10.96.0.0/12\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301445      75 flags.go:59] FLAG: --show-hidden-metrics-for-version=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301446      75 flags.go:59] FLAG: --skip-headers=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301448      75 flags.go:59] FLAG: --skip-log-headers=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301478      75 flags.go:59] FLAG: --stderrthreshold=\"2\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301484      75 flags.go:59] FLAG: --terminated-pod-gc-threshold=\"12500\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301486      75 flags.go:59] FLAG: --tls-cert-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301487      75 flags.go:59] FLAG: --tls-cipher-suites=\"[]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301490      75 flags.go:59] FLAG: --tls-min-version=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301491      75 flags.go:59] FLAG: --tls-private-key-file=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301493      75 flags.go:59] FLAG: --tls-sni-cert-key=\"[]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301525      75 flags.go:59] FLAG: --unhealthy-zone-threshold=\"0.55\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301528      75 flags.go:59] FLAG: --use-service-account-credentials=\"true\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301530      75 flags.go:59] FLAG: --v=\"1\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301531      75 flags.go:59] FLAG: --version=\"false\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301535      75 flags.go:59] FLAG: --vmodule=\"\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301566      75 flags.go:59] FLAG: --volume-host-allow-local-loopback=\"true\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.301568      75 flags.go:59] FLAG: --volume-host-cidr-denylist=\"[]\"" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="starting DummyLeaderElector"
time="2021-10-15 22:12:20" level=info msg="registering new stack" component=applier-manager stack=/var/lib/k0s/manifests/konnectivity
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.426812      64 serving.go:347] Generated self-signed cert in-memory" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="starting Manager"
time="2021-10-15 22:12:20" level=info msg="starting K0SControlAPI"
time="2021-10-15 22:12:20" level=info msg="Starting to supervise" component=k0s-control-api
time="2021-10-15 22:12:20" level=info msg="Started successfully, go nuts" component=k0s-control-api
time="2021-10-15 22:12:20" level=info msg="time=\"2021-10-15 22:12:20\" level=info msg=\"no config file given, using defaults\"" component=k0s-control-api
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.621493      75 serving.go:347] Generated self-signed cert in-memory" component=kube-controller-manager
time="2021-10-15 22:12:20" level=info msg="starting Component"
time="2021-10-15 22:12:20" level=info msg="starting CSRApprover"
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.814760      64 server.go:140] \"Starting Kubernetes Scheduler version\" version=\"v1.22.2+k0s\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="W1015 22:12:20.815146      64 authorization.go:47] Authorization is disabled" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="W1015 22:12:20.815155      64 authentication.go:47] Authentication is disabled" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.815162      64 deprecated_insecure_serving.go:54] Serving healthz insecurely on [::]:10251" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.816669      64 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.816679      64 shared_informer.go:240] Waiting for caches to sync for RequestHeaderAuthRequestController" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.816703      64 configmap_cafile_content.go:201] \"Starting controller\" name=\"client-ca::kube-system::extension-apiserver-authentication::client-ca-file\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.816708      64 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.816769      64 configmap_cafile_content.go:201] \"Starting controller\" name=\"client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.816785      64 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.816889      64 secure_serving.go:200] Serving securely on 127.0.0.1:10259" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.816931      64 tlsconfig.go:240] \"Starting DynamicServingCertificateController\"" component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="starting Status"
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.916844      64 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file " component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.916911      64 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file " component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="I1015 22:12:20.916994      64 shared_informer.go:247] Caches are synced for RequestHeaderAuthRequestController " component=kube-scheduler
time="2021-10-15 22:12:20" level=info msg="initializing network reconciler for provider kuberouter"
time="2021-10-15 22:12:20" level=info msg="running reconciler: coredns"
time="2021-10-15 22:12:20" level=info msg="registering new stack" component=applier-manager stack=/var/lib/k0s/manifests/kuberouter
time="2021-10-15 22:12:20" level=info msg="running reconciler: kube-router"
time="2021-10-15 22:12:20" level=info msg="starting to dump manifests" component=kube-router
time="2021-10-15 22:12:20" level=info msg="registering new stack" component=applier-manager stack=/var/lib/k0s/manifests/helm
time="2021-10-15 22:12:20" level=info msg="registering new stack" component=applier-manager stack=/var/lib/k0s/manifests/coredns
time="2021-10-15 22:12:20" level=info msg="running reconciler: systemRBAC"
time="2021-10-15 22:12:20" level=info msg="running reconciler: default-psp"
time="2021-10-15 22:12:20" level=info msg="registering new stack" component=applier-manager stack=/var/lib/k0s/manifests/bootstraprbac
time="2021-10-15 22:12:20" level=info msg="registering new stack" component=applier-manager stack=/var/lib/k0s/manifests/defaultpsp
time="2021-10-15 22:12:20" level=info msg="running reconciler: kube-proxy"
time="2021-10-15 22:12:20" level=info msg="running reconciler: helmCrd"
time="2021-10-15 22:12:20" level=info msg="running reconciler: helmAddons"
time="2021-10-15 22:12:20" level=info msg="run begin" component=helmaddons
time="2021-10-15 22:12:20" level=info msg="No helm addons specified, do not run HelmAddons reconciler" component=helmaddons
time="2021-10-15 22:12:20" level=info msg="running reconciler: metricServer"
time="2021-10-15 22:12:20" level=info msg="running reconciler: kubeletConfig"
time="2021-10-15 22:12:20" level=info msg="registering new stack" component=applier-manager stack=/var/lib/k0s/manifests/kubeproxy
time="2021-10-15 22:12:20" level=info msg="registering new stack" component=applier-manager stack=/var/lib/k0s/manifests/metricserver
time="2021-10-15 22:12:20" level=info msg="registering new stack" component=applier-manager stack=/var/lib/k0s/manifests/kubelet
time="2021-10-15 22:12:20" level=warning msg="failed to load br_netfilter kernel module: /sbin/modprobe br_netfilter"
time="2021-10-15 22:12:20" level=warning msg="Failed to enable /proc/sys/net/bridge/bridge-nf-call-iptables: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory"
time="2021-10-15 22:12:20" level=warning msg="Failed to enable /proc/sys/net/bridge/bridge-nf-call-ip6tables: open /proc/sys/net/bridge/bridge-nf-call-ip6tables: no such file or directory"
time="2021-10-15 22:12:20" level=info msg="initializing ContainerD\n"
time="2021-10-15 22:12:20" level=info msg="initializing OCIBundleReconciler\n"
time="2021-10-15 22:12:20" level=info msg="initializing Kubelet\n"
time="2021-10-15 22:12:20" level=info msg="Staging /var/lib/k0s/bin/kubelet"
time="2021-10-15 22:12:20" level=info msg="Staging /var/lib/k0s/bin/runc"
time="2021-10-15 22:12:20" level=info msg="Staging /var/lib/k0s/bin/containerd-shim"
time="2021-10-15 22:12:20" level=info msg="Staging /var/lib/k0s/bin/containerd"
time="2021-10-15 22:12:20" level=info msg="Staging /var/lib/k0s/bin/containerd-shim-runc-v1"
time="2021-10-15 22:12:20" level=info msg="Staging /var/lib/k0s/bin/containerd-shim-runc-v2"
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.155050      75 controllermanager.go:186] Version: v1.22.2+k0s" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.155533      75 dynamic_cafile_content.go:155] \"Starting controller\" name=\"request-header::/var/lib/k0s/pki/front-proxy-ca.crt\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.155556      75 dynamic_cafile_content.go:155] \"Starting controller\" name=\"client-ca-bundle::/var/lib/k0s/pki/ca.crt\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.155731      75 secure_serving.go:200] Serving securely on 127.0.0.1:10257" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.155784      75 tlsconfig.go:240] \"Starting DynamicServingCertificateController\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.166830      75 controllermanager.go:562] Starting \"nodelifecycle\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.166844      75 shared_informer.go:240] Waiting for caches to sync for tokens" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.177133      38 controller.go:611] quota admission added evaluator for: serviceaccounts" component=kube-apiserver
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.178349      75 node_lifecycle_controller.go:377] Sending events to api server." component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.178444      75 taint_manager.go:163] \"Sending events to api server\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.178483      75 node_lifecycle_controller.go:505] Controller will reconcile labels." component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.178509      75 controllermanager.go:577] Started \"nodelifecycle\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.178518      75 controllermanager.go:562] Starting \"persistentvolume-binder\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.178631      75 node_lifecycle_controller.go:539] Starting node controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.178639      75 shared_informer.go:240] Waiting for caches to sync for taint" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182359      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/host-path\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182372      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/nfs\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182380      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/glusterfs\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182384      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/rbd\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182388      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/quobyte\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182392      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/flocker\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182401      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/portworx-volume\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182406      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/local-volume\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182411      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/storageos\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182429      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/csi\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182471      75 controllermanager.go:577] Started \"persistentvolume-binder\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182477      75 controllermanager.go:562] Starting \"persistentvolume-expander\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182566      75 pv_controller_base.go:308] Starting persistent volume controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.182571      75 shared_informer.go:240] Waiting for caches to sync for persistent volume" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.188578      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/portworx-volume\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.188596      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/glusterfs\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.188610      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/rbd\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.188617      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/storageos\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.188629      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/fc\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.188683      75 controllermanager.go:577] Started \"persistentvolume-expander\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.188695      75 expand_controller.go:327] Starting expand controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.188703      75 shared_informer.go:240] Waiting for caches to sync for expand" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.188696      75 controllermanager.go:562] Starting \"serviceaccount\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.191626      75 controllermanager.go:577] Started \"serviceaccount\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.191641      75 controllermanager.go:562] Starting \"daemonset\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.191705      75 serviceaccounts_controller.go:117] Starting service account controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.191713      75 shared_informer.go:240] Waiting for caches to sync for service account" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.195007      75 controllermanager.go:577] Started \"daemonset\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.195022      75 controllermanager.go:562] Starting \"csrsigning\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.195084      75 daemon_controller.go:284] Starting daemon sets controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.195091      75 shared_informer.go:240] Waiting for caches to sync for daemon sets" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199042      75 controllermanager.go:577] Started \"csrsigning\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199058      75 controllermanager.go:562] Starting \"endpoint\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199119      75 certificate_controller.go:118] Starting certificate controller \"csrsigning-legacy-unknown\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199127      75 shared_informer.go:240] Waiting for caches to sync for certificate-csrsigning-legacy-unknown" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199150      75 certificate_controller.go:118] Starting certificate controller \"csrsigning-kubelet-serving\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199153      75 shared_informer.go:240] Waiting for caches to sync for certificate-csrsigning-kubelet-serving" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199165      75 certificate_controller.go:118] Starting certificate controller \"csrsigning-kubelet-client\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199168      75 shared_informer.go:240] Waiting for caches to sync for certificate-csrsigning-kubelet-client" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199181      75 certificate_controller.go:118] Starting certificate controller \"csrsigning-kube-apiserver-client\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199183      75 shared_informer.go:240] Waiting for caches to sync for certificate-csrsigning-kube-apiserver-client" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199201      75 dynamic_serving_content.go:129] \"Starting controller\" name=\"csr-controller::/var/lib/k0s/pki/ca.crt::/var/lib/k0s/pki/ca.key\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199225      75 dynamic_serving_content.go:129] \"Starting controller\" name=\"csr-controller::/var/lib/k0s/pki/ca.crt::/var/lib/k0s/pki/ca.key\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199238      75 dynamic_serving_content.go:129] \"Starting controller\" name=\"csr-controller::/var/lib/k0s/pki/ca.crt::/var/lib/k0s/pki/ca.key\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.199249      75 dynamic_serving_content.go:129] \"Starting controller\" name=\"csr-controller::/var/lib/k0s/pki/ca.crt::/var/lib/k0s/pki/ca.key\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.203452      75 controllermanager.go:577] Started \"endpoint\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.203466      75 controllermanager.go:562] Starting \"endpointslicemirroring\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.203538      75 endpoints_controller.go:195] Starting endpoint controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.203547      75 shared_informer.go:240] Waiting for caches to sync for endpoint" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.206780      75 controllermanager.go:577] Started \"endpointslicemirroring\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.206793      75 controllermanager.go:562] Starting \"replicationcontroller\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.206877      75 endpointslicemirroring_controller.go:212] Starting EndpointSliceMirroring controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.206890      75 shared_informer.go:240] Waiting for caches to sync for endpoint_slice_mirroring" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.210297      75 controllermanager.go:577] Started \"replicationcontroller\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.210306      75 controllermanager.go:562] Starting \"podgc\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.210315      75 replica_set.go:186] Starting replicationcontroller controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.210322      75 shared_informer.go:240] Waiting for caches to sync for ReplicationController" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.213725      75 controllermanager.go:577] Started \"podgc\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.213736      75 controllermanager.go:562] Starting \"disruption\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.213766      75 gc_controller.go:89] Starting GC controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.213772      75 shared_informer.go:240] Waiting for caches to sync for GC" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.267068      75 shared_informer.go:247] Caches are synced for tokens " component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.281601      38 cacher.go:799] cacher (*core.Secret): 1 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.281620      38 cacher.go:799] cacher (*core.Secret): 2 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.281627      38 cacher.go:799] cacher (*core.Secret): 3 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.281631      38 cacher.go:799] cacher (*core.Secret): 4 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.281642      38 cacher.go:799] cacher (*core.Secret): 5 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.281645      38 cacher.go:799] cacher (*core.Secret): 6 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.289540      38 cacher.go:799] cacher (*core.ServiceAccount): 1 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.289618      38 cacher.go:799] cacher (*core.ServiceAccount): 2 objects queued in incoming channel." component=kube-apiserver
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.368708      75 controllermanager.go:577] Started \"disruption\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.368724      75 controllermanager.go:562] Starting \"attachdetach\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.368730      75 disruption.go:363] Starting disruption controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.368740      75 shared_informer.go:240] Waiting for caches to sync for disruption" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="Staging /var/lib/k0s/bin/xtables-legacy-multi"
time="2021-10-15 22:12:21" level=warning msg="failed to load br_netfilter kernel module: /sbin/modprobe br_netfilter"
time="2021-10-15 22:12:21" level=warning msg="Failed to enable /proc/sys/net/bridge/bridge-nf-call-iptables: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory"
time="2021-10-15 22:12:21" level=warning msg="Failed to enable /proc/sys/net/bridge/bridge-nf-call-ip6tables: open /proc/sys/net/bridge/bridge-nf-call-ip6tables: no such file or directory"
time="2021-10-15 22:12:21" level=info msg="starting ContainerD"
time="2021-10-15 22:12:21" level=info msg="Starting containerD"
time="2021-10-15 22:12:21" level=info msg="Starting to supervise" component=containerd
time="2021-10-15 22:12:21" level=info msg="Started successfully, go nuts" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.419598606Z\" level=info msg=\"starting containerd\" revision=8686ededfc90076914c5238eb96c883ea093a8ba version=v1.5.7" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.490322045Z\" level=info msg=\"loading plugin \\\"io.containerd.content.v1.content\\\"...\" type=io.containerd.content.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.490430981Z\" level=info msg=\"loading plugin \\\"io.containerd.snapshotter.v1.aufs\\\"...\" type=io.containerd.snapshotter.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.491095556Z\" level=info msg=\"skip loading plugin \\\"io.containerd.snapshotter.v1.aufs\\\"...\" error=\"aufs is not supported (modprobe aufs failed: exit status 1 \\\"modprobe: can't change directory to '/lib/modules': No such file or directory\\\\n\\\"): skip plugin\" type=io.containerd.snapshotter.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.491116011Z\" level=info msg=\"loading plugin \\\"io.containerd.snapshotter.v1.btrfs\\\"...\" type=io.containerd.snapshotter.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.491344554Z\" level=info msg=\"loading plugin \\\"io.containerd.snapshotter.v1.devmapper\\\"...\" type=io.containerd.snapshotter.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.491371934Z\" level=warning msg=\"failed to load plugin io.containerd.snapshotter.v1.devmapper\" error=\"devmapper not configured\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.491380024Z\" level=info msg=\"loading plugin \\\"io.containerd.snapshotter.v1.native\\\"...\" type=io.containerd.snapshotter.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.491415254Z\" level=info msg=\"loading plugin \\\"io.containerd.snapshotter.v1.overlayfs\\\"...\" type=io.containerd.snapshotter.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="starting OCIBundleReconciler"
time="2021-10-15 22:12:21" level=info msg="W1015 22:12:21.546948      75 probe.go:268] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating." component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.548172      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/portworx-volume\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.548185      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/storageos\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.548193      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/fc\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.548200      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/iscsi\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.548207      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/rbd\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.548235      75 plugins.go:641] \"Loaded volume plugin\" pluginName=\"kubernetes.io/csi\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.548394      75 controllermanager.go:577] Started \"attachdetach\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.548409      75 controllermanager.go:562] Starting \"ttl-after-finished\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.548438      75 attach_detach_controller.go:328] Starting attach detach controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.548442      75 shared_informer.go:240] Waiting for caches to sync for attach detach" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.584093898Z\" level=info msg=\"loading plugin \\\"io.containerd.snapshotter.v1.zfs\\\"...\" type=io.containerd.snapshotter.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.584313101Z\" level=info msg=\"skip loading plugin \\\"io.containerd.snapshotter.v1.zfs\\\"...\" error=\"path /var/lib/k0s/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin\" type=io.containerd.snapshotter.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.584336221Z\" level=info msg=\"loading plugin \\\"io.containerd.metadata.v1.bolt\\\"...\" type=io.containerd.metadata.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.584383010Z\" level=warning msg=\"could not use snapshotter devmapper in metadata plugin\" error=\"devmapper not configured\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.584394670Z\" level=info msg=\"metadata content store policy set\" policy=shared" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589441545Z\" level=info msg=\"loading plugin \\\"io.containerd.differ.v1.walking\\\"...\" type=io.containerd.differ.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589464528Z\" level=info msg=\"loading plugin \\\"io.containerd.gc.v1.scheduler\\\"...\" type=io.containerd.gc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589498614Z\" level=info msg=\"loading plugin \\\"io.containerd.service.v1.introspection-service\\\"...\" type=io.containerd.service.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589525848Z\" level=info msg=\"loading plugin \\\"io.containerd.service.v1.containers-service\\\"...\" type=io.containerd.service.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589538673Z\" level=info msg=\"loading plugin \\\"io.containerd.service.v1.content-service\\\"...\" type=io.containerd.service.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589546959Z\" level=info msg=\"loading plugin \\\"io.containerd.service.v1.diff-service\\\"...\" type=io.containerd.service.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589556874Z\" level=info msg=\"loading plugin \\\"io.containerd.service.v1.images-service\\\"...\" type=io.containerd.service.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589566603Z\" level=info msg=\"loading plugin \\\"io.containerd.service.v1.leases-service\\\"...\" type=io.containerd.service.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589575156Z\" level=info msg=\"loading plugin \\\"io.containerd.service.v1.namespaces-service\\\"...\" type=io.containerd.service.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589583807Z\" level=info msg=\"loading plugin \\\"io.containerd.service.v1.snapshots-service\\\"...\" type=io.containerd.service.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589591624Z\" level=info msg=\"loading plugin \\\"io.containerd.runtime.v1.linux\\\"...\" type=io.containerd.runtime.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.589848413Z\" level=info msg=\"loading plugin \\\"io.containerd.runtime.v2.task\\\"...\" type=io.containerd.runtime.v2" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590029541Z\" level=info msg=\"loading plugin \\\"io.containerd.monitor.v1.cgroups\\\"...\" type=io.containerd.monitor.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590275606Z\" level=info msg=\"loading plugin \\\"io.containerd.service.v1.tasks-service\\\"...\" type=io.containerd.service.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590299130Z\" level=info msg=\"loading plugin \\\"io.containerd.internal.v1.restart\\\"...\" type=io.containerd.internal.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590340726Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.containers\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590352139Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.content\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590363510Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.diff\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590371166Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.events\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590378182Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.healthcheck\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590392388Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.images\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590402368Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.leases\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590410038Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.namespaces\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590416750Z\" level=info msg=\"loading plugin \\\"io.containerd.internal.v1.opt\\\"...\" type=io.containerd.internal.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590901443Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.snapshots\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590923223Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.tasks\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590933129Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.version\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.590940929Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.cri\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.591244561Z\" level=info msg=\"Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc DefaultRuntime:{Type: Engine: PodAnnotations:[] ContainerAnnotations:[] Root: Options:map[] PrivilegedWithoutHostDevices:false BaseRuntimeSpec:} UntrustedWorkloadRuntime:{Type: Engine: PodAnnotations:[] ContainerAnnotations:[] Root: Options:map[] PrivilegedWithoutHostDevices:false BaseRuntimeSpec:} Runtimes:map[runc:{Type:io.containerd.runc.v2 Engine: PodAnnotations:[] ContainerAnnotations:[] Root: Options:map[BinaryName: CriuImagePath: CriuPath: CriuWorkPath: IoGid:0 IoUid:0 NoNewKeyring:false NoPivotRoot:false Root: ShimCgroup: SystemdCgroup:false] PrivilegedWithoutHostDevices:false BaseRuntimeSpec:}] NoPivot:false DisableSnapshotAnnotations:true DiscardUnpackedLayers:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginConfTemplate:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:k8s.gcr.io/pause:3.5 StatsCollectPeriod:10 SystemdCgroup:false EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false} ContainerdRootDir:/var/lib/k0s/containerd ContainerdEndpoint:/run/k0s/containerd.sock RootDir:/var/lib/k0s/containerd/io.containerd.grpc.v1.cri StateDir:/run/k0s/containerd/io.containerd.grpc.v1.cri}\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.591308859Z\" level=info msg=\"Connect containerd service\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.591369376Z\" level=info msg=\"Get image filesystem path \\\"/var/lib/k0s/containerd/io.containerd.snapshotter.v1.overlayfs\\\"\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.591390669Z\" level=warning msg=\"Running containerd in a user namespace typically requires disable_cgroup, disable_apparmor, restrict_oom_score_adj set to be true\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592227475Z\" level=error msg=\"failed to load cni during init, please check CRI plugin status before setting up network for pods\" error=\"cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed to load cni config\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592258623Z\" level=info msg=\"loading plugin \\\"io.containerd.grpc.v1.introspection\\\"...\" type=io.containerd.grpc.v1" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592342324Z\" level=info msg=\"Start subscribing containerd event\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592399267Z\" level=info msg=\"Start recovering state\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592517307Z\" level=info msg=\"Start event monitor\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592533705Z\" level=info msg=\"Start snapshots syncer\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592554400Z\" level=info msg=\"Start cni network conf syncer\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592569204Z\" level=info msg=\"Start streaming server\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592605187Z\" level=info msg=serving... address=/run/k0s/containerd.sock.ttrpc" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592825761Z\" level=info msg=serving... address=/run/k0s/containerd.sock" component=containerd
time="2021-10-15 22:12:21" level=info msg="time=\"2021-10-15T22:12:21.592846787Z\" level=info msg=\"containerd successfully booted in 0.208559s\"" component=containerd
time="2021-10-15 22:12:21" level=info msg="starting Kubelet"
time="2021-10-15 22:12:21" level=info msg="Starting kubelet"
time="2021-10-15 22:12:21" level=info msg="starting kubelet with args: map[--bootstrap-kubeconfig:/var/lib/k0s/kubelet-bootstrap.conf --cert-dir:/var/lib/k0s/kubelet/pki --config:/var/lib/k0s/kubelet-config.yaml --container-runtime:remote --container-runtime-endpoint:unix:///run/k0s/containerd.sock --containerd:/run/k0s/containerd.sock --kubeconfig:/var/lib/k0s/kubelet.conf --root-dir:/var/lib/k0s/kubelet --runtime-cgroups:/system.slice/containerd.service --v:1]"
time="2021-10-15 22:12:21" level=warning msg="failed to get initial kubelet config with join token: failed to get kubelet config from API: configmaps \"kubelet-config-default-1.22\" is forbidden: User \"system:bootstrap:jcvosp\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"kube-system\""
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.677399      75 controllermanager.go:577] Started \"ttl-after-finished\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.677465      75 controllermanager.go:562] Starting \"ephemeral-volume\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.677572      75 ttlafterfinished_controller.go:109] Starting TTL after finished controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.677595      75 shared_informer.go:240] Waiting for caches to sync for TTL after finished" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.825687      75 controllermanager.go:577] Started \"ephemeral-volume\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.825744      75 controllermanager.go:562] Starting \"csrcleaner\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.825932      75 controller.go:170] Starting ephemeral volume controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.825974      75 shared_informer.go:240] Waiting for caches to sync for ephemeral" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.880744      75 controllermanager.go:577] Started \"csrcleaner\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.880823      75 controllermanager.go:562] Starting \"nodeipam\"" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.880938      75 cleaner.go:82] Starting CSR cleaner controller" component=kube-controller-manager
time="2021-10-15 22:12:21" level=info msg="I1015 22:12:21.924151      75 node_ipam_controller.go:91] Sending events to api server." component=kube-controller-manager
time="2021-10-15 22:12:22" level=warning msg="failed to get initial kubelet config with join token: failed to get kubelet config from API: configmaps \"kubelet-config-default-1.22\" is forbidden: User \"system:bootstrap:jcvosp\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"kube-system\""
@FilBot3 FilBot3 added the bug Something isn't working label Oct 15, 2021
@mikhail-sakhnov
Copy link
Contributor

mikhail-sakhnov commented Oct 19, 2021 •
edited
Loading

Have you joined any worker? by default the controller node starts with no workers.

Please follow the instruction here https://docs.k0sproject.io/v1.22.2+k0s.1/k0s-in-docker/#2-optional-create-additional-workers

@FilBot3
Copy link
Author

FilBot3 commented Oct 19, 2021 •
edited
Loading

No, I saw it was optional and figured that k0s would untaint the control plane or master node. Here is the output with me following the optional steps.

➜  ~ podman run -d --name k0s --hostname k0s --privileged -v /var/lib/k0s -p 6443:6443 docker.io/k0sproject/k0s:latest
3151eef3f342342e5fb49bebd3429a28ca1748d622a7423b4ef1549cae5ee6fc
➜  ~ token=$(podman exec -t -i k0s k0s token create --role=worker)
➜  ~ echo $token
H4sIAAAAAAAC/2yV3Y6jMBKF7/sp8gI9Y0PT2o60F0OCSUjjjG3KTnwHmAlgk9CEzg+rffdVZ2akXWnvylVH37Esq85T3jeyGs7N6TifXfBT6T7PYzWc50/Psz/1/Gk2m83O1XCphvmsHsf+PP/+HaNv6Jv3DSM0f3158R+ashrG5ldT5mP1nH+O9Wloxvuzycd8PnsXaHwXeMHBJFkTLrlMQIBOGCLAHzM0LixOBPCQR3TJlOw1Ir6AJNTISQ5kK6LbmYIcWVvTbfwWq86t9kimEDFPkJobxLmC/r3AfZKvdJtZwjlcN8yR0CATcWkoc2TFMvLFe8kgqFOse+XqjZH9HrpbW2XkQ0keatB+unJURtLPJB+qCL+yLKE8Smp+LK/Vof87a/97lkFyrCL9waRcaBQoZgmVEd+ZJdGF1MG+wyol/T7bcZ1a6WvPkBxwmEZmCT5ZLRxN9i2NAOSCSSIZREhAEhlEHlqByOXrTQD4UlmdyJasZEfGFPM1s4TLWF9oR4IsxgH18DY//kAmKjc5DlvRgZ+BOYNH8btvXvPFmBY755ldwvIju5p4fReW5AzrU25HVMTyzNr1NVXkso2CRelgSAFvTMc/ypivc9YnHBK6XYy5xsaXxAzZivrZ6nBlbXjKl7KW8eGFo/G03fEUIudl3puTzlnwbmvp16/bHXlX4Ia9JKPZ9a5cus3CuauOA8Em4pdeshEreoVlqIrpB2Y7edbOiLJ1i3QKSeH1fdqNY7o0J01kapA+KtCB6sxlu+M13/Uxy9Z3mNhG3McIZOoVXt2b6QemCF3zzn3A0ghjgyZH+MJ28q69cbO3QSNkeku75JK2LsxduCl9uoGo7jYezzLvfNf2NHDWn5ki2zyuJ+GTm+7e+gpqJSaZlllIoaOatgbnLRG6TadU6S2f6nth6ybzRrSFNwFZ6TNlKLMk5F7Pc5uEC4tDGpkwldLylSGsefMBZGiQW7CuDyFKjpnTa7B4wdT+hUMCDHgIcLhksL5yFN2k4pHGZMdsAkAOw17ajdyll3yqsXJ1QG2KUzTu9VK+aJ+SypIt68xQqnrIoQzMFH79wYlbEmaIhl/+ue/SjCTbqgt6swxPVdQrYP1WT6TTyrzKrrewSzJ2lELb/qeWYZNZt1MrfhfHPd5bfBUEvGKVMGHpTw3rgMauVrsfV+olnBMZCcV86AAvupvk6ISLuN4WLdVUridj8TsD7fF4/Fk2/wiEpT34Fhc7GXKVBgKTzzzqX4qlnja+xHuP7tkkRxG/XLbqzUqENmBtUJDkBEvHZWv6MnaxAUmEO+C9tXfaBXfuYRDLEmdLFzFvfC/AJTrjH7yzF6puLo/Wk+zqZo9Il07Uh2sfliuH8kl/SEffS3urjTVnEUd+KvXA7uNVkjIQET7BsvZFfBgy1+8B871UfS+Osi1cPRXRW6KaURVZGahMbhbNGRsnAaLDTSzrJIPDLW1GyxTdatKjtDNrFZFIZ+kVkB7Zbh0Ym/DMHoZcOSehvxu1DoosQqUNrGzehCGwSRUhqev5zwxt/uxmkll2YEgKGblYAA1lBI+9/M5O/3yazY55V81nFp2fytNxrG7j75T4Xf9Oib+R8VB9NT7Pj9NnUblqfC5Op/E8Dnn/v7TPYaiO4/Nf0qNpm6OZzxan46/m8NQP1a9qqI5ldZ7P/vXvpy/qw/wP5P/gH8aPK4wnWx3nM3Ns8+DlW3upPtpD+3E4t5V/wB9P/wkAAP//B5DgyQIHAAA=
➜  ~ podman run -d --name k0s-worker1 --hostname k0s-worker1 --privileged -v /var/lib/k0s docker.io/k0sproject/k0s:latest k0s worker $token

b14cf0ae2031eb25830090290a46adfb5d655630ffdab615b4c42f232312e5ba
➜  ~ podman exec k0s kubectl get nodes
No resources found
➜  ~

Logs, too large for comments

@mikhail-sakhnov
Copy link
Contributor

@FilBot3 oh yeah, you are absolutely right, I forgot that docker image for k0s by default enables worker for the controller node by using --enable-worker 🤦

I haven't been using podman that much, based on the logs there is some issue with volume for the /var/lib/k0s

time="2021-10-19 13:16:38" level=info msg="E1019 13:16:38.353154     306 kubelet.go:1423] \"Failed to start ContainerManager\" err=\"failed to get rootfs info: failed to get device for dir \\\"/var/lib/k0s/kubelet\\\": could not find device with major: 0, minor: 39 in cached partitions map\"" component=kubelet

Need to check in details how podman works with volumes

@FilBot3
Copy link
Author

FilBot3 commented Oct 20, 2021

@Soider what would you like me to do to help out more?

@makhov makhov assigned makhov and unassigned mikhail-sakhnov Jan 12, 2022
@makhov
Copy link
Contributor

makhov commented Jan 12, 2022

Looks like it's related to cgroups v2, there is a similar issue in k3s/k3d: k3d-io/k3d#493
K3d has a workaround with a short explanation (see CHANGELOG.md).

@makhov
Copy link
Contributor

makhov commented Jan 14, 2022 •
edited
Loading

@FilBot3 By default, podman is running in a rootless mode that we currently don't support. Try running it with sudo.

@makhov makhov closed this as completed Jan 21, 2022
@jiridanek jiridanek mentioned this issue Apr 2, 2022
Closed
4 tasks
@lchunleo
Copy link

Am using rhel 8 and can i check if we need to do anything to use podman instead of containerd? or it works out of the box?
https://docs.k0sproject.io/stable/runtime/

@twz123
Copy link
Member

twz123 commented Feb 19, 2024

Am using rhel 8 and can i check if we need to do anything to use podman instead of containerd? or it works out of the box? https://docs.k0sproject.io/stable/runtime/

@lchunleo This specific issue is about running k0s in Podman, as an alternative to Docker. If I understand your question correctly, you want k0s to use Podman as a container runtime via CRI, instead of the bundled containerd? I doubt that's possible. As far as I know, Podman doesn't even implement the Container Runtime Interface. Do you have any a reference for it? If so, feel free to file a new issue or reach out on the forums.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@makhov makhov

Labels
bug Something isn't working needs-triage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@makhov @twz123 @mikhail-sakhnov @FilBot3 @jnummelin @lchunleo