This terraform implementation will deploy OpenShift 4.x into an Azure VNET, with two subnets for controlplane and worker nodes. Traffic to the master nodes is handled via a pair of loadbalancers, one for internal traffic and another for external API traffic. Application loadbalancing is handled by a third loadbalancer that talks to the router pods on the infra nodes. Worker, Infra and Master nodes are deployed across 3 Availability Zones
-
Create a Service Principal with proper IAM roles
azure_region = "eastus2"
cluster_name = "ocp43"
# From Prereq. Step #1
base_domain = "azure.ncolon.xyz"
azure_base_domain_resource_group_name = "openshift4-common-rg"
# From Prereq. Step #2
azure_subscription_id = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
azure_tenant_id = "YYYYYYYY-YYYY-YYYY-YYYY-YYYYYYYYYYYY"
azure_client_id = "ZZZZZZZZ-ZZZZ-ZZZZ-ZZZZ-ZZZZZZZZZZZZ"
azure_client_secret = "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA"
Variable | Description | Default | Type |
---|---|---|---|
azure_subscription_id | Subscription ID for Azure Account | - | string |
azure_tenant_id | Tenant ID for Azure Subscription | - | string |
azure_client_id | Application Client ID (from Prereq Step #2) | - | string |
azure_client_secret | Application Client Secret (from Prereq Step #2) | - | string |
azure_region | Azure Region to deploy to | - | string |
cluster_name | Cluster Identifier | - | string |
master_count | Number of master nodes to deploy | 3 | string |
worker_count | Number of worker nodes to deploy | 3 | string |
infra_count | Number of infra nodes to deploy | 0 | string |
machine_v4_cidrs | IPv4 CIDR for OpenShift VNET | [10.0.0.0/16] | list |
machine_v6_cidrs | IPv6 CIDR for OpenShift VNET | [] | list |
base_domain | DNS name for your deployment | - | string |
azure_base_domain_resource_group_name | Resource group where DNS is hosted. Must be on zame region. | - | string |
azure_bootstrap_vm_type | Size of bootstrap VM | Standard_D4s_v3 | string |
azure_master_vm_type | Size of master node VMs | Standard_D4s_v3 | string |
azure_infra_vm_type | Size of infra node VMs | Standard_D4s_v3 | string |
azure_worker_vm_type | Sizs of worker node VMs | Standard_D4s_v3 | string |
openshift_cluster_network_cidr | CIDR for Kubernetes pods | 10.128.0.0/14 | string |
openshift_cluster_network_host_prefix | Detemines the number of pods a node can host. 23 gives you 510 pods per node. | 23 | string |
openshift_service_network_cidr | CIDR for Kubernetes services | 172.30.0.0/16 | string |
openshift_pull_secret | Filename that holds your OpenShift pull-secret | - | string |
azure_master_root_volume_size | Size of master node root volume | 512 | string |
azure_worker_root_volume_size | Size of worker node root volume | 128 | string |
azure_infra_root_volume_size | Size of infra node root volume | 128 | string |
azure_master_root_volume_type | Storage type for master root volume | Premium_LRS | string |
azure_image_url | URL of the CoreOS image. | URL | string |
openshift_version | Version of OpenShift to deploy. | 4.3.26 | strig |
bootstrap_completed | Control variable to delete bootstrap node after initialization | false | bool |
azure_private | If set to true will deploy api and *.apps endpoints as private LoadBalancers |
- | bool |
azure_extra_tags | Extra Azure tags to be applied to created resources | {} | map |
airgapped | Configuration for an AirGapped environment | AirGapped | map |
azure_environment | The target Azure cloud environment for the cluster | public | string |
azure_master_availability_zones | The availability zones in which to create the masters. The length of this list must match master_count |
["1","2","3"] | list |
azure_preexisting_network | Specifies whether an existing network should be used or a new one created for installation. | false | bool |
azure_network_resource_group_name | The name of the network resource group, either existing or to be created | null |
string |
azure_virtual_network | The name of the virtual network, either existing or to be created | null |
string |
azure_control_plane_subnet | The name of the subnet for the control plane, either existing or to be created | null |
string |
azure_compute_subnet | The name of the subnet for worker nodes, either existing or to be created | null |
string |
azure_emulate_single_stack_ipv6 | This determines whether a dual-stack cluster is configured to emulate single-stack IPv6 | false | bool |
azure_outbound_user_defined_routing | This determined whether User defined routing will be used for egress to Internet. When false , Standard LB will be used for egress to the Internet. |
false | bool |
use_ipv4 | This determines wether your cluster will use IPv4 networking | true | bool |
use_ipv6 | This determines wether your cluster will use IPv6 networking | false | bool |
proxy_config | Configuration for Cluster wide proxy | AirGapped | map |
- Clone github repository
git clone git@github.com:ibm-cloud-architecture/terraform-openshift4-azure.git
-
Create your
terraform.tfvars
file -
Deploy with terraform
$ terraform init
$ terraform plan
$ terraform apply
- Destroy bootstrap node
$ TF_VAR_bootstrap_complete=true terraform apply
- To access your cluster
$ export KUBECONFIG=$PWD/installer-files/auth/kubeconfig
$ oc get nodes
NAME STATUS ROLES AGE VERSION
ocp43-6s3ag-infra-eastus21-zgv9p Ready infra,worker 2m52s v1.14.6+c07e432da
ocp43-6s3ag-infra-eastus22-7dr7x Ready infra,worker 2m42s v1.14.6+c07e432da
ocp43-6s3ag-infra-eastus23-7d22g Ready infra,worker 2m4s v1.14.6+c07e432da
ocp43-6s3ag-master-0 Ready master 9m24s v1.14.6+c07e432da
ocp43-6s3ag-master-1 Ready master 10m v1.14.6+c07e432da
ocp43-6s3ag-master-2 Ready master 9m44s v1.14.6+c07e432da
ocp43-6s3ag-worker-eastus21-q62tb Ready worker 2m42s v1.14.6+c07e432da
ocp43-6s3ag-worker-eastus22-tdc2q Ready worker 2m53s v1.14.6+c07e432da
ocp43-6s3ag-worker-eastus23-wmwzz Ready worker 2m56s v1.14.6+c07e432da
Deployment of Openshift Worker and Infra nodes is handled by the machine-operator-api cluster operator.
$ oc get machineset -n openshift-machine-api
NAME DESIRED CURRENT READY AVAILABLE AGE
ocp43-f5k8m-infra-eastus21 1 1 1 1 12h
ocp43-f5k8m-infra-eastus22 1 1 1 1 12h
ocp43-f5k8m-infra-eastus23 1 1 1 1 12h
ocp43-f5k8m-worker-eastus21 1 1 1 1 12h
ocp43-f5k8m-worker-eastus22 1 1 1 1 12h
ocp43-f5k8m-worker-eastus23 1 1 1 1 12h
$ oc get machines -n openshift-machine-api
NAME STATE TYPE REGION ZONE AGE
ocp43-f5k8m-infra-eastus21-7f9sv Running Standard_D4s_v3 eastus2 1 12h
ocp43-f5k8m-infra-eastus22-tsh7s Running Standard_D4s_v3 eastus2 2 12h
ocp43-f5k8m-infra-eastus23-vw5mc Running Standard_D4s_v3 eastus2 3 12h
ocp43-f5k8m-worker-eastus21-8sgs5 Running Standard_D8s_v3 eastus2 1 12h
ocp43-f5k8m-worker-eastus22-zqmc5 Running Standard_D8s_v3 eastus2 2 12h
ocp43-f5k8m-worker-eastus23-q9g5v Running Standard_D8s_v3 eastus2 3 12h
The infra nodes host the router/ingress pods, all the monitoring infrastrucutre, and the image registry.