External DNS is not creating DNS records in Azure resource

[Gordonby]

Describe the bug
An error is occurring when the Java app smoke test step runs in the scheduled CI/CD

time="2022-01-10T10:32:43Z" level=info msg="Using managed identity extension to retrieve access token for Azure API."
time="2022-01-10T10:32:43Z" level=info msg="Resolving to user assigned identity, client id is ***REDACTED***."
time="2022-01-10T10:32:48Z" level=error msg="azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscript
ions/***REDACTED***/resourceGroups/***REDACTED***/providers/Microsoft.Network/dnsZones?api-version=2018-05-01: StatusCode=0 -- Original Error: the MSI
endpoint is not available. Failed HTTP request to MSI endpoint: Get \"http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01\": dial tcp 169.254.1
69.254:80: i/o timeout"

To Reproduce
Re-Run the action : https://github.com/Azure/Aks-Construction/runs/4760776084?check_suite_focus=true

Plan

1. Strap up the External DNS Verification step to throw an error
   

[Gordonby]

Deleting and recreating the cluster has fixed the issue.

[Gordonby]

Need to investigate if this was just a blimp in AKS, or if the NetworkPolicy default deny interfered.

[Gordonby]

It was the NetworkPolicy.... 😆

New version of the ExternalDNS helm chart (with NetworkPolicy for egress): https://github.com/Gordonby/minihelm/tree/main/samples/externaldns