Feat/mariner deploy aws gateway.

[UchicagoZchen138]

Jira Ticket: PXP-8909

New Features

• mariner persistent volume and persistent volume claim added as part of automatic deployment
• add mariner pod and mariner engine to indexd ingress policy

[jawadqur]

this should not be hardcoded, but need to get this dynamically, either from manifest or programmatically.

[m0nhawk]

AWS CLI can return this:

aws storagegateway describe-gateway-information --gateway-arn <gateway_arn> | jq -r '.GatewayNetworkInterfaces[].Ipv4Address'

The only thing we'll need a gateway ARN somewhere in the manifest, unless we can map S3 bucket to gateway ARN and use that.

[m0nhawk]

Actually, we can use aws storagegateway list-file-shares to list available file shares, use aws storagegateway describe-nfs-file-shares --file-share-arn-list <file_share_arn> to get S3 bucket.

Something like this:

aws storagegateway describe-gateway-information --gateway-arn \
  $(aws storagegateway describe-nfs-file-shares \
    --file-share-arn-list \
    $(aws storagegateway list-file-shares | jq -r '.FileShareInfoList[].FileShareARN') \
    | jq -r '.NFSFileShareInfoList[] | select(.Path == "<replace_this_by_mount_path>") | .GatewayARN') \
  | jq -r '.GatewayNetworkInterfaces[].Ipv4Address'

In this case, <replace_this_by_mount_path> equals to /newstoragegw-poc.

Also, there may be multiple file share to the same bucket and mount path, so it's probably should be configurable. And there may be multiple IP addresses.