Broken S3 implementation for netcdf files on non-AWS + Authenticated buckets

[mannreis]

I stumble upon this when trying to read a netcdf file from a non-AWS S3 bucket that requires authorization.

$ ncdump -L 11  -v time  "https://play.min.io/unidata/u_4km.nc#mode=bytes,s3&aws.profile=play"
        log_level changed to 11
        HDF5 error messages have been turned off.
ncdump/ncdump: https://play.min.io/unidata/u_4km.nc#mode=bytes,s3&aws.profile=play: NetCDF: Unknown file format
                                        NC4_clear_provenance
                                        NCZ_clear_provenance

I used minio public testing S3 server (~/.aws/credentials)

[play]
aws_access_key_id = Q3AM3UQ867SPQQA43P2F
aws_secret_access_key = zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG

It seems that this is working for publicly available netcdf files in AWS S3 like the ones on the used on the nczarr_test/run_external.sh

I fixed this with 7a11d8c

But I also noticed that one must always have a profile defined in order to make it work, so c791ddb will load try to load AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION from the environment if the profile is not set & there is no default profile (which is a valid case).

I'd like to add this as a test but I really think relying on the play.min.io is not the way to go. Perhaps a github actions workflow where minio-server gets deployed and serves the S3 tests.