Possibility to spin up Polaris with MinIO

[noklam]

To quickly try Polaris out, it would be nice to have an open source stack that can easily be spin up with a single Docker/Docker compose file. I have seen demo with Polaris + S3 + Spark but struggled a bit with setting up roles etc with MinIO (not sure if it's possible).

[lefebsy]

Hello,
About MinIO and setuping a role to do authentification :
There is an explication here in chapter "Role Policies" : https://blog.min.io/minio-openid-connect-integration/
In my comprehension it need to be associated to an external identity provider (like keycloak) to simulate the same behavior than in AWS, even relying on MinIO server to avoid custom claim in IDP.

But without testing it, I am not convinced that it will really be similar.

[noklam]

From my understanding it would be easier to just setup with s3 then. Is it possible to setup Polaris without authentication? Probably not but just confirming.

Note that I intend to use that for quick demo so I am trying to look for a shortest minimal path.

[lefebsy]

I've failed to do that too.

Then I have very recently submitted a PR (review just started) trying to implement the feature request 32 (S3 OnPrem).
Basically it is removing role_arn and adding endpoint and keys...

The regression script test is built with MinIO ; so I confirm that a modified Polaris+MinIO are ok together.

If you do not care to wait for your demo that the PR is accepted or not, you can have a look on the PR S3 OnPrem, pull the branch, and launch the regtests/run_spark_sql_s3compatible.sh.

It's under review, please be kind :-)

[dvaldivia]

You mean the support you assume role in MinIO like AWS via the STS capability? It is supported via an external IDP, but you could also use the API to exchange credentials for temporary access tokens (STS)