feat(iceberg): support iceberg engine connection

[chenzl25]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

• Related: Tracking: Iceberg engine table #19418
• Introduce a session variable iceberg_engine_connection to require users to provide their own bucket and catalog for the iceberg engine via iceberg connection. With this config, it can make us much easier to share iceberg tables with users, since the underlying warehouse is managed by users and they can have a better control of the warehouse credential. No compaction would be supported currently.

Example

create secret my_secret with (
  backend = 'meta'
) as 'hummockadmin';

create connection my_conn
with (
    type = 'iceberg',
    warehouse.path = 's3://hummock001/iceberg_connection',
    s3.access.key = secret my_secret,
    s3.secret.key = secret my_secret,
    s3.endpoint = 'http://127.0.0.1:9301',
    s3.region = 'us-west-2',
    catalog.type = 'storage',
);

-- Alter the system variable
alter system set iceberg_engine_connection = 'public.my_conn';
-- Alter the current session variable
set iceberg_engine_connection = 'public.my_conn';

create table t(id int primary key, name varchar) with(commit_checkpoint_interval = 1) engine = iceberg;

Checklist

• I have written necessary rustdoc comments.
• I have added necessary unit tests and integration tests.
• I have added test labels as necessary.
• I have added fuzzing tests or opened an issue to track them.
• My PR contains breaking changes.
• My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
• My PR contains critical fixes that are necessary to be merged into the latest release.

Documentation

• My PR needs documentation updates.

Release note

• Introduce iceberg engine connection variable iceberg_engine_connection. To use iceberg engine table, users must create an iceberg connection first and set the variable iceberg_engine_connection first. Iceberg engine table will use the connection provided to access the iceberg catalog and warehouse as how iceberg sink and iceberg source work.

[gru-agent]

This pull request has been modified. If you want me to regenerate unit test for any of the files related, please find the file in "Files Changed" tab and add a comment @gru-agent. (The github "Comment on this file" feature is in the upper right corner of each file in "Files Changed" tab.)

[chenzl25]

Now, the iceberg engine allows users to use the iceberg engine connection to provide catalog and storage information.

[github-actions]

Hi, there.

📝 Telemetry Reminder:
If you're implementing this feature, please consider adding telemetry metrics to track its usage. This helps us understand how the feature is being used and improve it further.
You can find the function report_event of telemetry reporting in the following files. Feel free to ask questions if you need any guidance!

• src/frontend/src/telemetry.rs
• src/meta/src/telemetry.rs
• src/stream/src/telemetry.rs
• src/storage/compactor/src/telemetry.rs
  Or calling report_event_common (src/common/telemetry_event/src/lib.rs) as if finding it hard to implement.
  ✨ Thank you for your contribution to RisingWave! ✨

---

This is an automated comment created by the peaceiris/actions-label-commenter. Responding to the bot or mentioning it won't have any effect.