Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate kafka-operator to use listener-operator for exposing brokers #443

Merged
merged 51 commits into from
Oct 7, 2024

Conversation

nightkr
Copy link
Member

@nightkr nightkr commented Aug 4, 2022

Description

Fixes #714.

Blockers:

Followup issues/regressions:

Review Checklist

  • Code contains useful comments
  • CRD change approved (or not applicable)
  • (Integration-)Test cases added (or not applicable)
  • Documentation added (or not applicable)
  • Changelog updated (or not applicable)
  • Cargo.toml only contains references to git tags (not specific commits or branches)
  • Helm chart can be installed and deployed operator works (or not applicable)

Once the review is done, comment bors r+ (or bors merge) to merge. Further information

This is far from final (for example, we'd need to expose a configurable
LoadBalancerClass, for example), but I wanted to see whether lb-op would be a
good fit for this operator first.
@nightkr
Copy link
Member Author

nightkr commented Aug 24, 2022

Blocked on stackabletech/listener-operator#1

@maltesander
Copy link
Member

Tested and works!

@lfrancke
Copy link
Member

I've moved this back into "Ready for Development" because the Listener operator is now merged. AFAIU this should now be ready to be worked on again?

@siegfriedweber
Copy link
Member

I've moved this back into "Ready for Development" because the Listener operator is now merged. AFAIU this should now be ready to be worked on again?

This pull request is updated as requested by stackabletech/listener-operator#5. When stackabletech/operator-rs#496 is merged and operator-rs and the listener-operator are released then this pull request can be made ready for review.

@nightkr
Copy link
Member Author

nightkr commented Sep 12, 2023

Tests currently fail due to stackabletech/listener-operator#110.

@nightkr nightkr marked this pull request as ready for review September 20, 2023 00:42
@nightkr
Copy link
Member Author

nightkr commented Sep 20, 2023

The tests now pass!

@sbernauer
Copy link
Member

sbernauer commented Sep 20, 2023

@nightkr do you plan to merge this or add the WIP functionality of secret-op beforehand, so that the kafka broker certificates will include the listener addresses?
It'd be awesome if we get it right in the first run :)

@nightkr
Copy link
Member Author

nightkr commented Sep 20, 2023

Preferably not, since we're still working out a few API kinks there (truststore passwords), and I'd rather keep this PR focused. ^^

@sbernauer
Copy link
Member

That smells like a minor bug that I would suggest filing against list-op.

I was not sure if this was a bug. Created stackabletech/listener-operator#229

@nightkr nightkr requested a review from sbernauer October 2, 2024 14:34
@nightkr
Copy link
Member Author

nightkr commented Oct 4, 2024

@sbernauer
Copy link
Member

Thanks, LGTM!
Hopefully last remaining question: We do we have a bootstrap service per rolegroup? Don't all bootstrap services return all brokers (from all rolegroups) regardless?
Why aren't we using a single bootstrap for the entire KafkaCluster?

@nightkr
Copy link
Member Author

nightkr commented Oct 7, 2024

Once connected to the bootstrap service, yes, you'll get all brokers. It's mostly a matter of consistency and "maaaybe this will be useful in some odd networking context, somewhere?".

sbernauer
sbernauer previously approved these changes Oct 7, 2024
Copy link
Member

@sbernauer sbernauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, we just need to fix the yanked crate and you wanted to update the doc sentence I added

Copy link
Member

@sbernauer sbernauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@nightkr nightkr added this pull request to the merge queue Oct 7, 2024
Merged via the queue into main with commit c576b7e Oct 7, 2024
31 checks passed
@nightkr nightkr deleted the spike/lb-operator branch October 7, 2024 13:02
@NickLarsenNZ NickLarsenNZ added release-note/action-required Denotes a PR that introduces potentially breaking changes that require user action. release/24.11.0 labels Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release/24.11.0 release-note/action-required Denotes a PR that introduces potentially breaking changes that require user action.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Service exposition with ListenerClasses
6 participants