feat: add confluent-hub to ksqlDB docker image #4729
Conversation
| @@ -136,15 +136,18 @@ services: | |||
| 2. Get the JDBC connector | |||
| ------------------------- | |||
|
|
|||
| [Download the JDBC connector](https://www.confluent.io/hub/confluentinc/kafka-connect-jdbc) | |||
| to your local working directory. Next, unzip the downloaded archive: | |||
| The easiest way to download connectors for use in ksqlDB with embedded {{ site.kconnect }} | |||
There was a problem hiding this comment.
@JimGalasyn the changes in this file should not go live until the next released version of ksqlDB, as version 0.7.0 does not contain confluent-hub (but 0.8.0 will). Is targeting master appropriate, or should I hold off on merging until the next ksqlDB release?
There was a problem hiding this comment.
We should set the local directory to be whatever Compose will mount.
There was a problem hiding this comment.
We should set the local directory to be whatever Compose will mount.
Do you mean setting the value for /path/to/local/dir in both the docker-compose and in the confluent-hub command, in order to remove the need for the user to manually uncomment the volume mount and update the directory in both places?
That makes sense to me assuming there's a directory we're always guaranteed to be able to download to, but I couldn't think of one since only absolute paths are allowed in the volume mount.
There was a problem hiding this comment.
I think you can do ./local-path/here/.
There was a problem hiding this comment.
Doesn't work for me locally:
$ docker run -v ./plugins:/share/confluent-hub-components <test image> confluent-hub install --no-prompt confluentinc/kafka-connect-datagen:0.2.0
docker: Error response from daemon: create ./plugins: "./plugins" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.
Am I doing something wrong?
There was a problem hiding this comment.
Resolved offline. Compose has different behavior than Docker :( docker/cli#1203
There was a problem hiding this comment.
Updated with a workaround.
There was a problem hiding this comment.
@vcrfxia iiuc, 0.8.0 goes out with CP 5.5, so if you retarget this PR to the 5.5.x branch, it will go live when CP 5.5 goes live..
There was a problem hiding this comment.
Ah hm. I wasn't planning to backport to 5.5.x since it's past feature freeze, so it sounds like I should just merge to master and let the change be picked up when the 0.8.0 docs branch is cut. Thanks for clarifying!
| ```bash | ||
| unzip confluentinc-kafka-connect-jdbc-*.zip | ||
| docker run -v /path/to/local/dir:/share/confluent-hub-components confluentinc/ksqldb-server:0.7.0 confluent-hub install --no-prompt confluentinc/kafka-connect-jdbc:5.4.1 |
There was a problem hiding this comment.
5.4.1 is the current latest version of the JDBC Connector, but how should we keep this version up to date as newer versions are released?
There was a problem hiding this comment.
Updated the version to {{ site.cprelease }} as discussed offline.
Description
Fixes #4558
This PR:
confluent-hubto the ksqlDB docker imageTesting done
Ran through the new version of the embedded Connect tutorial.
Reviewer checklist