Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

java-cdks: update all libraries #45432

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions airbyte-cdk/bulk/core/base/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -13,29 +13,30 @@ dependencies {
exclude group: 'org.apache.commons'
exclude group: 'commons-io'
}
api 'io.github.oshai:kotlin-logging-jvm:5.1.0'
api 'io.github.oshai:kotlin-logging-jvm:7.0.0'
api 'io.micronaut:micronaut-runtime'
api 'org.apache.sshd:sshd-mina:2.12.1'
api 'org.apache.sshd:sshd-mina:2.13.2'
api 'org.jetbrains.kotlinx:kotlinx-coroutines-core'

implementation 'com.datadoghq:dd-trace-api:1.28.0'
implementation 'com.datadoghq:dd-trace-ot:1.28.0'
implementation 'com.datadoghq:dd-trace-api:1.39.0'
implementation 'com.datadoghq:dd-trace-ot:1.39.0'
implementation 'com.fasterxml.jackson.module:jackson-module-kotlin'
implementation 'com.fasterxml.jackson.module:jackson-module-afterburner'
implementation 'info.picocli:picocli:4.7.6'
implementation 'io.micronaut.picocli:micronaut-picocli:5.5.0'
implementation 'jakarta.validation:jakarta.validation-api:3.0.2'
implementation 'org.apache.commons:commons-lang3:3.14.0'
implementation 'jakarta.validation:jakarta.validation-api:3.1.0'
implementation 'org.apache.commons:commons-lang3:3.17.0'
implementation 'org.apache.logging.log4j:log4j-api'
implementation 'org.apache.logging.log4j:log4j-core'
implementation 'org.apache.logging.log4j:log4j-slf4j-impl'
implementation 'org.apache.logging.log4j:log4j-slf4j2-impl'
implementation 'org.apache.logging.log4j:log4j-layout-template-json:2.17.2'
implementation 'org.bouncycastle:bcprov-jdk18on:1.77'
implementation 'org.apache.logging.log4j:log4j-layout-template-json:2.24.0'
implementation 'org.bouncycastle:bcprov-jdk18on:1.78.1'
implementation 'org.openapi4j:openapi-schema-validator:1.0.7'

runtimeOnly 'com.google.guava:guava:33.2.0-jre'
runtimeOnly 'org.apache.commons:commons-compress:1.26.1'
// this is only needed because of exclusions in airbyte-protocol
runtimeOnly 'com.google.guava:guava:33.3.0-jre'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Watch out for these. These, I think but I'm not sure, are here to turn transitive dependencies into runtime-only. There's no point in using Guava in a Kotlin codebase.

I really should have commented their purpose, because now I can't really remember any more!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll find out what their purpose is and will report back shortly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two runtimeOnly dependencies exist because of the exclusions defined for airbyte-protocol. Can you add a comment to this effect please?

In any case bumping their versions is fine.

runtimeOnly 'org.apache.commons:commons-compress:1.27.1'

testFixturesApi 'org.jetbrains.kotlin:kotlin-test-junit'
testFixturesApi 'org.jetbrains.kotlin:kotlin-reflect'
Expand All @@ -47,5 +48,5 @@ dependencies {
}
testFixturesApi 'io.micronaut.test:micronaut-test-core:4.5.0'
testFixturesApi 'io.micronaut.test:micronaut-test-junit5:4.5.0'
testFixturesApi 'io.github.deblockt:json-diff:1.0.1'
testFixturesApi 'io.github.deblockt:json-diff:1.1.0'
}
2 changes: 1 addition & 1 deletion airbyte-cdk/bulk/core/extract/build.gradle
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
dependencies {
implementation project(':airbyte-cdk:bulk:core:bulk-cdk-core-base')
implementation 'org.apache.commons:commons-lang3:3.14.0'
implementation 'org.apache.commons:commons-lang3:3.17.0'
implementation 'hu.webarticum:tree-printer:3.2.1'

testFixturesApi testFixtures(project(':airbyte-cdk:bulk:core:bulk-cdk-core-base'))
Expand Down
6 changes: 3 additions & 3 deletions airbyte-cdk/bulk/core/load/build.gradle
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
dependencies {
implementation project(':airbyte-cdk:bulk:core:bulk-cdk-core-base')
implementation 'org.apache.commons:commons-lang3:3.14.0'
implementation 'org.apache.commons:commons-lang3:3.17.0'

// For ranges and rangesets
implementation("com.google.guava:guava:33.3.0-jre")

testFixturesApi testFixtures(project(':airbyte-cdk:bulk:core:bulk-cdk-core-base'))

testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.7.1")
implementation "org.jetbrains.kotlin:kotlin-reflect:2.0.0"
testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.8.1")
implementation "org.jetbrains.kotlin:kotlin-reflect:2.0.20"
}
1 change: 1 addition & 0 deletions airbyte-cdk/java/airbyte-cdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ corresponds to that version.

| Version | Date | Pull Request | Subject |
|:-----------|:-----------|:------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 0.46.0 | 2024-09-18 | [\#45432](https://github.com/airbytehq/airbyte/pull/45432) | upgrade all libraries to latest version |
| 0.45.1 | 2024-09-17 | [\#45638](https://github.com/airbytehq/airbyte/pull/45638) | upgrade apache mina sshd to 2.13.2 to handle openssh tcpkeepalive requests |
| 0.45.0 | 2024-09-16 | [\#45469](https://github.com/airbytehq/airbyte/pull/45469) | Fix some race conditions, improve thread filtering, improve test logging |
| 0.44.22 | 2024-09-10 | [\#45368](https://github.com/airbytehq/airbyte/pull/45368) | Remove excessive debezium logging |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ dependencies {
implementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-dependencies')
implementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core')

implementation 'com.azure:azure-storage-blob:12.12.0'
implementation 'com.azure:azure-storage-blob:12.27.1'

testFixturesApi testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core'))
}
28 changes: 14 additions & 14 deletions airbyte-cdk/java/airbyte-cdk/core/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -29,37 +29,37 @@ compileTestKotlin {

dependencies {

api 'com.datadoghq:dd-trace-api:1.28.0'
api 'com.datadoghq:dd-trace-ot:1.28.0'
api 'com.datadoghq:dd-trace-api:1.39.0'
api 'com.datadoghq:dd-trace-ot:1.39.0'
api 'com.zaxxer:HikariCP:5.1.0'
api 'org.jooq:jooq:3.16.23'
api 'org.apache.commons:commons-csv:1.10.0'
api 'org.jooq:jooq:3.19.11'
api 'org.apache.commons:commons-csv:1.11.0'

implementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-dependencies')

implementation 'commons-cli:commons-cli:1.4'
implementation 'commons-cli:commons-cli:1.9.0'
implementation 'io.aesy:datasize:1.0.0'
implementation 'net.i2p.crypto:eddsa:0.3.0'
implementation 'org.apache.httpcomponents:httpcore:4.4.16'
implementation 'org.apache.logging.log4j:log4j-layout-template-json:2.17.2'
implementation 'org.apache.logging.log4j:log4j-layout-template-json:2.24.0'
implementation 'org.apache.sshd:sshd-mina:2.13.2'
// bouncycastle is pinned to version-match the transitive dependency from kubernetes client-java
// because a version conflict causes "parameter object not a ECParameterSpec" on ssh tunnel initiation
implementation 'org.bouncycastle:bcpkix-jdk15on:1.66'
implementation 'org.bouncycastle:bcprov-jdk15on:1.66'
implementation 'org.bouncycastle:bctls-jdk15on:1.66'
implementation 'org.bouncycastle:bcpkix-jdk18on:1.78.1'
implementation 'org.bouncycastle:bcprov-jdk18on:1.78.1'
implementation 'org.bouncycastle:bctls-jdk18on:1.78.1'

testFixturesApi 'org.testcontainers:testcontainers:1.19.0'
testFixturesApi 'org.testcontainers:jdbc:1.19.0'
testFixturesApi 'org.testcontainers:testcontainers:1.20.1'
testFixturesApi 'org.testcontainers:jdbc:1.20.1'

testImplementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-dependencies')
testImplementation testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-db-sources'))
testImplementation testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-datastore-postgres'))

testImplementation 'mysql:mysql-connector-java:8.0.33'
testImplementation 'org.postgresql:postgresql:42.7.3'
testImplementation 'org.testcontainers:mysql:1.19.0'
testImplementation 'org.testcontainers:postgresql:1.19.0'
testImplementation 'org.testcontainers:mysql:1.20.1'
testImplementation 'org.testcontainers:postgresql:1.20.1'
testImplementation 'org.xbib.elasticsearch:joptsimple:6.3.2.1'
testImplementation 'org.mockito.kotlin:mockito-kotlin:5.2.1'
testImplementation 'org.mockito.kotlin:mockito-kotlin:5.4.0'
}
Original file line number Diff line number Diff line change
@@ -1 +1 @@
version=0.45.1
version=0.46.0
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ dependencies {
implementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-dependencies')
implementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core')

api 'com.google.cloud:google-cloud-bigquery:2.37.0'
api 'com.google.cloud:google-cloud-bigquery:2.42.2'

testFixturesApi testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core'))
}
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ constructor(
val result = executeQuery(bigQuery, getQueryConfig(sql, params))

if (result.getLeft() != null) {
val fieldList = result.getLeft()!!.getQueryResults().schema.fields
val fieldList = result.getLeft()!!.getQueryResults().schema!!.fields
return Streams.stream(result.getLeft()!!.getQueryResults().iterateAll()).map {
fieldValues: FieldValueList ->
sourceOperations!!.rowToJson(BigQueryResultSet(fieldValues, fieldList))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,5 @@ dependencies {

testFixturesApi testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core'))

testFixturesApi 'org.testcontainers:mongodb:1.19.0'
testFixturesApi 'org.testcontainers:mongodb:1.20.1'
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ dependencies {
implementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-dependencies')
implementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core')

api 'org.postgresql:postgresql:42.6.2'
api 'org.postgresql:postgresql:42.7.4'

testFixturesApi testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core'))

testFixturesApi 'org.testcontainers:postgresql:1.19.0'
testFixturesApi 'org.testcontainers:postgresql:1.20.1'
}
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ object PostgresSslConnectionUtils {
if (encryption.has(PARAM_CLIENT_KEY_PASSWORD))
encryption[PARAM_CLIENT_KEY_PASSWORD].asText()
else ""
var keyStorePassword = RandomStringUtils.randomAlphanumeric(10)
var keyStorePassword = RandomStringUtils.insecure().nextAlphanumeric(10)
if (sslPassword.isEmpty()) {
val file = File(ENCRYPT_FILE_NAME)
if (file.exists()) {
Expand Down
8 changes: 4 additions & 4 deletions airbyte-cdk/java/airbyte-cdk/db-destinations/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@ compileTestFixturesKotlin.compilerOptions.allWarningsAsErrors = false
compileTestKotlin.compilerOptions.allWarningsAsErrors = false

dependencies {
api 'org.apache.commons:commons-csv:1.10.0'
api 'org.apache.commons:commons-csv:1.11.0'

implementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-dependencies')
implementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core')
compileOnly project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-typing-deduping')

implementation 'io.aesy:datasize:1.0.0'

testFixturesCompileOnly 'org.projectlombok:lombok:1.18.30'
testFixturesAnnotationProcessor 'org.projectlombok:lombok:1.18.30'
testFixturesCompileOnly 'org.projectlombok:lombok:1.18.34'
testFixturesAnnotationProcessor 'org.projectlombok:lombok:1.18.34'
testFixturesImplementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-dependencies')
testFixturesImplementation testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-dependencies'))
testFixturesImplementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core')
Expand All @@ -20,6 +20,6 @@ dependencies {
testFixturesImplementation testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-typing-deduping'))

testImplementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-typing-deduping')
testImplementation 'org.mockito.kotlin:mockito-kotlin:5.2.1'
testImplementation 'org.mockito.kotlin:mockito-kotlin:5.4.0'

}
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,7 @@ import java.time.Instant
import java.util.Locale
import java.util.Optional
import kotlin.Int
import org.jooq.Condition
import org.jooq.CreateTableColumnStep
import org.jooq.DSLContext
import org.jooq.DataType
import org.jooq.Field
import org.jooq.InsertValuesStepN
import org.jooq.Name
import org.jooq.Record
import org.jooq.SQLDialect
import org.jooq.SelectConditionStep
import org.jooq.SelectFieldOrAsterisk
import org.jooq.*
import org.jooq.conf.ParamType
import org.jooq.impl.DSL
import org.jooq.impl.SQLDataType
Expand Down Expand Up @@ -331,7 +321,7 @@ constructor(
): String {
val hasGenerationId = columns == DestinationColumns.V2_WITH_GENERATION

val createTable: CreateTableColumnStep =
val createTable: CreateTableElementListStep =
dslContext
.createTable(rawTableName)
.column(
Expand Down
10 changes: 5 additions & 5 deletions airbyte-cdk/java/airbyte-cdk/db-sources/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -34,21 +34,21 @@ dependencies {

implementation 'io.debezium:debezium-api:2.4.0.Final'
implementation 'io.debezium:debezium-embedded:2.4.0.Final'
implementation 'org.codehaus.plexus:plexus-utils:4.0.0'
implementation 'org.codehaus.plexus:plexus-utils:4.0.1'

testFixturesImplementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-dependencies')
testFixturesImplementation testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-dependencies'))
testFixturesImplementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core')
testFixturesApi testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core'))

testFixturesImplementation 'net.sourceforge.argparse4j:argparse4j:0.9.0'
testFixturesImplementation 'io.swagger:swagger-annotations:1.6.13'
testFixturesImplementation 'io.swagger:swagger-annotations:1.6.14'
testFixturesImplementation 'org.hamcrest:hamcrest-all:1.3'
testFixturesImplementation 'org.junit.platform:junit-platform-launcher:1.10.1'
testFixturesImplementation 'org.junit.platform:junit-platform-launcher:1.11.0'

testImplementation project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-datastore-postgres')
testImplementation testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-datastore-postgres'))

testImplementation 'uk.org.webcompere:system-stubs-jupiter:2.0.1'
testImplementation 'org.mockito.kotlin:mockito-kotlin:5.2.1'
testImplementation 'uk.org.webcompere:system-stubs-jupiter:2.1.6'
testImplementation 'org.mockito.kotlin:mockito-kotlin:5.4.0'
}
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ class JdbcSSLConnectionUtils {
) {
sslModeConfig[PARAM_CLIENT_KEY_PASSWORD].asText()
} else {
RandomStringUtils.randomAlphanumeric(10)
RandomStringUtils.insecure().nextAlphanumeric(10)
}
return clientKeyPassword
}
Expand Down
50 changes: 25 additions & 25 deletions airbyte-cdk/java/airbyte-cdk/dependencies/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -10,48 +10,48 @@ compileTestKotlin.compilerOptions.allWarningsAsErrors = false
def generate = tasks.register('generate')

dependencies {
api platform('com.fasterxml.jackson:jackson-bom:2.15.2')
api platform('com.fasterxml.jackson:jackson-bom:2.17.2')
api 'com.fasterxml.jackson.core:jackson-annotations'
api 'com.fasterxml.jackson.core:jackson-databind'
api 'com.fasterxml.jackson.dataformat:jackson-dataformat-yaml'
api 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310'
api 'com.fasterxml.jackson.module:jackson-module-kotlin'
api 'com.google.guava:guava:33.0.0-jre'
api 'commons-io:commons-io:2.15.1'
api 'com.google.guava:guava:33.3.0-jre'
api 'commons-io:commons-io:2.16.1'
api ('io.airbyte.airbyte-protocol:protocol-models:0.11.0') { exclude group: 'com.google.api-client', module: 'google-api-client' }
api 'javax.annotation:javax.annotation-api:1.3.2'
api 'org.apache.commons:commons-compress:1.25.0'
api 'org.apache.commons:commons-lang3:3.14.0'
api 'org.apache.logging.log4j:log4j-api:2.21.1'
api 'org.apache.logging.log4j:log4j-core:2.21.1'
api 'org.apache.logging.log4j:log4j-slf4j-impl:2.21.1'
api 'org.apache.logging.log4j:log4j-slf4j2-impl:2.21.1'
api 'org.slf4j:log4j-over-slf4j:2.0.11'
api 'org.slf4j:slf4j-api:2.0.11'
api 'io.github.oshai:kotlin-logging-jvm:6.0.9'
api 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.8.0'


implementation 'com.jayway.jsonpath:json-path:2.7.0'
implementation 'com.networknt:json-schema-validator:1.0.72'
implementation 'commons-cli:commons-cli:1.4'
implementation 'io.swagger:swagger-annotations:1.6.2'
api 'org.apache.commons:commons-compress:1.27.1'
api 'org.apache.commons:commons-lang3:3.17.0'
api 'org.apache.logging.log4j:log4j-api:2.24.0'
api 'org.apache.logging.log4j:log4j-core:2.24.0'
api 'org.apache.logging.log4j:log4j-slf4j-impl:2.24.0'
api 'org.apache.logging.log4j:log4j-slf4j2-impl:2.24.0'
api 'org.slf4j:log4j-over-slf4j:2.0.16'
api 'org.slf4j:slf4j-api:2.0.16'
api 'io.github.oshai:kotlin-logging-jvm:7.0.0'
api 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.8.1'


implementation 'com.jayway.jsonpath:json-path:2.9.0'
implementation 'com.networknt:json-schema-validator:1.5.1'
implementation 'commons-cli:commons-cli:1.9.0'
implementation 'io.swagger:swagger-annotations:1.6.14'
implementation 'javax.validation:validation-api:2.0.1.Final'
implementation 'javax.ws.rs:javax.ws.rs-api:2.1.1'
implementation 'me.andrz.jackson:jackson-json-reference-core:0.3.2' // needed so that we can follow $ref when parsing json
implementation 'org.openapitools:jackson-databind-nullable:0.2.1'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.8.0'
implementation 'org.openapitools:jackson-databind-nullable:0.2.6'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.8.1'

testFixturesApi testFixtures(project(':airbyte-cdk:java:airbyte-cdk:airbyte-cdk-core'))

testFixturesApi ('io.airbyte:airbyte-api:0.55.2') { transitive = false }
testFixturesApi 'org.jetbrains.kotlin:kotlin-test'

testFixturesImplementation 'io.swagger:swagger-annotations:1.6.2'
testFixturesImplementation 'org.apache.ant:ant:1.10.11'
testFixturesImplementation 'io.swagger:swagger-annotations:1.6.14'
testFixturesImplementation 'org.apache.ant:ant:1.10.15'

testImplementation 'com.squareup.okhttp3:mockwebserver:4.9.1'
testImplementation 'org.mockito.kotlin:mockito-kotlin:5.2.1'
testImplementation 'com.squareup.okhttp3:mockwebserver:4.12.0'
testImplementation 'org.mockito.kotlin:mockito-kotlin:5.4.0'
}


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ object Strings {
fun addRandomSuffix(base: String, separator: String, suffixLength: Int): String {
return base +
separator +
RandomStringUtils.randomAlphabetic(suffixLength).lowercase(Locale.getDefault())
RandomStringUtils.insecure()
.nextAlphanumeric(suffixLength)
.lowercase(Locale.getDefault())
}

@JvmStatic
Expand Down
Loading
Loading