Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DateTime parsing Exception #54

Closed
subkanthi opened this issue Aug 26, 2022 · 9 comments
Closed

DateTime parsing Exception #54

subkanthi opened this issue Aug 26, 2022 · 9 comments
Assignees
@subkanthi
Labels
bug Something isn't working datatype Related to supporting or bugs with current supported datatypes.

Comments

@subkanthi
Copy link
Collaborator

UInt64,sign Int8') [com.altinity.clickhouse.sink.connector.db.DbWriter]
2022-08-26 02:46:17,652 ERROR || ******* ERROR inserting Batch ***************** [com.altinity.clickhouse.sink.connector.db.DbWriter]
java.time.format.DateTimeParseException: Text '2020-09-01T12:45:17' could not be parsed at index 10
at java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2046)
at java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1948)
at java.base/java.time.LocalDateTime.parse(LocalDateTime.java:492)
at com.clickhouse.client.data.ClickHouseDateTimeValue.update(ClickHouseDateTimeValue.java:377)
at com.clickhouse.client.data.ClickHouseDateTimeValue.update(ClickHouseDateTimeValue.java:24)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.setString(InputBasedPreparedStatement.java:265)
at com.altinity.clickhouse.sink.connector.db.DbWriter.insertPreparedStatement(DbWriter.java:630)
at com.altinity.clickhouse.sink.connector.db.DbWriter.addToPreparedStatementBatch(DbWriter.java:418)
at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.flushRecordsToClickHouse(ClickHouseBatchRunnable.java:228)
at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.processRecordsByTopic(ClickHouseBatchRunnable.java:189)
at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.run(ClickHouseBatchRunnable.java:105)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-08-26 02:46:17,654 ERROR || ClickHouseBatchRunnable exception - Task(0) [com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable]
java.lang.NullPointerException
at com.altinity.clickhouse.sink.connector.common.Metrics.updateCounters(Metrics.java:226)
at com.altinity.clickhouse.sink.connector.db.DbWriter.addToPreparedStatementBatch(DbWriter.java:456)
at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.flushRecordsToClickHouse(ClickHouseBatchRunnable.java:228)
at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.processRecordsByTopic(ClickHouseBatchRunnable.java:189)
at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.run(ClickHouseBatchRunnable.java:105)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
@subkanthi subkanthi added the bug Something isn't working label Aug 26, 2022
@subkanthi subkanthi self-assigned this Aug 26, 2022
@subkanthi
Copy link
Collaborator Author

Fixed in #52

@aadant
Copy link
Collaborator

aadant commented Sep 11, 2022

This issue still occurs in the latest connector :

022-09-11 04:20:53,153 ERROR  ||  ******* ERROR inserting Batch *****************   [com.altinity.clickhouse.sink.connector.db.DbWriter]
java.time.format.DateTimeParseException: Text '2005-05-25T11:30:37' could not be parsed at index 10
        at java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2046)
        at java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1948)
        at java.base/java.time.LocalDateTime.parse(LocalDateTime.java:492)
        at com.clickhouse.client.data.ClickHouseDateTimeValue.update(ClickHouseDateTimeValue.java:377)
        at com.clickhouse.client.data.ClickHouseDateTimeValue.update(ClickHouseDateTimeValue.java:24)
        at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.setString(InputBasedPreparedStatement.java:265)
        at com.altinity.clickhouse.sink.connector.db.DbWriter.insertPreparedStatement(DbWriter.java:630)
        at com.altinity.clickhouse.sink.connector.db.DbWriter.addToPreparedStatementBatch(DbWriter.java:418)
        at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.flushRecordsToClickHouse(ClickHouseBatchRunnable.java:228)
        at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.processRecordsByTopic(ClickHouseBatchRunnable.java:189)
        at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.run(ClickHouseBatchRunnable.java:105)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

@aadant aadant reopened this Sep 11, 2022
@aadant
Copy link
Collaborator

aadant commented Sep 11, 2022

Can not reproduce in the latest version. My mistake.

@aadant aadant closed this as completed Sep 11, 2022
@HelloSunilSaini
Copy link

HelloSunilSaini commented Oct 12, 2022 •
edited
Loading

@subkanthi getting the same error with apicurio schema registry this is if datetime string have milliseconds also in ZonedTimestamp
Error:

2022-10-12 11:07:39,287 ERROR [my-sink-connector|task-0] ******* ERROR inserting Batch ***************** (com.altinity.clickhouse.sink.connector.db.DbWriter) [pool-4-thread-6]
java.time.format.DateTimeParseException: Text '2022-06-21T11:15:03.960428Z' could not be parsed at index 19
	at java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2046)
	at java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1948)
	at java.base/java.time.LocalDateTime.parse(LocalDateTime.java:492)
	at com.altinity.clickhouse.sink.connector.converters.DebeziumConverter$ZonedTimestampConverter.convert(DebeziumConverter.java:140)
	at com.altinity.clickhouse.sink.connector.db.DbWriter.insertPreparedStatement(DbWriter.java:595)
	at com.altinity.clickhouse.sink.connector.db.DbWriter.addToPreparedStatementBatch(DbWriter.java:418)
	at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.flushRecordsToClickHouse(ClickHouseBatchRunnable.java:228)
	at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.processRecordsByTopic(ClickHouseBatchRunnable.java:189)
	at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.run(ClickHouseBatchRunnable.java:105)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

@subkanthi
Copy link
Collaborator Author

@HelloSunilSaini please use this docker tag 2022-10-08 , will tag this as latest today.

@subkanthi subkanthi added the datatype Related to supporting or bugs with current supported datatypes. label Oct 12, 2022
@HelloSunilSaini
Copy link

@subkanthi Hey i am using this image subkanthi/clickhouse-kafka-sink-connector-strimzi:latest as running strimzi kafka not able to find tag for that can you provide git commit / git branch which have code for this fix

@subkanthi
Copy link
Collaborator Author

@HelloSunilSaini , here is the link to the docker hub page , please use develop branch. https://hub.docker.com/layers/altinity/clickhouse-sink-connector/2022-10-08/images/sha256-fe5dc42a5bbaefe5c0aac2312315989a2423a489a637deb6779a7abbf0d4a5b7?context=repo

@HelloSunilSaini
Copy link

HelloSunilSaini commented Oct 13, 2022 •
edited
Loading

@subkanthi while i am running the above image getting error

Error: failed to start container "my-sink-connect-cluster-connect": Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/opt/kafka/kafka_connect_run.sh": stat /opt/kafka/kafka_connect_run.sh: no such file or directory: unknown

here is my kafkaConnect file

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-sink-connect-cluster
  annotations:
    strimzi.io/use-connector-resources: "true"
spec:
  image: altinity/clickhouse-sink-connector:2022-10-08 
  replicas: 1
  bootstrapServers: data-pipeline-cluster-kafka-bootstrap:9093
  tls:
    trustedCertificates:
      - secretName: data-pipeline-cluster-cluster-ca-cert
        certificate: ca.crt
  config:
    group.id: sink-connect-cluster
    config.storage.topic: config-storage-topic-sink-2
    offset.storage.topic: offset-storage-topic-sink-2
    status.storage.topic: status-storage-topic-sink-2
    config.storage.replication.factor: 3
    offset.storage.replication.factor: 3
    status.storage.replication.factor: 3

also tried this image

https://hub.docker.com/layers/hellosunilsaini/clickhouse-sink-connector-strimzi/1.0.1/images/sha256-5f8d3accde402cc04ff5c30162b2c28f3ea007aa3890ff00540bbded880c860b?context=explore

which i have build from clickhouse-sink-connector/docker/Dockerfile-sink-on-strimzi-image with this image getting error as

exec /opt/kafka/kafka_connect_run.sh: exec format error

i am not able to see changes in images 1 & 2 but 1 is working just getting error for ZonedDateTime

  1. https://hub.docker.com/layers/subkanthi/clickhouse-kafka-sink-connector-strimzi/latest/images/sha256-8a35c6c964f2da25cd1aaf2b3535b9e59af81c36174ba92fcf92e66e907e991d?context=explore
  2. https://hub.docker.com/layers/hellosunilsaini/clickhouse-sink-connector-strimzi/1.0.1/images/sha256-5f8d3accde402cc04ff5c30162b2c28f3ea007aa3890ff00540bbded880c860b?context=repo

@subkanthi
Copy link
Collaborator Author

@HelloSunilSaini will check and will update the instructions for running on k8s(using strimzi)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@subkanthi subkanthi

Labels
bug Something isn't working datatype Related to supporting or bugs with current supported datatypes.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@aadant @subkanthi @HelloSunilSaini