Code

Release 0_3_0#128

Code

Merged

subkanthi merged 94 commits intomainAltinity/clickhouse-sink-connector:mainfrom developAltinity/clickhouse-sink-connector:developCopy head branch name to clipboardOct 17, 2022

+14,890-337

Commits

Commits on Sep 12, 2022

• python scripts to run checksums

  

  Arnaud Adant

  committed
  4696573

  Copy full SHA for 4696573

• Adding README.md

  

  Arnaud Adant

  committed
  ad600ef

  Copy full SHA for ad600ef

• Fix time and concurrent runs due to bad indentation

  

  Arnaud Adant

  committed
  0f99a72

  Copy full SHA for 0f99a72

• Adding scripts to test classic databases

  

  Arnaud Adant

  committed
  839be07

  Copy full SHA for 839be07

• Useful settings

  

  Arnaud Adant

  committed
  64362ab

  Copy full SHA for 64362ab

• Fixing the database name

  

  Arnaud Adant

  committed
  0d1d9a2

  Copy full SHA for 0d1d9a2

Commits on Sep 13, 2022

• Merge pull request #83 from aadant/mysql_config

  Show description for 5cfcb3d
  subkanthi

  authored
  5cfcb3d

  Copy full SHA for 5cfcb3d

• Merge pull request #82 from aadant/test_databases

  Show description for 465537b
  subkanthi

  authored
  465537b

  Copy full SHA for 465537b

• Merge pull request #80 from aadant/table_checksums

  Show description for 09a228a
  subkanthi

  authored
  09a228a

  Copy full SHA for 09a228a

Commits on Sep 14, 2022

• Changed camel case functions to snake case.

  subkanthi

  committed
  12415c3

  Copy full SHA for 12415c3

Commits on Sep 15, 2022

• Added steps to download menagerie db and apply to docker image.

  subkanthi

  committed
  adee53b

  Copy full SHA for adee53b

• Change version and sign column to _version and _sign column to align with MaterializedMySQL.

  subkanthi

  committed
  5b9b27f

  Copy full SHA for 5b9b27f

• Set snapshot mode to initial and add customers to initial snapshot.

  subkanthi

  committed
  f5bb81b

  Copy full SHA for f5bb81b

• Added test files that were moved.

  subkanthi

  committed
  e8bec2a

  Copy full SHA for e8bec2a

• Commented out timezone dependent tests.

  subkanthi

  committed
  3187eb2

  Copy full SHA for 3187eb2

• Comment out checkstlye in github actions.

  subkanthi

  committed
  8d3c3ec

  Copy full SHA for 8d3c3ec

• Merge pull request #88 from Altinity/78-the-virtual-columns-should-be-_version-and-_sign

  Show description for 79d0c86
  subkanthi

  authored
  79d0c86

  Copy full SHA for 79d0c86

Commits on Sep 16, 2022

• Remove zip files in meanagerie database.

  subkanthi

  committed
  4d4941a

  Copy full SHA for 4d4941a

• Merge pull request #86 from Altinity/minor_fixes_checksum

  Show description for 1902c37
  subkanthi

  authored
  1902c37

  Copy full SHA for 1902c37

Commits on Sep 19, 2022

• Added documentation for Confluent Schema Registry REST API.

  subkanthi

  committed
  fcf50b9

  Copy full SHA for fcf50b9

• Revert back adding sakila DB to init mysql scripts

  subkanthi

  committed
  1cb2751

  Copy full SHA for 1cb2751

• Revert back adding sakila DB to init mysql scripts

  subkanthi

  committed
  4001165

  Copy full SHA for 4001165

• Merge pull request #93 from Altinity/77-add-sakila-employees-database

  Show description for 8417afc
  subkanthi

  authored
  8417afc

  Copy full SHA for 8417afc

• Merge pull request #91 from Altinity/schema_registry_docs

  Show description for 6c59d56
  subkanthi

  authored
  6c59d56

  Copy full SHA for 6c59d56

Commits on Sep 20, 2022

• Added logic to transform GEOMETRY to string.

  Show description for 4e798c5
  subkanthi

  committed
  4e798c5

  Copy full SHA for 4e798c5

Commits on Sep 21, 2022

• Modified mapping of datetime to datetime(3)

  subkanthi

  committed
  466bf9f

  Copy full SHA for 466bf9f

Commits on Sep 22, 2022

• clickhouse_load initial release

  

  Arnaud Adant

  committed
  fe9afd4

  Copy full SHA for fe9afd4

• Redo case

  

  Arnaud Adant

  committed
  db1d08a

  Copy full SHA for db1d08a

• Fix case

  

  Arnaud Adant

  committed
  ffa0381

  Copy full SHA for ffa0381

• Fixed unit tests related to JSON column change.

  subkanthi

  committed
  d0bbf64

  Copy full SHA for d0bbf64

• Make local modules visible to python

  

  Arnaud Adant

  committed
  9190517

  Copy full SHA for 9190517

Commits on Sep 23, 2022

• Merge pull request #96 from aadant/clickhouse_loader

  Show description for 4eeaa8e
  subkanthi

  authored
  4eeaa8e

  Copy full SHA for 4eeaa8e

• better support for dates, schema generation fixes for fractional dates and timestamps

  

  Arnaud Adant

  committed
  0fd3c6d

  Copy full SHA for 0fd3c6d

• Changed JSON mapping to CH string.

  subkanthi

  committed
  25b0b5c

  Copy full SHA for 25b0b5c

• Adding instructions

  

  Arnaud Adant

  committed
  b415b97

  Copy full SHA for b415b97

• Adding instructions

  

  Arnaud Adant

  committed
  e1dae9c

  Copy full SHA for e1dae9c

• Merge pull request #97 from aadant/clickhouse_loader

  Show description for 3d9b5be
  aadant

  authored
  3d9b5be

  Copy full SHA for 3d9b5be

Commits on Sep 25, 2022

• Merge pull request #98 from Altinity/sakila_db_fixes

  Show description for c5a8479
  subkanthi

  authored
  c5a8479

  Copy full SHA for c5a8479

Commits on Sep 26, 2022

• Removed init scripts for MySQL and ClickHouse. Added logic to download sakila employees data and script to perform checksum comparison.

  subkanthi

  committed
  9bf36ce

  Copy full SHA for 9bf36ce

Commits on Sep 27, 2022

• Updated sysbench scripts to iterate through multiple tests and perform a diff.

  subkanthi

  committed
  1aeb7b8

  Copy full SHA for 1aeb7b8

• Updated sysbench scripts to iterate through multiple tests and perform a diff.

  subkanthi

  committed
  0b58bdd

  Copy full SHA for 0b58bdd

• Merge pull request #103 from Altinity/sakila_db_fixes

  Show description for d3adb12
  subkanthi

  authored
  d3adb12

  Copy full SHA for d3adb12

Commits on Sep 28, 2022

• Change logic of creating input batch by keeping the CH columns as source of truth.

  subkanthi

  committed
  7063b49

  Copy full SHA for 7063b49

• Change logic of creating input batch by keeping the CH columns as source of truth.

  subkanthi

  committed
  ec3a3d6

  Copy full SHA for ec3a3d6

• Merge remote-tracking branch 'origin/refactor_insert_batch' into refactor_insert_batch

  subkanthi

  committed
  cd3cb6e

  Copy full SHA for cd3cb6e

Commits on Sep 29, 2022

• Move hardcoded strings to separate class.

  subkanthi

  committed
  c4b7110

  Copy full SHA for c4b7110

• Move hardcoded strings to separate class.

  subkanthi

  committed
  1e09ba8

  Copy full SHA for 1e09ba8

Commits on Sep 30, 2022

• Support most data types

  

  Arnaud Adant

  committed
  4c5c21b

  Copy full SHA for 4c5c21b

• Adding credits

  

  Arnaud Adant

  committed
  0ddce4d

  Copy full SHA for 0ddce4d

• data_types database

  

  Arnaud Adant

  committed
  22a0067

  Copy full SHA for 22a0067

• Merge pull request #112 from aadant/clickhouse_loader

  Show description for 5f502ca
  subkanthi

  authored
  5f502ca

  Copy full SHA for 5f502ca

• Changed Date/DateTime lower/upper limit to use clickhouse-jdbc limits.

  subkanthi

  committed
  6875a43

  Copy full SHA for 6875a43

• Fix QueryFormatterTest unit tests.

  subkanthi

  committed
  b8b256e

  Copy full SHA for b8b256e

• Fix QueryFormatterTest unit tests.

  subkanthi

  committed
  3c558d9

  Copy full SHA for 3c558d9

Commits on Oct 2, 2022

• Added unit test for date/datetime fields.

  subkanthi

  committed
  fa664fa

  Copy full SHA for fa664fa

Commits on Oct 3, 2022

• Changed date limit logic to use epoch in Days. Updated checksum to use CH date limits.

  subkanthi

  committed
  a6526c2

  Copy full SHA for a6526c2

• Change logic of creating input batch by keeping the CH columns as source of truth.

  subkanthi

  committed
  cd367a4

  Copy full SHA for cd367a4

• Move hardcoded strings to separate class.

  subkanthi

  committed
  4366662

  Copy full SHA for 4366662

• Move hardcoded strings to separate class.

  subkanthi

  committed
  72642c7

  Copy full SHA for 72642c7

• Changed Date/DateTime lower/upper limit to use clickhouse-jdbc limits.

  subkanthi

  committed
  9ec1666

  Copy full SHA for 9ec1666

• Fix QueryFormatterTest unit tests.

  subkanthi

  committed
  253ec46

  Copy full SHA for 253ec46

• Fix QueryFormatterTest unit tests.

  subkanthi

  committed
  d82f49a

  Copy full SHA for d82f49a

• Added unit test for date/datetime fields.

  subkanthi

  committed
  1b925ad

  Copy full SHA for 1b925ad

• Changed date limit logic to use epoch in Days. Updated checksum to use CH date limits.

  subkanthi

  committed
  798e22c

  Copy full SHA for 798e22c

• Merge branch 'refactor_insert_batch' of github.com:Altinity/clickhouse-sink-connector into refactor_insert_batch

  subkanthi

  committed
  394c177

  Copy full SHA for 394c177

Commits on Oct 4, 2022

• upd4oct

  AlmostIvanSidorov

  committed
  3aaa168

  Copy full SHA for 3aaa168

• "upd"

  AlmostIvanSidorov

  committed
  82927b3

  Copy full SHA for 82927b3

Commits on Oct 5, 2022

• Support all MySQL data types + bug fix

  

  Arnaud Adant

  committed
  c2d671e

  Copy full SHA for c2d671e

• Support all MySQL data types + bug fix

  

  Arnaud Adant

  committed
  009e816

  Copy full SHA for 009e816

• Support basic partitioning by range

  

  Arnaud Adant

  committed
  85c0830

  Copy full SHA for 85c0830

• "Date->Date32, DateTime -> DateTime64, JSON -> String and Year -> INT32"

  AlmostIvanSidorov

  committed
  d0a77b4

  Copy full SHA for d0a77b4

• Added unit tests for DateTime limits.

  subkanthi

  committed
  df2a153

  Copy full SHA for df2a153

• "insert, alter, delete, update data consistency upd"

  AlmostIvanSidorov

  committed
  5aa089b

  Copy full SHA for 5aa089b

• Merge pull request #117 from aadant/clickhouse_loader

  Show description for cc9fadf
  subkanthi

  authored
  cc9fadf

  Copy full SHA for cc9fadf

• Added logic to override datetime64 mapping for auto create table, set DateTime64(3) and DateTime64(6)

  subkanthi

  committed
  1e08d2f

  Copy full SHA for 1e08d2f

• Fixed Debezium time converter test.

  subkanthi

  committed
  108172a

  Copy full SHA for 108172a

Commits on Oct 6, 2022

• Changed logic to use setTimestamp in JDBC for writing DateTime fields with microseconds.

  subkanthi

  committed
  c65a68c

  Copy full SHA for c65a68c

Commits on Oct 7, 2022

• Merge pull request #115 from Altinity/TestFlows_integration

  Show description for 60f4f77
  subkanthi

  authored
  60f4f77

  Copy full SHA for 60f4f77

• Add meaningful digits to data_types.sql

  

  Arnaud Adant

  committed
  611abda

  Copy full SHA for 611abda

Commits on Oct 8, 2022

• Fixed TIME range

  subkanthi

  committed
  e6803c7

  Copy full SHA for e6803c7

• Merge pull request #121 from aadant/clickhouse_loader

  Show description for ae1d6dc
  subkanthi

  authored
  ae1d6dc

  Copy full SHA for ae1d6dc

• Map TIMESTAMP to DateTime64(3)

  subkanthi

  committed
  19d621f

  Copy full SHA for 19d621f

• Merge pull request #110 from Altinity/refactor_insert_batch

  Show description for e7a9a99
  subkanthi

  authored
  e7a9a99

  Copy full SHA for e7a9a99

Commits on Oct 11, 2022

• Added to logic to remove trailing zeros in timestamp and time columns. Updated unit tests.

  subkanthi

  committed
  9eeb56c

  Copy full SHA for 9eeb56c

Commits on Oct 13, 2022

• Changed mysql checksum calculation to cast datetime(1, 2, 3) to datetime(3) and datetime(4, 5, 6) to datetime(6) so that it matches CH results.

  subkanthi

  committed
  0c4774c

  Copy full SHA for 0c4774c

• Changed mysql checksum calculation to remove trailing zeros in timestamp.

  subkanthi

  committed
  7cac23b

  Copy full SHA for 7cac23b

Commits on Oct 14, 2022

• Refactor datatypes diff script to query the table names from information_schema.

  subkanthi

  committed
  fddee83

  Copy full SHA for fddee83

Commits on Oct 17, 2022

• Map BYTES, bits to String in ClickHouse.

  subkanthi

  committed
  abc29e8

  Copy full SHA for abc29e8

• Merge pull request #126 from Altinity/122-remove-trailing-zeros-in-timestamp-and-time-columns

  Show description for df694ea
  subkanthi

  authored
  df694ea

  Copy full SHA for df694ea

• Updating README.md

  subkanthi

  committed
  9564451

  Copy full SHA for 9564451

• Merge pull request #127 from Altinity/119-document-supported-versions-of-ch-mysql-and-kafka-in-readmemd

  Show description for b19e5a0
  subkanthi

  authored
  b19e5a0

  Copy full SHA for b19e5a0

• Map Float64(Debezium) to Float32(CH)

  subkanthi

  committed
  399db88

  Copy full SHA for 399db88

• Merge pull request #131 from Altinity/fix_links

  Show description for 5d07702
  subkanthi

  authored
  5d07702

  Copy full SHA for 5d07702

• Resolved conflicts

  subkanthi

  committed
  ce76f4e

  Copy full SHA for ce76f4e