clickhouse_loader.py : support schema overrides in yaml

[aadant]

We should be able to override the schema using a yaml file

• data type mapping

• default table settings

• default type conversion

  • choice between LowCardinality versus String
  • choice how to convert binary types hex / base64

• custom table settings per regex / table

  • sort key
  • column data type
  • partitioning key
  • table settings
  • table engine

• filtering out a table from migration / replication

  • include via list and regex
  • exclude via list and regex

[subkanthi]

Filtering
filtering out a table from migration / replication

include via list and regex
exclude via list and regex
This is already supported in Debezium using table.include.list and table.exclude.list.

[subkanthi]

default_column_datatype_mapping:
    # we are no longer turning Date/DateTime/Timestamp as a String
    transaction_id: String
    exchange_transaction_id: String
    unique_transaction_id: String
    account_ref: String
    otm_identifier: String
    tag_reserved_4: String
    initiator: String

databases:
    dbo:
         tables:
           tr_live:
                partition_by: tr_date_id
                primary_key: gmt_time
                settings: allow_nullable_key=1