From d141d18a86acc75bcac07309f25e83beafecaa07 Mon Sep 17 00:00:00 2001 From: Evan Tahler Date: Thu, 29 Aug 2024 10:03:02 -0700 Subject: [PATCH] Destinations - Increase Jackson parse limit to 100MB (#44476) --- airbyte-cdk/java/airbyte-cdk/README.md | 37 +++++++-------- .../src/main/resources/version.properties | 2 +- .../kotlin/io/airbyte/commons/json/Jsons.kt | 4 +- .../destination-bigquery/build.gradle | 2 +- .../destination-bigquery/metadata.yaml | 2 +- .../destination-redshift/build.gradle | 2 +- .../destination-redshift/metadata.yaml | 2 +- .../connectors/destination-s3/build.gradle | 2 +- .../connectors/destination-s3/metadata.yaml | 2 +- .../destination-snowflake/build.gradle | 2 +- .../destination-snowflake/metadata.yaml | 2 +- docs/integrations/destinations/bigquery.md | 3 +- docs/integrations/destinations/redshift.md | 1 + docs/integrations/destinations/s3.md | 46 +++++++++---------- docs/integrations/destinations/snowflake.md | 7 +-- 15 files changed, 60 insertions(+), 56 deletions(-) diff --git a/airbyte-cdk/java/airbyte-cdk/README.md b/airbyte-cdk/java/airbyte-cdk/README.md index 386fe1df3b1b..bd6ef17a9043 100644 --- a/airbyte-cdk/java/airbyte-cdk/README.md +++ b/airbyte-cdk/java/airbyte-cdk/README.md @@ -2,22 +2,22 @@ This page will walk through the process of developing with the Java CDK. -* [Developing with the Java CDK](#developing-with-the-java-cdk) - * [Intro to the Java CDK](#intro-to-the-java-cdk) - * [What is included in the Java CDK?](#what-is-included-in-the-java-cdk) - * [How is the CDK published?](#how-is-the-cdk-published) - * [Using the Java CDK](#using-the-java-cdk) - * [Building the CDK](#building-the-cdk) - * [Bumping the CDK version](#bumping-the-cdk-version) - * [Publishing the CDK](#publishing-the-cdk) - * [Developing Connectors with the Java CDK](#developing-connectors-with-the-java-cdk) - * [Referencing the CDK from Java connectors](#referencing-the-cdk-from-java-connectors) - * [Developing a connector alongside the CDK](#developing-a-connector-alongside-the-cdk) - * [Publishing the CDK and switching to a pinned CDK reference](#publishing-the-cdk-and-switching-to-a-pinned-cdk-reference) - * [Troubleshooting CDK Dependency Caches](#troubleshooting-cdk-dependency-caches) - * [Developing a connector against a pinned CDK version](#developing-a-connector-against-a-pinned-cdk-version) - * [Changelog](#changelog) - * [Java CDK](#java-cdk) +- [Developing with the Java CDK](#developing-with-the-java-cdk) + - [Intro to the Java CDK](#intro-to-the-java-cdk) + - [What is included in the Java CDK?](#what-is-included-in-the-java-cdk) + - [How is the CDK published?](#how-is-the-cdk-published) + - [Using the Java CDK](#using-the-java-cdk) + - [Building the CDK](#building-the-cdk) + - [Bumping the CDK version](#bumping-the-cdk-version) + - [Publishing the CDK](#publishing-the-cdk) + - [Developing Connectors with the Java CDK](#developing-connectors-with-the-java-cdk) + - [Referencing the CDK from Java connectors](#referencing-the-cdk-from-java-connectors) + - [Developing a connector alongside the CDK](#developing-a-connector-alongside-the-cdk) + - [Publishing the CDK and switching to a pinned CDK reference](#publishing-the-cdk-and-switching-to-a-pinned-cdk-reference) + - [Troubleshooting CDK Dependency Caches](#troubleshooting-cdk-dependency-caches) + - [Developing a connector against a pinned CDK version](#developing-a-connector-against-a-pinned-cdk-version) + - [Changelog](#changelog) + - [Java CDK](#java-cdk) ## Intro to the Java CDK @@ -172,8 +172,9 @@ corresponds to that version. ### Java CDK -| Version | Date | Pull Request | Subject | -|:-----------|:-----------|:-------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Version | Date | Pull Request | Subject | +| :--------- | :--------- | :---------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0.44.19 | 2024-08-20 | [\#44476](https://github.com/airbytehq/airbyte/pull/44476) | Increase Jackson message length limit to 100mb | | 0.44.18 | 2024-08-22 | [\#44505](https://github.com/airbytehq/airbyte/pull/44505) | Improve handling of incoming debezium change events | | 0.44.17 | 2024-08-27 | [\#44832](https://github.com/airbytehq/airbyte/pull/44832) | Fix issues where some error messages with upper cases do not get matched by the error translation framework. | | 0.44.16 | 2024-08-22 | [\#44505](https://github.com/airbytehq/airbyte/pull/44505) | Destinations: add sqlgenerator testing for mixed-case stream name | diff --git a/airbyte-cdk/java/airbyte-cdk/core/src/main/resources/version.properties b/airbyte-cdk/java/airbyte-cdk/core/src/main/resources/version.properties index 400a5cdc6b37..4420b56589c4 100644 --- a/airbyte-cdk/java/airbyte-cdk/core/src/main/resources/version.properties +++ b/airbyte-cdk/java/airbyte-cdk/core/src/main/resources/version.properties @@ -1 +1 @@ -version=0.44.18 +version=0.44.19 diff --git a/airbyte-cdk/java/airbyte-cdk/dependencies/src/main/kotlin/io/airbyte/commons/json/Jsons.kt b/airbyte-cdk/java/airbyte-cdk/dependencies/src/main/kotlin/io/airbyte/commons/json/Jsons.kt index a058887a5cf7..4db3c01202f4 100644 --- a/airbyte-cdk/java/airbyte-cdk/dependencies/src/main/kotlin/io/airbyte/commons/json/Jsons.kt +++ b/airbyte-cdk/java/airbyte-cdk/dependencies/src/main/kotlin/io/airbyte/commons/json/Jsons.kt @@ -29,9 +29,9 @@ private val LOGGER = KotlinLogging.logger {} object Jsons { - // allow jackson to deserialize anything under 50 MiB + // allow jackson to deserialize anything under 100 MiB // (the default, at time of writing 2024-05-29, with jackson 2.15.2, is 20 MiB) - private const val JSON_MAX_LENGTH = 50 * 1024 * 1024 + private const val JSON_MAX_LENGTH = 100 * 1024 * 1024 private val STREAM_READ_CONSTRAINTS = StreamReadConstraints.builder().maxStringLength(JSON_MAX_LENGTH).build() diff --git a/airbyte-integrations/connectors/destination-bigquery/build.gradle b/airbyte-integrations/connectors/destination-bigquery/build.gradle index 42ed81723e86..00691d5a489b 100644 --- a/airbyte-integrations/connectors/destination-bigquery/build.gradle +++ b/airbyte-integrations/connectors/destination-bigquery/build.gradle @@ -3,7 +3,7 @@ plugins { } airbyteJavaConnector { - cdkVersionRequired = '0.44.14' + cdkVersionRequired = '0.44.19' features = [ 'db-destinations', 'datastore-bigquery', diff --git a/airbyte-integrations/connectors/destination-bigquery/metadata.yaml b/airbyte-integrations/connectors/destination-bigquery/metadata.yaml index b1237f20cbe5..7e828f535789 100644 --- a/airbyte-integrations/connectors/destination-bigquery/metadata.yaml +++ b/airbyte-integrations/connectors/destination-bigquery/metadata.yaml @@ -5,7 +5,7 @@ data: connectorSubtype: database connectorType: destination definitionId: 22f6c74f-5699-40ff-833c-4a879ea40133 - dockerImageTag: 2.8.8 + dockerImageTag: 2.8.9 dockerRepository: airbyte/destination-bigquery documentationUrl: https://docs.airbyte.com/integrations/destinations/bigquery githubIssueLabel: destination-bigquery diff --git a/airbyte-integrations/connectors/destination-redshift/build.gradle b/airbyte-integrations/connectors/destination-redshift/build.gradle index 19ed950ebab1..a6011737393d 100644 --- a/airbyte-integrations/connectors/destination-redshift/build.gradle +++ b/airbyte-integrations/connectors/destination-redshift/build.gradle @@ -4,7 +4,7 @@ plugins { } airbyteJavaConnector { - cdkVersionRequired = '0.44.14' + cdkVersionRequired = '0.44.19' features = ['db-destinations', 's3-destinations', 'typing-deduping'] useLocalCdk = false } diff --git a/airbyte-integrations/connectors/destination-redshift/metadata.yaml b/airbyte-integrations/connectors/destination-redshift/metadata.yaml index e0583418f6f3..e4f1a09e3c2b 100644 --- a/airbyte-integrations/connectors/destination-redshift/metadata.yaml +++ b/airbyte-integrations/connectors/destination-redshift/metadata.yaml @@ -5,7 +5,7 @@ data: connectorSubtype: database connectorType: destination definitionId: f7a7d195-377f-cf5b-70a5-be6b819019dc - dockerImageTag: 3.4.3 + dockerImageTag: 3.4.4 dockerRepository: airbyte/destination-redshift documentationUrl: https://docs.airbyte.com/integrations/destinations/redshift githubIssueLabel: destination-redshift diff --git a/airbyte-integrations/connectors/destination-s3/build.gradle b/airbyte-integrations/connectors/destination-s3/build.gradle index a029e46ff734..d4ec9c437feb 100644 --- a/airbyte-integrations/connectors/destination-s3/build.gradle +++ b/airbyte-integrations/connectors/destination-s3/build.gradle @@ -4,7 +4,7 @@ plugins { } airbyteJavaConnector { - cdkVersionRequired = '0.44.15' + cdkVersionRequired = '0.44.19' features = ['db-destinations', 's3-destinations'] useLocalCdk = false } diff --git a/airbyte-integrations/connectors/destination-s3/metadata.yaml b/airbyte-integrations/connectors/destination-s3/metadata.yaml index ed62ba0f14f7..fd9531de7898 100644 --- a/airbyte-integrations/connectors/destination-s3/metadata.yaml +++ b/airbyte-integrations/connectors/destination-s3/metadata.yaml @@ -2,7 +2,7 @@ data: connectorSubtype: file connectorType: destination definitionId: 4816b78f-1489-44c1-9060-4b19d5fa9362 - dockerImageTag: 1.0.2 + dockerImageTag: 1.0.3 dockerRepository: airbyte/destination-s3 githubIssueLabel: destination-s3 icon: s3.svg diff --git a/airbyte-integrations/connectors/destination-snowflake/build.gradle b/airbyte-integrations/connectors/destination-snowflake/build.gradle index 73dfbf7046b7..efa6fbd2dd74 100644 --- a/airbyte-integrations/connectors/destination-snowflake/build.gradle +++ b/airbyte-integrations/connectors/destination-snowflake/build.gradle @@ -3,7 +3,7 @@ plugins { } airbyteJavaConnector { - cdkVersionRequired = '0.44.14' + cdkVersionRequired = '0.44.19' features = ['db-destinations', 's3-destinations', 'typing-deduping'] useLocalCdk = false } diff --git a/airbyte-integrations/connectors/destination-snowflake/metadata.yaml b/airbyte-integrations/connectors/destination-snowflake/metadata.yaml index b51fee363669..99d68c9cb339 100644 --- a/airbyte-integrations/connectors/destination-snowflake/metadata.yaml +++ b/airbyte-integrations/connectors/destination-snowflake/metadata.yaml @@ -5,7 +5,7 @@ data: connectorSubtype: database connectorType: destination definitionId: 424892c4-daac-4491-b35d-c6688ba547ba - dockerImageTag: 3.11.10 + dockerImageTag: 3.11.11 dockerRepository: airbyte/destination-snowflake documentationUrl: https://docs.airbyte.com/integrations/destinations/snowflake githubIssueLabel: destination-snowflake diff --git a/docs/integrations/destinations/bigquery.md b/docs/integrations/destinations/bigquery.md index d0139052f780..702f6e673274 100644 --- a/docs/integrations/destinations/bigquery.md +++ b/docs/integrations/destinations/bigquery.md @@ -210,7 +210,8 @@ tutorials: Expand to review | Version | Date | Pull Request | Subject | -|:--------|:-----------|:-----------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| :------ | :--------- | :--------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2.8.9 | 2024-08-20 | [44476](https://github.com/airbytehq/airbyte/pull/44476) | Increase message parsing limit to 100mb | | 2.8.8 | 2024-08-22 | [44526](https://github.com/airbytehq/airbyte/pull/44526) | Revert protocol compliance fix | | 2.8.7 | 2024-08-15 | [42504](https://github.com/airbytehq/airbyte/pull/42504) | Fix bug in refreshes logic (already mitigated in platform, just fixing protocol compliance) | | 2.8.6 | 2024-07-30 | [42511](https://github.com/airbytehq/airbyte/pull/42511) | Added a copy operation to validate copy permissions in the check function | diff --git a/docs/integrations/destinations/redshift.md b/docs/integrations/destinations/redshift.md index c3c0f8455c2e..29838e92cbc8 100644 --- a/docs/integrations/destinations/redshift.md +++ b/docs/integrations/destinations/redshift.md @@ -222,6 +222,7 @@ Each stream will be output into its own raw table in Redshift. Each table will c | Version | Date | Pull Request | Subject | | :------ | :--------- | :--------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 3.4.4 | 2024-08-20 | [44476](https://github.com/airbytehq/airbyte/pull/44476) | Increase message parsing limit to 100mb | | 3.4.3 | 2024-08-22 | [44526](https://github.com/airbytehq/airbyte/pull/44526) | Revert protocol compliance fix | | 3.4.2 | 2024-08-15 | [42506](https://github.com/airbytehq/airbyte/pull/42506) | Fix bug in refreshes logic (already mitigated in platform, just fixing protocol compliance) | | 3.4.1 | 2024-08-13 | [xxx](https://github.com/airbytehq/airbyte/pull/xxx) | Simplify Redshift Options | diff --git a/docs/integrations/destinations/s3.md b/docs/integrations/destinations/s3.md index 5cb0b55c198e..3beefa7c4bdc 100644 --- a/docs/integrations/destinations/s3.md +++ b/docs/integrations/destinations/s3.md @@ -14,7 +14,6 @@ If you are using STS Assume Role, you must provide the following: - **Role ARN** - Otherwise, if you are using AWS credentials you must provide the following: - **Access Key ID** @@ -305,20 +304,20 @@ sync may create multiple files as the output files can be partitioned by size (t ## Supported sync modes | Feature | Support | Notes | -|:-------------------------------|:-------:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| FullRefresh - Overwrite Sync | ✅ | Warning: this mode deletes all previously synced data in the configured bucket path. | -| Incremental - Append Sync | ✅ | Warning: Airbyte provides at-least-once delivery. Depending on your source, you may see duplicated data. Learn more [here](/using-airbyte/core-concepts/sync-modes/incremental-append#inclusive-cursors) | -| Incremental - Append + Deduped | ❌ | | -| Namespaces | ❌ | Setting a specific bucket path is equivalent to having separate namespaces. | +| :----------------------------- | :-----: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| FullRefresh - Overwrite Sync | ✅ | Warning: this mode deletes all previously synced data in the configured bucket path. | +| Incremental - Append Sync | ✅ | Warning: Airbyte provides at-least-once delivery. Depending on your source, you may see duplicated data. Learn more [here](/using-airbyte/core-concepts/sync-modes/incremental-append#inclusive-cursors) | +| Incremental - Append + Deduped | ❌ | | +| Namespaces | ❌ | Setting a specific bucket path is equivalent to having separate namespaces. | The Airbyte S3 destination allows you to sync data to AWS S3 or Minio S3. Each stream is written to its own directory under the bucket. -⚠️ Please be aware that in "Full Refresh Overwrite Sync" mode, -data from the same generation is retained while all previous data is deleted upon a successful sync. -In case of failures between different generations, data from multiple generations may persist until -a subsequent successful sync. Each S3 object is tagged with `x-amz-meta-ab-generation-id` to identify its -generation. We recommend provisioning a dedicated S3 resource for this sync to avoid accidental data +⚠️ Please be aware that in "Full Refresh Overwrite Sync" mode, +data from the same generation is retained while all previous data is deleted upon a successful sync. +In case of failures between different generations, data from multiple generations may persist until +a subsequent successful sync. Each S3 object is tagged with `x-amz-meta-ab-generation-id` to identify its +generation. We recommend provisioning a dedicated S3 resource for this sync to avoid accidental data deletion due to misconfiguration. ⚠️ ## Supported Output schema @@ -382,10 +381,10 @@ an emission timestamp, and the data blob. With the CSV output, it is possible to \(flatten\) the data blob to multiple columns. | Column | Condition | Description | -|:-------------------------|:---------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------| +| :----------------------- | :------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------- | | `_airbyte_raw_id` | Always exists. | A uuid assigned by Airbyte to each processed record. | | `_airbyte_extracted_at` | Always exists. | A timestamp representing when the event was extracted from the data source. | -| `_airbyte_generation_id` | Always exists. | An integer id that increases with each new refresh. | +| `_airbyte_generation_id` | Always exists. | An integer id that increases with each new refresh. | | `_airbyte_meta` | Always exists. | A structured object containing metadata about the record. | | `_airbyte_data` | When no normalization \(flattening\) is needed, all data resides under this column as a JSON blob. | | | root level fields | When root level normalization \(flattening\) is selected, the root level fields are expanded. | | @@ -393,14 +392,14 @@ an emission timestamp, and the data blob. With the CSV output, it is possible to The schema for `_airbyte_meta` is: | Field Name | Type | Description | -|:-----------|:--------|:----------------------------------------| +| :--------- | :------ | :-------------------------------------- | | `changes` | list | A list of structured change objects. | | `sync_id` | integer | An integer identifier for the sync job. | The schema for a change object is: | Field Name | Type | Description | -|:-----------|:-------|:-------------------------------------------------------------------------------------------------------------------------| +| :--------- | :----- | :----------------------------------------------------------------------------------------------------------------------- | | `field` | string | The name of the field that changed. | | `change` | string | The type of change (eg, `NULLED`, `TRUNCATED`). | | `reason` | string | The reason for the change, including its system of origin (ie, whether it was a source, destination, or platform error). | @@ -420,13 +419,13 @@ For example, given the following JSON object from a source: With no normalization, the output CSV is: | `_airbyte_raw_id` | `_airbyte_extracted_at` | `_airbyte_generation_id` | `_airbyte_meta` | `_airbyte_data` | -|:---------------------------------------|:------------------------|:-------------------------|-------------------------------------|:---------------------------------------------------------------| +| :------------------------------------- | :---------------------- | :----------------------- | ----------------------------------- | :------------------------------------------------------------- | | `26d73cde-7eb1-4e1e-b7db-a4c03b4cf206` | 1622135805000 | 11 | `{"changes":[], "sync_id": 10111 }` | `{ "user_id": 123, name: { "first": "John", "last": "Doe" } }` | With root level normalization, the output CSV is: | `_airbyte_raw_id` | `_airbyte_extracted_at` | `_airbyte_generation_id` | `_airbyte_meta` | `user_id` | `name.first` | `name.last` | -|:---------------------------------------|:------------------------|:-------------------------|-------------------------------------|:---------:|:------------:|:-----------:| +| :------------------------------------- | :---------------------- | :----------------------- | ----------------------------------- | :-------: | :----------: | :---------: | | `26d73cde-7eb1-4e1e-b7db-a4c03b4cf206` | 1622135805000 | 11 | `{"changes":[], "sync_id": 10111 }` | 123 | John | Doe | Output files can be compressed. The default option is GZIP compression. If compression is selected, @@ -536,7 +535,8 @@ To see connector limitations, or troubleshoot your S3 connector, see more [in ou Expand to review | Version | Date | Pull Request | Subject | -|:--------|:-----------|:-----------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------| +| :------ | :--------- | :--------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1.0.3 | 2024-08-20 | [44476](https://github.com/airbytehq/airbyte/pull/44476) | Increase message parsing limit to 100mb | | 1.0.2 | 2024-08-19 | [44401](https://github.com/airbytehq/airbyte/pull/44401) | Fix: S3 Avro/Parquet: handle nullable top-level schema | | 1.0.1 | 2024-08-14 | [42579](https://github.com/airbytehq/airbyte/pull/42579) | OVERWRITE MODE: Deletes deferred until successful sync. | | 1.0.0 | 2024-08-08 | [42409](https://github.com/airbytehq/airbyte/pull/42409) | Major breaking changes: new destination schema, change capture, Avro/Parquet improvements, bugfixes | @@ -574,7 +574,7 @@ To see connector limitations, or troubleshoot your S3 connector, see more [in ou | 0.3.13 | 2022-08-09 | [\#15394](https://github.com/airbytehq/airbyte/pull/15394) | Added LZO compression support to Parquet format | | 0.3.12 | 2022-08-05 | [\#14801](https://github.com/airbytehq/airbyte/pull/14801) | Fix multiple log bindings | | 0.3.11 | 2022-07-15 | [\#14494](https://github.com/airbytehq/airbyte/pull/14494) | Make S3 output filename configurable. | -| 0.3.10 | 2022-06-30 | [\#14332](https://github.com/airbytehq/airbyte/pull/14332) | Change INSTANCE_PROFILE to use `AWSDefaultProfileCredential`, which supports more authentications on AWS | +| 0.3.10 | 2022-06-30 | [\#14332](https://github.com/airbytehq/airbyte/pull/14332) | Change INSTANCE\*PROFILE to use `AWSDefaultProfileCredential`, which supports more authentications on AWS | | 0.3.9 | 2022-06-24 | [\#14114](https://github.com/airbytehq/airbyte/pull/14114) | Remove "additionalProperties": false from specs for connectors with staging | | 0.3.8 | 2022-06-17 | [\#13753](https://github.com/airbytehq/airbyte/pull/13753) | Deprecate and remove PART_SIZE_MB fields from connectors based on StreamTransferManager | | 0.3.7 | 2022-06-14 | [\#13483](https://github.com/airbytehq/airbyte/pull/13483) | Added support for int, long, float data types to Avro/Parquet formats. | @@ -602,17 +602,17 @@ To see connector limitations, or troubleshoot your S3 connector, see more [in ou | 0.1.14 | 2021-11-09 | [\#7732](https://github.com/airbytehq/airbyte/pull/7732) | Support timestamp in Avro and Parquet | | 0.1.13 | 2021-11-03 | [\#7288](https://github.com/airbytehq/airbyte/issues/7288) | Support Json `additionalProperties`. | | 0.1.12 | 2021-09-13 | [\#5720](https://github.com/airbytehq/airbyte/issues/5720) | Added configurable block size for stream. Each stream is limited to 10,000 by S3 | -| 0.1.11 | 2021-09-10 | [\#5729](https://github.com/airbytehq/airbyte/pull/5729) | For field names that start with a digit, a `_` will be appended at the beginning for the`Parquet` and `Avro` formats. | +| 0.1.11 | 2021-09-10 | [\#5729](https://github.com/airbytehq/airbyte/pull/5729) | For field names that start with a digit, a `*` will be appended at the beginning for the`Parquet`and`Avro`formats. | | 0.1.10 | 2021-08-17 | [\#4699](https://github.com/airbytehq/airbyte/pull/4699) | Added json config validator | | 0.1.9 | 2021-07-12 | [\#4666](https://github.com/airbytehq/airbyte/pull/4666) | Fix MinIO output for Parquet format. | | 0.1.8 | 2021-07-07 | [\#4613](https://github.com/airbytehq/airbyte/pull/4613) | Patched schema converter to support combined restrictions. | | 0.1.7 | 2021-06-23 | [\#4227](https://github.com/airbytehq/airbyte/pull/4227) | Added Avro and JSONL output. | | 0.1.6 | 2021-06-16 | [\#4130](https://github.com/airbytehq/airbyte/pull/4130) | Patched the check to verify prefix access instead of full-bucket access. | -| 0.1.5 | 2021-06-14 | [\#3908](https://github.com/airbytehq/airbyte/pull/3908) | Fixed default `max_padding_size_mb` in `spec.json`. | +| 0.1.5 | 2021-06-14 | [\#3908](https://github.com/airbytehq/airbyte/pull/3908) | Fixed default`max_padding_size_mb`in`spec.json`. | | 0.1.4 | 2021-06-14 | [\#3908](https://github.com/airbytehq/airbyte/pull/3908) | Added Parquet output. | | 0.1.3 | 2021-06-13 | [\#4038](https://github.com/airbytehq/airbyte/pull/4038) | Added support for alternative S3. | -| 0.1.2 | 2021-06-10 | [\#4029](https://github.com/airbytehq/airbyte/pull/4029) | Fixed `_airbyte_emitted_at` field to be a UTC instead of local timestamp for consistency. | -| 0.1.1 | 2021-06-09 | [\#3973](https://github.com/airbytehq/airbyte/pull/3973) | Added `AIRBYTE_ENTRYPOINT` in base Docker image for Kubernetes support. | +| 0.1.2 | 2021-06-10 | [\#4029](https://github.com/airbytehq/airbyte/pull/4029) | Fixed `\_airbyte_emitted_at`field to be a UTC instead of local timestamp for consistency. | +| 0.1.1 | 2021-06-09 | [\#3973](https://github.com/airbytehq/airbyte/pull/3973) | Added`AIRBYTE_ENTRYPOINT` in base Docker image for Kubernetes support. | | 0.1.0 | 2021-06-03 | [\#3672](https://github.com/airbytehq/airbyte/pull/3672) | Initial release with CSV output. | diff --git a/docs/integrations/destinations/snowflake.md b/docs/integrations/destinations/snowflake.md index c3ddfac4ef90..a93742b26206 100644 --- a/docs/integrations/destinations/snowflake.md +++ b/docs/integrations/destinations/snowflake.md @@ -267,7 +267,8 @@ desired namespace. Expand to review | Version | Date | Pull Request | Subject | -|:----------------|:-----------|:-----------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| :-------------- | :--------- | :--------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 3.11.11 | 2024-08-20 | [44476](https://github.com/airbytehq/airbyte/pull/44476) | Increase message parsing limit to 100mb | | 3.11.10 | 2024-08-22 | [\#44526](https://github.com/airbytehq/airbyte/pull/44526) | Revert protocol compliance fix | | 3.11.9 | 2024-08-19 | [\#43367](https://github.com/airbytehq/airbyte/pull/43367) | Add opt in using MERGE statement for upserts and deletes | | 3.11.8 | 2024-08-16 | [\#42505](https://github.com/airbytehq/airbyte/pull/42505) | Fix bug in refreshes logic (already mitigated in platform, just fixing protocol compliance) | @@ -279,8 +280,8 @@ desired namespace. | 3.11.2 | 2024-07-12 | [\#41674](https://github.com/airbytehq/airbyte/pull/41674) | Upgrade to latest CDK | | 3.11.1 | 2024-07-08 | [\#41041](https://github.com/airbytehq/airbyte/pull/41041) | Fix resume logic in truncate refreshes to prevent data loss | | 3.11.0 | 2024-06-25 | [\#39473](https://github.com/airbytehq/airbyte/pull/39473) | Support for [refreshes](../../operator-guides/refreshes.md) and resumable full refresh. WARNING: You must upgrade to platform 0.63.7 before upgrading to this connector version. | -| 3.10.1 | 2024-06-11 | [\#39399](https://github.com/airbytehq/airbyte/pull/39399) | Bug fix for _airbyte_meta not migrated in OVERWRITE mode | -| 3.10.0 | 2024-06-10 | [\#39107](https://github.com/airbytehq/airbyte/pull/39107) | _airbyte_meta and _airbyte_generation_id in Raw tables and final tables | +| 3.10.1 | 2024-06-11 | [\#39399](https://github.com/airbytehq/airbyte/pull/39399) | Bug fix for \_airbyte_meta not migrated in OVERWRITE mode | +| 3.10.0 | 2024-06-10 | [\#39107](https://github.com/airbytehq/airbyte/pull/39107) | \_airbyte_meta and \_airbyte_generation_id in Raw tables and final tables | | 3.9.1 | 2024-06-05 | [\#39135](https://github.com/airbytehq/airbyte/pull/39135) | Improved error handling for Staging files | | 3.9.0 | 2024-05-23 | [\#38658](https://github.com/airbytehq/airbyte/pull/38658) | Adapting to newer interfaces from #38107 | | 3.8.4 | 2024-05-23 | [\#38632](https://github.com/airbytehq/airbyte/pull/38632) | convert all tests to kotlin |