Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: TIMESTAMP_NTZ() fails when casting to python type while sorting types #59

Closed
pnadolny13 opened this issue Jun 14, 2023 · 0 comments · Fixed by #60
Closed

bug: TIMESTAMP_NTZ() fails when casting to python type while sorting types #59

pnadolny13 opened this issue Jun 14, 2023 · 0 comments · Fixed by #60
Assignees
@pnadolny13

Comments

@pnadolny13
Copy link
Contributor

I have a table with a column _SDC_DELETED_AT that is of type VARCHAR(16777216). When I switch to this target it wants to change that to TIMESTAMP_NTZ() (rightfully so) but it fails at this line https://github.com/meltano/sdk/blob/3ceb28a97a77a5e22163fc887fe16ff129c2dc28/singer_sdk/connectors/sql.py#L912 when trying to adapt the column type because the snowflake SQLAlchemy type TIMESTAMP_NTZ() doesnt implement python_type https://github.com/snowflakedb/snowflake-sqlalchemy/blob/c200f971a745eb404ef68b9f8a248c7255056dc9/src/snowflake/sqlalchemy/custom_types.py#L56.
@pnadolny13 pnadolny13 changed the title bug: TIMESTAMP_NTZ() fails in type casting bug: TIMESTAMP_NTZ() fails when casting to python type while sorting types Jun 14, 2023
@pnadolny13 pnadolny13 mentioned this issue Jun 14, 2023
Merged
3 tasks
@pnadolny13 pnadolny13 self-assigned this Jun 14, 2023
@pnadolny13 pnadolny13 moved this to In Progress in MeltanoLabs Overview Jun 14, 2023
@pnadolny13 pnadolny13 mentioned this issue Jun 14, 2023
Merged
pnadolny13 added a commit that referenced this issue Jun 14, 2023
@pnadolny13
fix: custom snowflake type casting to python type (#60)
e1bb4c8 
Closes #59

I added a test that seeds a table before trying to load more data into
it. You can see the error and how it fails in
https://github.com/MeltanoLabs/target-snowflake/actions/runs/5271361651/jobs/9532121838?pr=60#step:6:118.

To fix it I overrode the custom snowflake type classes that we were
using and implemented the `python_type` property for each.
@github-project-automation github-project-automation bot moved this from In Progress to Done in MeltanoLabs Overview Jun 14, 2023
@github-project-automation github-project-automation bot moved this to Planned in Data Team Jun 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pnadolny13 pnadolny13

Labels
None yet
Projects
MeltanoLabs Overview
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: custom snowflake type casting to python type MeltanoLabs/target-snowflake
1 participant
@pnadolny13