Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot add new dataset from prestodb due to too long type character varying #14458

Closed
jawher-soudani opened this issue May 4, 2021 · 3 comments
Labels
#bug Bug report data:connect:presto Related to Presto inactive Inactive for >= 30 days P3 Priority item - Normal

Comments

@jawher-soudani
Copy link

I'm facing this problem whenever I try to add a dataset from a presto engine
superset_app | psycopg2.errors.StringDataRightTruncation: value too long for type character varying(32)

Here is a detailed log:

(Background on this error at: http://sqlalche.me/e/13/9h9h)
superset_app            | Error creating model DatasetRestApi: Dataset could not be created.
superset_app            | Traceback (most recent call last):
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
superset_app            |     cursor, statement, parameters, context
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
superset_app            |     cursor.execute(statement, parameters)
superset_app            | psycopg2.errors.StringDataRightTruncation: value too long for type character varying(32)
superset_app            | 
superset_app            | 
superset_app            | The above exception was the direct cause of the following exception:
superset_app            | 
superset_app            | Traceback (most recent call last):
superset_app            |   File "/app/superset/datasets/commands/create.py", line 52, in run
superset_app            |     dataset.fetch_metadata(commit=False)
superset_app            |   File "/app/superset/connectors/sqla/models.py", line 1538, in fetch_metadata
superset_app            |     self.add_missing_metrics(metrics)
superset_app            |   File "/app/superset/connectors/base/models.py", line 194, in add_missing_metrics
superset_app            |     existing_metrics = {m.metric_name for m in self.metrics}
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 287, in __get__
superset_app            |     return self.impl.get(instance_state(instance), dict_)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 723, in get
superset_app            |     value = self.callable_(state, passive)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 760, in _load_for_state
superset_app            |     session, state, primary_key_identity, passive
superset_app            |   File "<string>", line 1, in <lambda>
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 902, in _emit_lazyload
superset_app            |     .with_post_criteria(set_default_params)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 544, in all
superset_app            |     return list(self)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 439, in __iter__
superset_app            |     self.session._autoflush()
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1629, in _autoflush
superset_app            |     util.raise_(e, with_traceback=sys.exc_info()[2])
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
superset_app            |     raise exception
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1618, in _autoflush
superset_app            |     self.flush()
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2536, in flush
superset_app            |     self._flush(objects)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2678, in _flush
superset_app            |     transaction.rollback(_capture_exception=True)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
superset_app            |     with_traceback=exc_tb,
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
superset_app            |     raise exception
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2638, in _flush
superset_app            |     flush_context.execute()
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
superset_app            |     rec.execute(self)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/unitofwork.py", line 589, in execute
superset_app            |     uow,
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
superset_app            |     insert,
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 1136, in _emit_insert_statements
superset_app            |     statement, params
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
superset_app            |     return meth(self, multiparams, params)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
superset_app            |     return connection._execute_clauseelement(self, multiparams, params)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
superset_app            |     distilled_params,
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
superset_app            |     e, statement, parameters, cursor, context
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
superset_app            |     sqlalchemy_exception, with_traceback=exc_info[2], from_=e
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
superset_app            |     raise exception
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
superset_app            |     cursor, statement, parameters, context
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
superset_app            |     cursor.execute(statement, parameters)
superset_app            | sqlalchemy.exc.DataError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
superset_app            | (psycopg2.errors.StringDataRightTruncation) value too long for type character varying(32)
superset_app            | 
superset_app            | [SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(created_by_fk)s, %(changed_by_fk)s) RETURNING table_columns.id]
superset_app            | [parameters: {'uuid': UUID('de602348-5aba-4521-88d1-5c0871667490'), 'created_on': datetime.datetime(2021, 5, 4, 10, 52, 48, 305161), 'changed_on': datetime.datetime(2021, 5, 4, 10, 52, 48, 305169), 'column_name': 'event_time', 'verbose_name': None, 'is_active': True, 'type': "<class 'superset.models.sql_types.base.literal_dttm_type_factory.<locals>.TemporalWrapperType'>", 'groupby': True, 'filterable': True, 'description': None, 'table_id': 54, 'is_dttm': False, 'expression': None, 'python_date_format': None, 'created_by_fk': 1, 'changed_by_fk': 1}]

I didn't find where in supertset I can alter types parameters to increase max chars.
I got these issues when connecting to a presto engine, the presto engine is connected to a postgresql database , and when I use superset directly on postgresql to add the same table it worked fine , but when I pass through presto which is supposed to do the same , I got this error.
Using presto cli to check tables on postgresql database does not show any error and I'm able to fetch the table so I assume the problem is related to apache superset

@jawher-soudani jawher-soudani added the #bug Bug report label May 4, 2021
@junlincc junlincc added data:connect:presto Related to Presto P3 Priority item - Normal labels May 21, 2021
@ganczarek
Copy link
Contributor

#13572 looks like the same issue.

@stale
Copy link

stale bot commented Apr 17, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

@stale stale bot added the inactive Inactive for >= 30 days label Apr 17, 2022
@rusackas
Copy link
Member

rusackas commented Feb 2, 2023

Closed by #17360

@rusackas rusackas closed this as completed Feb 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
#bug Bug report data:connect:presto Related to Presto inactive Inactive for >= 30 days P3 Priority item - Normal
Projects
None yet
Development

No branches or pull requests

4 participants