superset 2.0.1 dashboard and charts async update

[leagueofsoups]

I can't use celery workers for update dashboard and charts

How to reproduce the bug

1. Use this configuration fragment

class CeleryConfig(object):
    broker_url = "redis://<fix_it>/0"
    imports = (
      'superset.sql_lab',
      'superset.tasks',
      'superset.tasks.thumbnails',
      'superset.charts',
      'superset.dashboards'
    )
    result_backend = "redis://<fix_it>/0"
    worker_log_level = 'DEBUG'
    worker_prefetch_multiplier = 10
    task_acks_late = True

    task_annotations = {
        "tasks.add": {"rate_limit": "10/s"},
        "sql_lab.get_sql_results": {"rate_limit": "100/s"},
        "email_reports.send": {
            "rate_limit": "1/s",
            "time_limit": 120,
            "soft_time_limit": 150,
            "ignore_result": True,
        },
    }
    beat_schedule = {
        "email_reports.schedule_hourly": {
            "task": "email_reports.schedule_hourly",
            "schedule": crontab(minute=1, hour="*"),
        },

    }
CELERY_CONFIG = CeleryConfig

2. Add async parametres

GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
    "port": 6379,
    "host": "<FIXIT>",
    "password": "<FIXIT>",
    "db": 3,
    "ssl": False,
}
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_PREFIX = "async-events-"
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT = 1000
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT_FIREHOSE = 1000000
GLOBAL_ASYNC_QUERIES_JWT_COOKIE_NAME = "async-token"
GLOBAL_ASYNC_QUERIES_JWT_COOKIE_SECURE = False
GLOBAL_ASYNC_QUERIES_JWT_COOKIE_DOMAIN = None
GLOBAL_ASYNC_QUERIES_JWT_SECRET =  "<FIXIT>" 
GLOBAL_ASYNC_QUERIES_TRANSPORT = "polling"
GLOBAL_ASYNC_QUERIES_POLLING_DELAY = int(
    timedelta(milliseconds=500).total_seconds() * 1000
)

3. add flag

FEATURE_FLAGS = {
'GLOBAL_ASYNC_QUERIES': True,
...

4. enable Advanced > Performance > Asynchronous query execution for datasource
5. open dashboards/chard and use this datasource (force refrash)

Expected results

Found async tasks in celery
This issues have nice picture in description #9190
How I can execute dashboards/charts update query on my celery workers?

Actual results

Have only this tasks names:
load_chart_data_into_cache
sql_lab.get_sql_results
load_explore_json_into_cache

Environment

• browser type and version: any
• superset version: 2.0.1
• python version: Python 3.8.13
• node.js version: without node js
• any feature flags active: GLOBAL_ASYNC_QUERIES + ENABLE_TEMPLATE_PROCESSING

Checklist

• I Have async tasks

Additional context

pip freeze

aiohttp==3.8.3
aiosignal==1.3.1
alembic==1.9.1
amqp==5.1.1
apache-superset==2.0.1
apispec==3.3.2
asttokens==2.2.1
async-generator==1.10
async-timeout==4.0.2
attrs==22.2.0
Authlib==1.2.0
Babel==2.11.0
backcall==0.2.0
backoff==2.2.1
billiard==3.6.4.0
bitarray==2.6.1
bleach==3.3.1
boto3==1.26.39
botocore==1.29.39
Brotli==1.0.9
cachelib==0.4.1
celery==5.2.2
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==2.0.12
click==8.1.3
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
colorama==0.4.6
convertdate==2.4.0
cron-descriptor==1.2.32
croniter==1.3.8
cryptography==38.0.2
decorator==5.1.1
defusedxml==0.7.1
deprecation==2.1.0
dnspython==2.2.1
email-validator==1.3.0
exceptiongroup==1.1.0
executing==1.2.0
Flask==2.0.3
Flask-AppBuilder==4.1.6
Flask-Babel==2.0.0
Flask-Caching==1.11.1
Flask-Compress==1.13
Flask-JWT-Extended==4.4.4
Flask-Login==0.6.2
Flask-Migrate==4.0.0
Flask-OpenID==1.3.0
Flask-SQLAlchemy==2.5.1
flask-talisman==1.0.0
Flask-WTF==0.14.3
flower==1.2.0
frozenlist==1.3.3
func-timeout==4.3.5
future==0.18.2
geographiclib==2.0
geopy==2.3.0
gevent==22.10.2
graphlib-backport==1.0.3
greenlet==2.0.1
gunicorn==20.1.0
h11==0.14.0
hashids==1.3.1
hijri-converter==2.2.4
holidays==0.14.2
humanize==4.4.0
idna==3.4
importlib-metadata==5.2.0
importlib-resources==5.10.2
infi.clickhouse-orm==2.1.3
ipdb==0.13.11
ipython==8.7.0
iso8601==1.1.0
isodate==0.6.1
itsdangerous==2.1.2
jedi==0.18.2
Jinja2==3.0.0
jmespath==1.0.1
jsonschema==4.17.3
kombu==5.2.4
korean-lunar-calendar==0.3.1
Mako==1.2.4
Markdown==3.4.1
MarkupSafe==2.1.1
marshmallow==3.19.0
marshmallow-enum==1.5.1
marshmallow-sqlalchemy==0.26.1
matplotlib-inline==0.1.6
msgpack==1.0.4
multidict==6.0.4
mysqlclient==2.1.1
numpy==1.22.1
outcome==1.2.0
packaging==22.0
pandas==1.3.5
parsedatetime==2.6
parso==0.8.3
pexpect==4.8.0
pgsanity==0.2.9
pickleshare==0.7.5
pkgutil_resolve_name==1.3.10
polyline==1.4.0
prison==0.2.1
prometheus-client==0.15.0
prompt-toolkit==3.0.36
psycopg2-binary==2.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
pure-sasl==0.6.2
pyarrow==5.0.0
pycparser==2.21
Pygments==2.13.0
PyJWT==2.6.0
PyMeeus==0.5.12
pyodbc==4.0.35
pyparsing==3.0.9
pyrsistent==0.19.2
PySocks==1.7.1
python-dateutil==2.8.2
python-dotenv==0.21.0
python-geohash==0.8.5
python3-openid==3.2.0
pytz==2022.7
PyYAML==6.0
redis==4.4.0
requests==2.28.1
s3transfer==0.6.0
selenium==4.7.2
simplejson==3.18.1
six==1.16.0
slackclient==2.5.0
sniffio==1.3.0
sortedcontainers==2.4.0
SQLAlchemy==1.3.24
SQLAlchemy-Utils==0.37.9
sqlparse==0.3.0
stack-data==0.6.2
tabulate==0.8.9
tomli==2.0.1
tornado==6.2
traitlets==5.8.0
trino==0.320.0
trio==0.22.0
trio-websocket==0.9.2
typing-extensions==3.10.0.2
urllib3==1.26.13
vertica-python==0.9.1
vine==5.0.0
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==2.0.0
wsproto==1.2.0
WTForms==2.3.3
WTForms-JSON==0.3.5
yarl==1.8.2
zipp==3.11.0
zope.event==4.6
zope.interface==5.5.2
PyHive==0.6.5
thrift==0.16.0
thrift-sasl==0.4.3
clickhouse-connect==0.4.8

[leagueofsoups]

My debug results:
My chart generate POST with body

result_type:	"full"
result_format: "json"

When I have valid cache, my web pod return result from cache https://github.com/apache/superset/blame/master/superset/charts/data/api.py#L322 and response contain "cache_key" https://github.com/apache/superset/blob/master/superset/charts/data/commands/get_data_command.py#L63

When my cache invalid, query run async on celery worker. https://github.com/apache/superset/blame/master/superset/charts/data/api.py#L338
https://github.com/seonguk9303/superset/blob/main/superset/charts/data/commands/create_async_job_command.py#L35

But when I try click force update inside chart page, my result return from redis, is this expected?

[rusackas]

This version of Superset is no longer supported. If anyone is still encountering this in current versions (3.x) please open a new Issue or a PR to address the problem... or we can simply re-open this one :)