You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
We want to implement a lore framework to support all operators.
Describe the solution you'd like
We need to figure out a way to allow user to tell us the operator id at runtime, e.g. we call it lore_id. The lore_id should be determinstic when given same spark configration, spark sql, and input data. Then in the second run we will dump the operators' input data, meta data(e.g. plan information) so that we can replay it in local. Ideally, we will also dump nsight tracing utilizing work here: #10870
Describe alternatives you've considered
No.
Additional context
No.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
We want to implement a lore framework to support all operators.
Describe the solution you'd like
We need to figure out a way to allow user to tell us the operator id at runtime, e.g. we call it
lore_id
. Thelore_id
should be determinstic when given same spark configration, spark sql, and input data. Then in the second run we will dump the operators' input data, meta data(e.g. plan information) so that we can replay it in local. Ideally, we will also dump nsight tracing utilizing work here: #10870Describe alternatives you've considered
No.
Additional context
No.
The text was updated successfully, but these errors were encountered: