-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create composite indexes [ResourceType, ReferenceUUID, ReferenceType] #1836
Conversation
…ype] It would improve the list runs call which contains filtering on [ResourceType, ReferenceUUID, ReferenceType] We've seen cases list runs take long to run when resource_reference table is large. ``` SELECT subq.*, CONCAT("[", GROUP_CONCAT(r.Payload SEPARATOR ", "), "]") AS refs FROM ( SELECT rd.*, CONCAT("[", GROUP_CONCAT(m.Payload SEPARATOR ", "), "]") AS metrics FROM ( SELECT UUID, DisplayName, Name, StorageState, Namespace, Description, CreatedAtInSec, ScheduledAtInSec, FinishedAtInSec, Conditions, PipelineId, PipelineSpecManifest, WorkflowSpecManifest, Parameters, pipelineRuntimeManifest, WorkflowRuntimeManifest FROM run_details WHERE UUID in ( SELECT ResourceUUID FROM resource_references as rf WHERE ( rf.ResourceType = 'Run' AND rf.ReferenceUUID = '488b0263-f4ee-4398-b7dc-768ffe967372' AND rf.ReferenceType = 'Experiment' ) ) AND StorageState <> 'STORAGESTATE_ARCHIVED' ORDER BY CreatedAtInSec DESC, UUID DESC LIMIT 6 ) AS rd LEFT JOIN run_metrics AS m ON rd.UUID = m.RunUUID GROUP BY rd.UUID ) AS subq LEFT JOIN ( select * from resource_references where ResourceType = 'Run' ) AS r ON subq.UUID = r.ResourceUUID GROUP BY subq.UUID ORDER BY CreatedAtInSec DESC, UUID DESC ``` /assign @hongye-sun
/test kubeflow-pipeline-sample-test |
/lgtm |
/test kubeflow-pipeline-sample-test |
1 similar comment
/test kubeflow-pipeline-sample-test |
/approve |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: IronPan The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
I don't believe this addresses the issue. I've opened #2071 |
* adding tgz/zip support for all storages * Fix lint Co-authored-by: Dan Sun <dsun20@bloomberg.net>
It would improve the list runs call which contains filtering on [ResourceType, ReferenceUUID, ReferenceType]
We've seen cases list runs take long to run when resource_reference table is large.
/assign @hongye-sun
This change is