Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update frontend handling of graphs #293

Conversation

rileyjbauer
Copy link
Contributor

@rileyjbauer rileyjbauer commented Nov 15, 2018

Updates the StaticGraphParser and tests to work with the newly simplified format of conditional pipelines.

Note: this PR is looking to merge with avolkov/Fixed-compilation-of-dsl.Conditional-2 not master.

Once this PR is merged, the entire branch will be merged all at once (#177) so that the compiler and frontend stay in sync.


This change is Reviewable

@rileyjbauer rileyjbauer requested a review from yebrahim November 15, 2018 22:36
@k8s-ci-robot k8s-ci-robot requested a review from vicaire November 15, 2018 22:36
@rileyjbauer rileyjbauer requested review from Ark-kun and removed request for vicaire November 15, 2018 22:36
@Ark-kun
Copy link
Contributor

Ark-kun commented Nov 15, 2018

Waiting for @yebrahim before merging.

}

graph.setNode(task.name, {
bgColor: task.when ? 'cornsilk' : undefined,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: This should probably be performed using a CSS class.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. This styling is still technically under discussion, so I'll leave it for now, but once it's decided (if we're keeping it), I'll move it to somewhere more appropriate


// Parent here will be the task that pointed to this DAG template
if (parent) {
graph.setEdge(parent, task.name);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this break if the task names are not globally unique?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That depends on what you mean by break. It will not crash, but it may not display what you would expect? The nodes have only names as identifiers, so if there are two separate tasks with the same names, they will be represented by a single node within the graph, combining all of their inbound and outbound edges.

Copy link
Contributor

@Ark-kun Ark-kun Nov 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Task names are only unique inside their dag namespace. It's OK to push this now, but in future, the id of a task node should be the full path root-task/level1-task/level2-task/leaf-task which won't clash with root-task/level1-task/level2-2-task/leaf-task

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can do that in this PR pretty easily but with one caveat: for now, I'll have to assume that tasks can only have siblings as dependencies. Is this a safe assumption to make at this time?

@Ark-kun
Copy link
Contributor

Ark-kun commented Nov 16, 2018

/retest

@Ark-kun
Copy link
Contributor

Ark-kun commented Nov 16, 2018

/lgtm
/approve

@Ark-kun
Copy link
Contributor

Ark-kun commented Nov 16, 2018

Forgot that we've still not heard from @yebrahim
/approve cancel

@Ark-kun
Copy link
Contributor

Ark-kun commented Nov 17, 2018

/lgtm

@Ark-kun
Copy link
Contributor

Ark-kun commented Nov 17, 2018

/approve

1 similar comment
@rileyjbauer
Copy link
Contributor Author

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun, rileyjbauer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

1 similar comment
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun, rileyjbauer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 7e56199 into avolkov/Fixed-compilation-of-dsl.Conditional-2 Nov 17, 2018
k8s-ci-robot pushed a commit that referenced this pull request Nov 19, 2018
…rt done (#177)

* Fixed compilation of dsl.Conditional
The compiler no longer produced intermediate steps.

* Got rid of _create_new_groups

* Changed the sub_group.type check

* Update frontend handling of graphs (#293)

* Updates the frontend to correctly parse the new format of conditional pipelines

* WIP - Assume tasks and templates don't share names

* Greatly simplifies graphing of conditional and non-conditional pipelines

* Adds/updates StaticParser tests

* Give nodes unique names
@IronPan IronPan deleted the update-frontend-handling-of-graphs branch June 28, 2019 18:47
Linchin pushed a commit to Linchin/pipelines that referenced this pull request Apr 11, 2023
* Improve the cleanup of Kubeflow deployments.

* Use the changes in kubeflow/kubeflow#2344; specify command line arguments
  for delete_deployment.sh by name not position
  * Get the zone from the deployment manifest

* kubeflow/kubeflow#2344 doesn't depend on iam_patch.yaml; instead it
  loos for bindings with the specified name.

* Handle no clusters in zone.

* Fix lint.
magdalenakuhn17 pushed a commit to magdalenakuhn17/pipelines that referenced this pull request Oct 22, 2023
* Default cpu/memory limits

* Add resource limit tests

* Use same variable for requests/limits

* Fix limit tests
HumairAK referenced this pull request in red-hat-data-services/data-science-pipelines Mar 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants