Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(backend): stop compressing artifacts #1640

Closed
wants to merge 4 commits into from
Closed

feat(backend): stop compressing artifacts #1640

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c159ed1
SDK - Stop compressing the artifacts
Ark-kun Jul 18, 2019
738a896
Setting the archive strategy in the backend.
Ark-kun Jul 18, 2019
b83e4cb
Remove compression option in SDK
Ark-kun Jul 19, 2019
8fc7900
Leave the mlmetadata- artifacts compressed for now
Ark-kun Sep 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions backend/src/apiserver/resource/resource_manager.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,8 @@ func (r *ResourceManager) CreateRun(apiRun *api.Run) (*model.RunDetail, error) {
for artIdx, artifact := range template.Outputs.Artifacts {
if artifact.Name == "mlpipeline-ui-metadata" || artifact.Name == "mlpipeline-metrics" {
workflow.Workflow.Spec.Templates[templateIdx].Outputs.Artifacts[artIdx].Optional = true
} else {
artifact.Archive = &workflowapi.ArchiveStrategy{None: &workflowapi.NoneStrategy{}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: why do we add this in an else clause here?
I think we can add it to all.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another issue is that, there should be some similar code in CreateJob method, you should add this part there too

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: why do we add this in an else clause here?

I'm not sure that Frontend and Backend Metrics are prepared for uncompressed artifacts.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Frontend server should support both compressed and uncompressed artifacts.

pipelines/frontend/server/minio-helper.ts

Line 134 in 4e5d724

export async function getObjectStream({

}
}
}
Expand Down