Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Package Import to support modifications #84

Closed
DennisClark opened this issue Apr 24, 2024 · 4 comments
Closed

Enhance Package Import to support modifications #84

DennisClark opened this issue Apr 24, 2024 · 4 comments
Assignees
Labels
design needed Design details needed to complete the issue enhancement New feature or request HighPriority High Priority integration Integration with other applications
Milestone

Comments

@DennisClark
Copy link
Member

Problem: Different sources for importing package data to DejaCode vary greatly in quality. If a Package already exists with no license or copyright values, it does not get updated if you use Package Import from a spreadsheet (csv) to import the same package with more comprehensive data.

Benefit: Provide a means to improve Package data when importing.

Solution: In a manner roughly equivalent to the Import SBOM to a Product feature, where there is an option to "Update existing packages with discovered packages data" where "only the empty fields will be updated", make that the default behavior for Package Import. Show the number of modified packages on the results page.

Notes: Spreadsheets have recently resurfaced as an important source for package data.

@DennisClark DennisClark added enhancement New feature or request design needed Design details needed to complete the issue integration Integration with other applications HighPriority High Priority labels Apr 24, 2024
@DennisClark DennisClark added this to the DejaCode 5.1 milestone Apr 24, 2024
tdruez added a commit that referenced this issue Apr 29, 2024
Signed-off-by: tdruez <tdruez@nexb.com>
@DennisClark
Copy link
Member Author

DennisClark commented May 1, 2024

@tdruez Testing of the package import in Staging (both Starship and nexB) went fine and I modified a package successfully and the process properly updated only the empty fields. My only problem encountered had to do with package identity; it seems that you have to provide identical filename, download_url, and purl fields to get an identity hit. If I leave out any of those that already have values in DejaCode the importer tries to add a new package. We can keep that logic but I think the importer page needs to explain it, especially since currently there is nothing specified in "Required columns:" so perhaps we can add a note somewhere prominent on the importer page that says something like this:

Note that a Package is uniquely defined in DejaCode by a combination of filename, download_url, and the six Package URL fields type, namespace, name, version, qualifiers, and subpath. You are not required to provide values in all of these fields (qualifiers and subpath are less commonly used) but if any of them are different from an existing similar package already in DejaCode, then the importer will perform an addition rather than a modification.

@DennisClark
Copy link
Member Author

test file sample attached
package_import_goresym.csv

@tdruez
Copy link
Contributor

tdruez commented May 2, 2024

My only problem encountered had to do with package identity; it seems that you have to provide identical filename, download_url, and purl fields to get an identity hit. If I leave out any of those that already have values in DejaCode the importer tries to add a new package.

I'm having a hard time to understand and reproduce the problem. The .csv only contains 1 entry and I don't have an insight about the content of the Dataspace being used.

tdruez added a commit that referenced this issue May 15, 2024
Signed-off-by: tdruez <tdruez@nexb.com>
tdruez added a commit that referenced this issue May 15, 2024
Signed-off-by: tdruez <tdruez@nexb.com>
tdruez added a commit that referenced this issue May 15, 2024
Signed-off-by: tdruez <tdruez@nexb.com>
@tdruez
Copy link
Contributor

tdruez commented May 15, 2024

@DennisClark I've added the help you provided on the Package import form.
Feature merged and deployed.

@tdruez tdruez closed this as completed May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
design needed Design details needed to complete the issue enhancement New feature or request HighPriority High Priority integration Integration with other applications
Projects
None yet
Development

No branches or pull requests

2 participants