diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 4e2089f8..4634d4a2 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -5,41 +5,41 @@ Release notes - Add visual indicator in hierarchy views, when an object on the far left or far right also belong or have a hierarchy (relationship tree). - https://github.com/nexB/dejacode/issues/70 + https://github.com/aboutcode-org/dejacode/issues/70 - Add search and pagination on the Product Inventory tab. - https://github.com/nexB/dejacode/issues/3 - https://github.com/nexB/dejacode/issues/112 + https://github.com/aboutcode-org/dejacode/issues/3 + https://github.com/aboutcode-org/dejacode/issues/112 - Fix an issue displaying the "Delete" button in the "Edit Product Relationship" modal form. - https://github.com/nexB/dejacode/issues/128 + https://github.com/aboutcode-org/dejacode/issues/128 - Add support for PURL(s) in the "Add Package" modal. If the PURL type is supported by the packageurl_python library, a download URL will be generated for creating the package and submitting a scan. - https://github.com/nexB/dejacode/issues/131 + https://github.com/aboutcode-org/dejacode/issues/131 - Leverage PurlDB during the "Add Package" process. DejaCode will look up the PurlDB to retrieve and fetch all available data to create the package. - https://github.com/nexB/dejacode/issues/131 + https://github.com/aboutcode-org/dejacode/issues/131 - Populate the Package notice_text using "*NOTICE*" file content from Scan "key files". - https://github.com/nexB/dejacode/issues/136 + https://github.com/aboutcode-org/dejacode/issues/136 - Added 2 new license related fields on the Component and Package models: * declared_license_expression * other_license_expression - https://github.com/nexB/dejacode/issues/63 + https://github.com/aboutcode-org/dejacode/issues/63 - Added 2 properties on the Component and Package models: * declared_license_expression_spdx (computed from declared_license_expression) * other_license_expression_spdx (computed from other_license_expression) - https://github.com/nexB/dejacode/issues/63 + https://github.com/aboutcode-org/dejacode/issues/63 - Removed 2 fields: Package.declared_license and Component.concluded_license - https://github.com/nexB/dejacode/issues/63 + https://github.com/aboutcode-org/dejacode/issues/63 - The new license fields are automatically populated from the Package scan "Update packages automatically from scan". @@ -49,103 +49,138 @@ Release notes "Add Component from Package data". The license expression values provided in the form for the new field is now properly checked and return a validation error when incorrect. - https://github.com/nexB/dejacode/issues/63 + https://github.com/aboutcode-org/dejacode/issues/63 - Use the declared_license_expression_spdx value in SPDX outputs. - https://github.com/nexB/dejacode/issues/63 + https://github.com/aboutcode-org/dejacode/issues/63 - Add new ProductDependency model to support relating Packages in the context of a Product. - https://github.com/nexB/dejacode/issues/138 + https://github.com/aboutcode-org/dejacode/issues/138 + +- Add a task scheduler service to the Docker Compose stack. + This service runs a dedicated ``setupcron`` management command to create the + application's scheduled cron jobs. + The scheduler is configured to run the daily vulnerabilities update task. + https://github.com/aboutcode-org/dejacode/issues/94 + +- Add a new Vulnerability model and all the code logic to fetch and create + Vulnerability records and assign those to Package/Component through ManyToMany + relationships. + A fetchvulnerabilities management command is available to fetch all the relevant + data from VulnerableCode for a given Dataspace. + The latest vulnerability data refresh date is displayed in the Admin dashboard in a + new "Data updates" section in the bottom right corner. + It is also available in the "Integration Status" page. + The Package/Component views that display vulnerability information (icon or tab) + are now using the data from the Vulnerability model in place of calling the + VulnerableCode API on each request. This results into much better performances as + we do not depend on the VulnerableCode service to render the DejaCode view anymore. + Also, this will make Vulnerability data available in the Reporting system. + The vulnerability icon is displayed next to the Package/Component identifier in the + Product views: "Inventory", "Hierarchy", "Dependencies" tabs. + The vulnerability data is available in Reporting either through the is_vulnerable + property on Package/Component column template or going through the full + affected_by_vulnerabilities m2m field. + This is available in both Query and ColumnTemplate. + The vulnerabilities are fetched each time a Package is created/modified + (note that a purl is required on the package for the lookup). + Also, all the Packages of a Product are updated with latest vulnerabilities from + the VulnerableCode service following importing data in Product using: + - Import data from Scan + - Load Packages from SBOMs + - Import Packages from manifests + - Pull ScanCode.io Project data + https://github.com/aboutcode-org/dejacode/issues/94 ### Version 5.1.0 - Upgrade Python version to 3.12 and Django to 5.0.x - https://github.com/nexB/dejacode/issues/50 + https://github.com/aboutcode-org/dejacode/issues/50 - Replace Celery by RQ for async job queue and worker. - https://github.com/nexB/dejacode/issues/6 + https://github.com/aboutcode-org/dejacode/issues/6 - Add support for CycloneDX spec version "1.6". In the UI and API, older spe version such as "1.4" and "1.5" are also available as download. - https://github.com/nexB/dejacode/pull/79 + https://github.com/aboutcode-org/dejacode/pull/79 - Lookup in PurlDB by purl in Add Package form. When a Package URL is available in the context of the "Add Package" form, for example when using a link from the Vulnerabilities tab, data is fetched from the PurlDB to initialize the form. - https://github.com/nexB/dejacode/issues/47 + https://github.com/aboutcode-org/dejacode/issues/47 - If you select two versions of the same Product in the Product list, or two different Products, and click the Compare button, you can now download the results of the comparison to a .xlsx file, making it easy to share the information with your colleagues. - https://github.com/nexB/dejacode/issues/7 + https://github.com/aboutcode-org/dejacode/issues/7 - Add dark theme support in UI. - https://github.com/nexB/dejacode/issues/25 + https://github.com/aboutcode-org/dejacode/issues/25 - Add "Load Packages from SBOMs", "Import scan results", and "Pull ScanCode.io project data" feature as Product action in the REST API. - https://github.com/nexB/dejacode/issues/59 + https://github.com/aboutcode-org/dejacode/issues/59 - Add REST API endpoints to download SBOMs as CycloneDX and SPDX. - https://github.com/nexB/dejacode/issues/60 + https://github.com/aboutcode-org/dejacode/issues/60 - Refactor the "Import manifest" feature as "Load SBOMs". - https://github.com/nexB/dejacode/issues/61 + https://github.com/aboutcode-org/dejacode/issues/61 - Add support to import packages from manifest. - https://github.com/nexB/dejacode/issues/65 + https://github.com/aboutcode-org/dejacode/issues/65 - Add a vulnerability link to the VulnerableCode app in the Vulnerability tab. - https://github.com/nexB/dejacode/issues/4 + https://github.com/aboutcode-org/dejacode/issues/4 - Add a DEJACODE_SUPPORT_EMAIL setting for support email address customization. - https://github.com/nexB/dejacode/issues/76 + https://github.com/aboutcode-org/dejacode/issues/76 - Show the individual PURL fields in the Package details view. - https://github.com/nexB/dejacode/issues/83 + https://github.com/aboutcode-org/dejacode/issues/83 - Fix the logout link of the admin app. - https://github.com/nexB/dejacode/issues/89 + https://github.com/aboutcode-org/dejacode/issues/89 - Display full commit in the version displayed in the UI - https://github.com/nexB/dejacode/issues/88 + https://github.com/aboutcode-org/dejacode/issues/88 - Refine the Product comparison logic for Packages. The type and namespace fields are now used along the name field to match similar Packages (excluding the version). - https://github.com/nexB/dejacode/issues/113 + https://github.com/aboutcode-org/dejacode/issues/113 - Refactor the implementation of Keywords on forms to allow more flexibilty. Existing Keywords are suggested for consistency but any values is now allowed. - https://github.com/nexB/dejacode/issues/48 + https://github.com/aboutcode-org/dejacode/issues/48 - Display Product inventory count on the Product list view. - https://github.com/nexB/dejacode/issues/81 + https://github.com/aboutcode-org/dejacode/issues/81 - Always display the full Package URL in the UI view including the "pkg:" prefix. - https://github.com/nexB/dejacode/issues/115 + https://github.com/aboutcode-org/dejacode/issues/115 - Add a new AboutCode tab in Package details view. - https://github.com/nexB/dejacode/issues/42 + https://github.com/aboutcode-org/dejacode/issues/42 - Enhance Package Import to support modifications. - https://github.com/nexB/dejacode/issues/84 + https://github.com/aboutcode-org/dejacode/issues/84 - Add an option on the "Add to Product" form to to replace any existing relationships with a different version of the same object by the selected object. - https://github.com/nexB/dejacode/issues/12 + https://github.com/aboutcode-org/dejacode/issues/12 ### Version 5.0.1 - Improve the stability of the "Check for new Package versions" feature. - https://github.com/nexB/dejacode/issues/17 + https://github.com/aboutcode-org/dejacode/issues/17 - Improve the support for SourgeForge download URLs. - https://github.com/nexB/dejacode/issues/26 + https://github.com/aboutcode-org/dejacode/issues/26 ### Version 5.0.0 diff --git a/Dockerfile b/Dockerfile index 711813e3..dae0b7fa 100644 --- a/Dockerfile +++ b/Dockerfile @@ -2,13 +2,13 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # FROM python:3.12-slim -LABEL org.opencontainers.image.source="https://github.com/nexB/dejacode" +LABEL org.opencontainers.image.source="https://github.com/aboutcode-org/dejacode" LABEL org.opencontainers.image.description="DejaCode" LABEL org.opencontainers.image.licenses="AGPL-3.0-only" diff --git a/Makefile b/Makefile index a668e43e..8479e6bb 100644 --- a/Makefile +++ b/Makefile @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/README.rst b/README.rst index 53c36d82..6984a1d5 100644 --- a/README.rst +++ b/README.rst @@ -30,7 +30,7 @@ Getting started The DejaCode documentation is available here: https://dejacode.readthedocs.io/ If you have questions please ask them in -`Discussions `_. +`Discussions `_. If you want to contribute to DejaCode, start with our `Contributing `_ page. @@ -75,8 +75,8 @@ You can learn more about this option by contacting us at https://www.nexb.com/contact-us/ -.. |ci-tests| image:: https://github.com/nexB/dejacode/actions/workflows/ci.yml/badge.svg?branch=main - :target: https://github.com/nexB/dejacode/actions/workflows/ci.yml +.. |ci-tests| image:: https://github.com/aboutcode-org/dejacode/actions/workflows/ci.yml/badge.svg?branch=main + :target: https://github.com/aboutcode-org/dejacode/actions/workflows/ci.yml :alt: CI Tests Status .. |docs-rtd| image:: https://readthedocs.org/projects/dejacode/badge/?version=latest diff --git a/component_catalog/__init__.py b/component_catalog/__init__.py index 7aa35de2..f61c4aae 100644 --- a/component_catalog/__init__.py +++ b/component_catalog/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/admin.py b/component_catalog/admin.py index 4b2751a4..3b25b50b 100644 --- a/component_catalog/admin.py +++ b/component_catalog/admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/api.py b/component_catalog/api.py index f19f71a7..e39d56e0 100644 --- a/component_catalog/api.py +++ b/component_catalog/api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/apps.py b/component_catalog/apps.py index 5c1f2e8e..2c69653e 100644 --- a/component_catalog/apps.py +++ b/component_catalog/apps.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/filters.py b/component_catalog/filters.py index bdec1f33..5efdefa5 100644 --- a/component_catalog/filters.py +++ b/component_catalog/filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -28,6 +28,21 @@ from license_library.models import License +class IsVulnerableFilter(HasRelationFilter): + def __init__(self, *args, **kwargs): + kwargs["lookup_expr"] = "isnull" + kwargs["empty_label"] = "Any" + kwargs.setdefault("label", _("Is Vulnerable")) + kwargs.setdefault( + "choices", + ( + ("yes", _("Affected by vulnerabilities")), + ("no", _("No vulnerabilities found")), + ), + ) + super().__init__(*args, **kwargs) + + class ComponentFilterSet(DataspacedFilterSet): related_only = [ "licenses", @@ -85,6 +100,10 @@ class ComponentFilterSet(DataspacedFilterSet): search_placeholder="Search keywords", ), ) + is_vulnerable = IsVulnerableFilter( + field_name="affected_by_vulnerabilities", + widget=DropDownRightWidget(link_content=''), + ) class Meta: model = Component @@ -219,6 +238,10 @@ class PackageFilterSet(DataspacedFilterSet): empty_label="Last modified (default)", widget=SortDropDownWidget, ) + is_vulnerable = IsVulnerableFilter( + field_name="affected_by_vulnerabilities", + widget=DropDownRightWidget(link_content=''), + ) class Meta: model = Package diff --git a/component_catalog/forms.py b/component_catalog/forms.py index d8952750..9c6a0de6 100644 --- a/component_catalog/forms.py +++ b/component_catalog/forms.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -446,6 +446,9 @@ def save(self, *args, **kwargs): ) self.cleaned_data["scan_submitted"] = True + if self.user.dataspace.enable_vulnerablecodedb_access: + instance.fetch_vulnerabilities() + return instance @@ -1043,6 +1046,11 @@ def save(self, commit=True): self._set_purldb_uuid_on_instance() return super().save(commit) + def _save_m2m(self): + super()._save_m2m() + if self.dataspace.enable_vulnerablecodedb_access: + self.instance.fetch_vulnerabilities() + class ComponentMassUpdateForm( LicenseExpressionFormMixin, diff --git a/component_catalog/fuzzy.py b/component_catalog/fuzzy.py index 33e07ac5..ecf833ac 100644 --- a/component_catalog/fuzzy.py +++ b/component_catalog/fuzzy.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/importers.py b/component_catalog/importers.py index f60467e6..760f9d55 100644 --- a/component_catalog/importers.py +++ b/component_catalog/importers.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -30,6 +30,7 @@ from component_catalog.models import Package from component_catalog.models import Subcomponent from component_catalog.programming_languages import PROGRAMMING_LANGUAGES +from component_catalog.vulnerabilities import fetch_for_queryset from dje.fields import SmartFileField from dje.forms import JSONListField from dje.importers import BaseImporter @@ -182,6 +183,7 @@ class Meta: "packages", "completion_level", "request_count", + "affected_by_vulnerabilities", # JSONField not supported "dependencies", ) @@ -261,6 +263,7 @@ class Meta: "file_references", "request_count", "parties", + "affected_by_vulnerabilities", ] def __init__(self, *args, **kwargs): @@ -343,7 +346,7 @@ def prepare_data_json(self, data): input_as_list_of_dict = packages # Using a dict comprehension to keep the original key order and - # ensure we have all possibile headers. + # ensure we have all possible headers. header_row = {key: None for package in packages for key in package.keys()} header_row = list(header_row.keys()) @@ -423,6 +426,15 @@ def get_form_kwargs(self): kwargs["is_from_scancode"] = getattr(self, "is_from_scancode", False) return kwargs + def save_all(self): + """Fetch vulnerabilities for imported Packages.""" + super().save_all() + + if self.dataspace.enable_vulnerablecodedb_access: + package_pks = [package.pk for package in self.results["added"]] + package_qs = Package.objects.scope(dataspace=self.dataspace).filter(pk__in=package_pks) + fetch_for_queryset(package_qs, self.dataspace) + class SubcomponentImportForm( ComponentRelatedFieldImportMixin, diff --git a/component_catalog/inlines.py b/component_catalog/inlines.py index dbd5a9d6..d23b23de 100644 --- a/component_catalog/inlines.py +++ b/component_catalog/inlines.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/license_expression_dje.py b/component_catalog/license_expression_dje.py index 2ec2f80e..b2f0e58e 100644 --- a/component_catalog/license_expression_dje.py +++ b/component_catalog/license_expression_dje.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/management/commands/collectcpes.py b/component_catalog/management/commands/collectcpes.py index cfd2b412..dc6955fe 100644 --- a/component_catalog/management/commands/collectcpes.py +++ b/component_catalog/management/commands/collectcpes.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/management/commands/collectpackagesdata.py b/component_catalog/management/commands/collectpackagesdata.py index f3f9438e..142f06a4 100644 --- a/component_catalog/management/commands/collectpackagesdata.py +++ b/component_catalog/management/commands/collectpackagesdata.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/management/commands/componentfrompackage.py b/component_catalog/management/commands/componentfrompackage.py index 5d2ab02d..101ce07a 100644 --- a/component_catalog/management/commands/componentfrompackage.py +++ b/component_catalog/management/commands/componentfrompackage.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/management/commands/fetchvulnerabilities.py b/component_catalog/management/commands/fetchvulnerabilities.py new file mode 100644 index 00000000..55fce0c1 --- /dev/null +++ b/component_catalog/management/commands/fetchvulnerabilities.py @@ -0,0 +1,46 @@ +# +# Copyright (c) nexB Inc. and others. All rights reserved. +# DejaCode is a trademark of nexB Inc. +# SPDX-License-Identifier: AGPL-3.0-only +# See https://github.com/aboutcode-org/dejacode for support or download. +# See https://aboutcode.org for more information about AboutCode FOSS projects. +# + +from django.core.management.base import CommandError + +from component_catalog.vulnerabilities import fetch_from_vulnerablecode +from dejacode_toolkit.vulnerablecode import VulnerableCode +from dje.management.commands import DataspacedCommand + + +class Command(DataspacedCommand): + help = "Fetch vulnerabilities for the provided Dataspace" + + def add_arguments(self, parser): + super().add_arguments(parser) + parser.add_argument( + "--batch-size", + type=int, + default=50, + help="Specifies the number of objects per requests to the VulnerableCode service", + ) + parser.add_argument( + "--timeout", + type=int, + default=30, + help="Request timeout in seconds", + ) + + def handle(self, *args, **options): + super().handle(*args, **options) + batch_size = options["batch_size"] + timeout = options["timeout"] + + if not self.dataspace.enable_vulnerablecodedb_access: + raise CommandError("VulnerableCode is not enabled on this Dataspace.") + + vulnerablecode = VulnerableCode(self.dataspace) + if not vulnerablecode.is_configured(): + raise CommandError("VulnerableCode is not configured.") + + fetch_from_vulnerablecode(self.dataspace, batch_size, timeout, log_func=self.stdout.write) diff --git a/component_catalog/management/commands/setpurls.py b/component_catalog/management/commands/setpurls.py index e5d5b357..e7067f76 100644 --- a/component_catalog/management/commands/setpurls.py +++ b/component_catalog/management/commands/setpurls.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/management/commands/updatecompletionlevels.py b/component_catalog/management/commands/updatecompletionlevels.py index 10a1a7ef..027cb760 100644 --- a/component_catalog/management/commands/updatecompletionlevels.py +++ b/component_catalog/management/commands/updatecompletionlevels.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/migrations/0006_vulnerability_model_and_missing_indexes.py b/component_catalog/migrations/0006_vulnerability_model_and_missing_indexes.py new file mode 100644 index 00000000..6b4b9130 --- /dev/null +++ b/component_catalog/migrations/0006_vulnerability_model_and_missing_indexes.py @@ -0,0 +1,129 @@ +# Generated by Django 5.0.6 on 2024-08-21 11:20 + +import component_catalog.models +import django.db.models.deletion +import dje.fields +import uuid +from django.conf import settings +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('component_catalog', '0005_remove_component_concluded_license_and_more'), + ('dje', '0004_dataspace_vulnerabilities_updated_at'), + ('license_library', '0002_initial'), + ('policy', '0002_initial'), + migrations.swappable_dependency(settings.AUTH_USER_MODEL), + ] + + operations = [ + migrations.AlterField( + model_name='package', + name='filename', + field=models.CharField(blank=True, help_text='The exact file name (typically an archive of some type) of the package. This is usually the name of the file as downloaded from a website.', max_length=255, validators=[component_catalog.models.validate_filename], verbose_name='Filename'), + ), + migrations.AlterField( + model_name='package', + name='license_expression', + field=models.CharField(blank=True, help_text='The License Expression assigned to a DejaCode Package or Component is an editable value equivalent to a "concluded license" as determined by a curator who has performed analysis to clarify or correct the declared license expression, which may have been assigned automatically (from a scan or an associated package definition) when the Package or Component was originally created. A license expression defines the relationship of one or more licenses to a software object. More than one applicable license can be expressed as "license-key-a AND license-key-b". A choice of applicable licenses can be expressed as "license-key-a OR license-key-b", and you can indicate the primary (preferred) license by placing it first, on the left-hand side of the OR relationship. The relationship words (OR, AND) can be combined as needed, and the use of parentheses can be applied to clarify the meaning; for example "((license-key-a AND license-key-b) OR (license-key-c))". An exception to a license can be expressed as "license-key WITH license-exception-key".', max_length=1024, verbose_name='Concluded license expression'), + ), + migrations.AlterField( + model_name='package', + name='md5', + field=models.CharField(blank=True, help_text='MD5 checksum hex-encoded, as in md5sum.', max_length=32, verbose_name='MD5'), + ), + migrations.AlterField( + model_name='package', + name='primary_language', + field=models.CharField(blank=True, help_text='The primary programming language associated with the package.', max_length=50), + ), + migrations.AlterField( + model_name='package', + name='project', + field=models.CharField(blank=True, help_text='Project is a free-form label that you can use to group and find packages and components that interest you; for example, you may be starting a new development project, evaluating them for use in a product or you may want to get approval to use them.', max_length=50), + ), + migrations.AlterField( + model_name='package', + name='sha1', + field=models.CharField(blank=True, help_text='SHA1 checksum hex-encoded, as in sha1sum.', max_length=40, verbose_name='SHA1'), + ), + migrations.AlterField( + model_name='package', + name='size', + field=models.BigIntegerField(blank=True, help_text='The size of the package file in bytes.', null=True), + ), + migrations.CreateModel( + name='Vulnerability', + fields=[ + ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), + ('uuid', models.UUIDField(default=uuid.uuid4, editable=False, verbose_name='UUID')), + ('created_date', models.DateTimeField(auto_now_add=True, db_index=True, help_text='The date and time the object was created.')), + ('last_modified_date', models.DateTimeField(auto_now=True, db_index=True, help_text='The date and time the object was last modified.')), + ('vulnerability_id', models.CharField(help_text="A unique identifier for the vulnerability, prefixed with 'VCID-'. For example, 'VCID-2024-0001'.", max_length=20)), + ('summary', models.TextField(blank=True, help_text='A brief summary of the vulnerability, outlining its nature and impact.')), + ('aliases', dje.fields.JSONListField(blank=True, default=list, help_text="A list of aliases for this vulnerability, such as CVE identifiers (e.g., 'CVE-2017-1000136').")), + ('references', dje.fields.JSONListField(blank=True, default=list, help_text='A list of references for this vulnerability. Each reference includes a URL, an optional reference ID, scores, and the URL for further details. ')), + ('fixed_packages', dje.fields.JSONListField(blank=True, default=list, help_text='A list of packages that are not affected by this vulnerability.')), + ('dataspace', models.ForeignKey(editable=False, help_text='A Dataspace is an independent, exclusive set of DejaCode data, which can be either nexB master reference data or installation-specific data.', on_delete=django.db.models.deletion.PROTECT, to='dje.dataspace')), + ], + options={ + 'verbose_name_plural': 'Vulnerabilities', + }, + ), + migrations.AddField( + model_name='component', + name='affected_by_vulnerabilities', + field=models.ManyToManyField(help_text='Vulnerabilities affecting this object.', related_name='affected_%(class)ss', to='component_catalog.vulnerability'), + ), + migrations.AddField( + model_name='package', + name='affected_by_vulnerabilities', + field=models.ManyToManyField(help_text='Vulnerabilities affecting this object.', related_name='affected_%(class)ss', to='component_catalog.vulnerability'), + ), + migrations.AddIndex( + model_name='package', + index=models.Index(fields=['type'], name='component_c_type_4af44a_idx'), + ), + migrations.AddIndex( + model_name='package', + index=models.Index(fields=['namespace'], name='component_c_namespa_d10c6b_idx'), + ), + migrations.AddIndex( + model_name='package', + index=models.Index(fields=['name'], name='component_c_name_c82010_idx'), + ), + migrations.AddIndex( + model_name='package', + index=models.Index(fields=['version'], name='component_c_version_ad217d_idx'), + ), + migrations.AddIndex( + model_name='package', + index=models.Index(fields=['filename'], name='component_c_filenam_b60780_idx'), + ), + migrations.AddIndex( + model_name='package', + index=models.Index(fields=['size'], name='component_c_size_22e128_idx'), + ), + migrations.AddIndex( + model_name='package', + index=models.Index(fields=['primary_language'], name='component_c_primary_2e8211_idx'), + ), + migrations.AddIndex( + model_name='package', + index=models.Index(fields=['project'], name='component_c_project_74b1c0_idx'), + ), + migrations.AddIndex( + model_name='package', + index=models.Index(fields=['license_expression'], name='component_c_license_20719b_idx'), + ), + migrations.AddIndex( + model_name='vulnerability', + index=models.Index(fields=['vulnerability_id'], name='component_c_vulnera_e18d50_idx'), + ), + migrations.AlterUniqueTogether( + name='vulnerability', + unique_together={('dataspace', 'uuid'), ('dataspace', 'vulnerability_id')}, + ), + ] diff --git a/component_catalog/models.py b/component_catalog/models.py index 8c1c516d..95ebeca7 100644 --- a/component_catalog/models.py +++ b/component_catalog/models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -64,6 +64,7 @@ from dje.models import DataspacedQuerySet from dje.models import ExternalReferenceMixin from dje.models import History +from dje.models import HistoryDateFieldsMixin from dje.models import HistoryFieldsMixin from dje.models import ParentChildModelMixin from dje.models import ParentChildRelationshipModel @@ -129,6 +130,18 @@ def validate_filename(value): ) +class VulnerabilityQuerySetMixin: + def with_vulnerability_count(self): + """Annotate the QuerySet with the vulnerability_count.""" + return self.annotate( + vulnerability_count=models.Count("affected_by_vulnerabilities", distinct=True) + ) + + def vulnerable(self): + """Return vulnerable Packages.""" + return self.with_vulnerability_count().filter(vulnerability_count__gt=0) + + class LicenseExpressionMixin: """Model mixin for models that store license expressions.""" @@ -450,6 +463,85 @@ def get_spdx_cpe_external_ref(self): ) +class VulnerabilityMixin(models.Model): + """Add the `vulnerability` many to many field.""" + + affected_by_vulnerabilities = models.ManyToManyField( + to="component_catalog.Vulnerability", + related_name="affected_%(class)ss", + help_text=_("Vulnerabilities affecting this object."), + ) + + class Meta: + abstract = True + + @property + def is_vulnerable(self): + return self.affected_by_vulnerabilities.exists() + + def get_entry_for_package(self, vulnerablecode): + if not self.package_url: + return + + vulnerable_packages = vulnerablecode.get_vulnerabilities_by_purl( + self.package_url, + timeout=10, + ) + + if vulnerable_packages: + affected_by_vulnerabilities = vulnerable_packages[0].get("affected_by_vulnerabilities") + return affected_by_vulnerabilities + + def get_entry_for_component(self, vulnerablecode): + if not self.cpe: + return + + # Support for Component is paused as the CPES endpoint do not work properly. + # https://github.com/aboutcode-org/vulnerablecode/issues/1557 + # vulnerabilities = vulnerablecode.get_vulnerabilities_by_cpe(self.cpe, timeout=10) + + def get_entry_from_vulnerablecode(self): + from dejacode_toolkit.vulnerablecode import VulnerableCode + + dataspace = self.dataspace + vulnerablecode = VulnerableCode(dataspace) + + is_vulnerablecode_enabled = all( + [ + vulnerablecode.is_configured(), + dataspace.enable_vulnerablecodedb_access, + ] + ) + if not is_vulnerablecode_enabled: + return + + if isinstance(self, Component): + return self.get_entry_for_component(vulnerablecode) + elif isinstance(self, Package): + return self.get_entry_for_package(vulnerablecode) + + def fetch_vulnerabilities(self): + affected_by_vulnerabilities = self.get_entry_from_vulnerablecode() + if affected_by_vulnerabilities: + self.create_vulnerabilities(vulnerabilities_data=affected_by_vulnerabilities) + + def create_vulnerabilities(self, vulnerabilities_data): + vulnerabilities = [] + vulnerability_qs = Vulnerability.objects.scope(self.dataspace) + + for vulnerability_data in vulnerabilities_data: + vulnerability_id = vulnerability_data["vulnerability_id"] + vulnerability = vulnerability_qs.get_or_none(vulnerability_id=vulnerability_id) + if not vulnerability: + vulnerability = Vulnerability.create_from_data( + dataspace=self.dataspace, + data=vulnerability_data, + ) + vulnerabilities.append(vulnerability) + + self.affected_by_vulnerabilities.add(*vulnerabilities) + + class URLFieldsMixin(models.Model): homepage_url = models.URLField( _("Homepage URL"), @@ -852,7 +944,7 @@ def as_cyclonedx(self, license_expression_spdx=None): BaseComponentMixin = component_mixin_factory("component") -class ComponentQuerySet(DataspacedQuerySet): +class ComponentQuerySet(VulnerabilityQuerySetMixin, DataspacedQuerySet): def with_has_hierarchy(self): subcomponents = Subcomponent.objects.filter( models.Q(child_id=OuterRef("pk")) | models.Q(parent_id=OuterRef("pk")) @@ -875,6 +967,7 @@ class Component( HolderMixin, KeywordsMixin, CPEMixin, + VulnerabilityMixin, LicenseFieldsMixin, ParentChildModelMixin, BaseComponentMixin, @@ -1622,7 +1715,11 @@ def __str__(self): PACKAGE_URL_FIELDS = ["type", "namespace", "name", "version", "qualifiers", "subpath"] -class PackageQuerySet(PackageURLQuerySetMixin, DataspacedQuerySet): +class PackageQuerySet(PackageURLQuerySetMixin, VulnerabilityQuerySetMixin, DataspacedQuerySet): + def has_package_url(self): + """Return objects with Package URL defined.""" + return self.filter(~models.Q(type="") & ~models.Q(name="")) + def annotate_sortable_identifier(self): """ Annotate the QuerySet with a `sortable_identifier` value that combines @@ -1670,6 +1767,7 @@ class Package( HolderMixin, KeywordsMixin, CPEMixin, + VulnerabilityMixin, URLFieldsMixin, HashFieldsMixin, PackageURLMixin, @@ -1678,7 +1776,6 @@ class Package( filename = models.CharField( _("Filename"), blank=True, - db_index=True, max_length=255, # 255 is the maximum on most filesystems validators=[validate_filename], help_text=_( @@ -1686,7 +1783,6 @@ class Package( "This is usually the name of the file as downloaded from a website." ), ) - download_url = models.CharField( _("Download URL"), max_length=1024, @@ -1694,30 +1790,11 @@ class Package( blank=True, help_text=_("The download URL for obtaining the package."), ) - - sha1 = models.CharField( - _("SHA1"), - max_length=40, - blank=True, - db_index=True, - help_text=_("The SHA1 signature of the package file."), - ) - - md5 = models.CharField( - _("MD5"), - max_length=32, - blank=True, - db_index=True, - help_text=_("The MD5 signature of the package file."), - ) - size = models.BigIntegerField( blank=True, null=True, - db_index=True, help_text=_("The size of the package file in bytes."), ) - release_date = models.DateField( blank=True, null=True, @@ -1726,39 +1803,30 @@ class Package( "original download source." ), ) - primary_language = models.CharField( - db_index=True, max_length=50, blank=True, help_text=_("The primary programming language associated with the package."), ) - description = models.TextField( blank=True, help_text=_("Free form description, preferably as provided by the author(s)."), ) - project = models.CharField( max_length=50, - db_index=True, blank=True, help_text=PROJECT_FIELD_HELP, ) - notes = models.TextField( blank=True, help_text=_("Descriptive information about the package."), ) - license_expression = models.CharField( _("Concluded license expression"), max_length=1024, blank=True, - db_index=True, help_text=LICENSE_EXPRESSION_HELP_TEXT, ) - copyright = models.TextField( blank=True, help_text=_( @@ -1766,7 +1834,6 @@ class Package( "source or as specified in an associated file." ), ) - notice_text = NoStripTextField( blank=True, help_text=_( @@ -1774,14 +1841,12 @@ class Package( "statement(s), contributors, and/or license obligations that apply to a package." ), ) - author = models.TextField( blank=True, help_text=_( "The name(s) of the author(s) of a software package as documented in the code." ), ) - dependencies = models.JSONField( blank=True, default=list, @@ -1791,7 +1856,6 @@ class Package( "on licensing and/or attribution obligations." ), ) - repository_homepage_url = models.URLField( _("Repository homepage URL"), max_length=1024, @@ -1801,7 +1865,6 @@ class Package( "This is typically different from the package homepage URL proper." ), ) - repository_download_url = models.URLField( _("Repository download URL"), max_length=1024, @@ -1812,7 +1875,6 @@ class Package( "This may be different from the actual download URL." ), ) - api_data_url = models.URLField( _("API data URL"), max_length=1024, @@ -1822,13 +1884,11 @@ class Package( "URL to a JSON or XML api its package repository." ), ) - datasource_id = models.CharField( max_length=64, blank=True, help_text=_("The identifier for the datafile handler used to obtain this package."), ) - file_references = models.JSONField( default=list, blank=True, @@ -1839,13 +1899,11 @@ class Package( "package type or datafile format." ), ) - parties = models.JSONField( default=list, blank=True, help_text=_("A list of parties such as a person, project or organization."), ) - licenses = models.ManyToManyField( to="license_library.License", through="PackageAssignedLicense", @@ -1876,11 +1934,21 @@ class Meta: ), ) indexes = [ + models.Index(fields=["type"]), + models.Index(fields=["namespace"]), + models.Index(fields=["name"]), + models.Index(fields=["version"]), + models.Index(fields=["filename"]), + models.Index(fields=["size"]), models.Index(fields=["md5"]), models.Index(fields=["sha1"]), models.Index(fields=["sha256"]), models.Index(fields=["sha512"]), + models.Index(fields=["primary_language"]), + models.Index(fields=["project"]), + models.Index(fields=["license_expression"]), ] + permissions = ( ("change_usage_policy_on_package", "Can change the usage_policy of package"), ) @@ -2510,3 +2578,91 @@ class Meta: def __str__(self): return f"<{self.component}>: {self.package}" + + +class Vulnerability(HistoryDateFieldsMixin, DataspacedModel): + """ + A software vulnerability with a unique identifier and alternate aliases. + + Adapted from the VulnerabeCode models at + https://github.com/nexB/vulnerablecode/blob/main/vulnerabilities/models.py#L164 + + Note that this model implements the HistoryDateFieldsMixin but not the + HistoryUserFieldsMixin as the Vulnerability records are usually created + automatically on object addition or during schedule tasks. + """ + + vulnerability_id = models.CharField( + max_length=20, + help_text=_( + "A unique identifier for the vulnerability, prefixed with 'VCID-'. " + "For example, 'VCID-2024-0001'." + ), + ) + summary = models.TextField( + help_text=_("A brief summary of the vulnerability, outlining its nature and impact."), + blank=True, + ) + aliases = JSONListField( + blank=True, + help_text=_( + "A list of aliases for this vulnerability, such as CVE identifiers " + "(e.g., 'CVE-2017-1000136')." + ), + ) + references = JSONListField( + blank=True, + help_text=_( + "A list of references for this vulnerability. Each reference includes a " + "URL, an optional reference ID, scores, and the URL for further details. " + ), + ) + fixed_packages = JSONListField( + blank=True, + help_text=_("A list of packages that are not affected by this vulnerability."), + ) + + class Meta: + verbose_name_plural = "Vulnerabilities" + unique_together = (("dataspace", "vulnerability_id"), ("dataspace", "uuid")) + indexes = [ + models.Index(fields=["vulnerability_id"]), + ] + + def __str__(self): + return self.vulnerability_id + + @property + def vcid(self): + return self.vulnerability_id + + def add_affected(self, instances): + """ + Assign the ``instances`` (Package or Component) as affected to this + vulnerability. + """ + if not isinstance(instances, list): + instances = [instances] + + for instance in instances: + if isinstance(instance, Package): + self.affected_packages.add(instance) + if isinstance(instance, Component): + self.affected_components.add(instance) + + def add_affected_packages(self, packages): + """Assign the ``packages`` as affected to this vulnerability.""" + self.affected_packages.add(*packages) + + def add_affected_components(self, components): + """Assign the ``components`` as affected to this vulnerability.""" + self.affected_components.add(*components) + + @classmethod + def create_from_data(cls, dataspace, data, validate=False, affecting=None): + instance = super().create_from_data(user=dataspace, data=data, validate=False) + + if affecting: + instance.add_affected(affecting) + + return instance diff --git a/component_catalog/programming_languages.py b/component_catalog/programming_languages.py index 3f600089..565771fa 100644 --- a/component_catalog/programming_languages.py +++ b/component_catalog/programming_languages.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/templates/component_catalog/includes/add_package_modal.html b/component_catalog/templates/component_catalog/includes/add_package_modal.html index 8f282819..9f214c78 100644 --- a/component_catalog/templates/component_catalog/includes/add_package_modal.html +++ b/component_catalog/templates/component_catalog/includes/add_package_modal.html @@ -33,6 +33,13 @@ {% endif %} + {% if user.dataspace.enable_vulnerablecodedb_access %} +
+ + VulnerableCodeDB access is enabled in your Dataspace, DejaCode will look up the VulnerableCode to retrieve and fetch all vulnerabilities for the package. + +
+ {% endif %} diff --git a/component_catalog/templates/component_catalog/includes/component_list_table.html b/component_catalog/templates/component_catalog/includes/component_list_table.html index e0d040ea..f2aee581 100644 --- a/component_catalog/templates/component_catalog/includes/component_list_table.html +++ b/component_catalog/templates/component_catalog/includes/component_list_table.html @@ -25,7 +25,7 @@ {% endif %} - {% if object.has_hierarchy or object.request_count or object.cpe %} + {% if object.has_hierarchy or object.request_count or object.vulnerability_count %}
    {% if object.has_hierarchy %}
  • @@ -37,9 +37,9 @@ R
  • {% endif %} - {% if object.cpe in vulnerable_cpes %} + {% if object.vulnerability_count %}
  • - {% include 'component_catalog/includes/vulnerability_icon_link.html' %} + {% include 'component_catalog/includes/vulnerability_icon_link.html' with count=object.vulnerability_count %}
  • {% endif %}
diff --git a/component_catalog/templates/component_catalog/includes/hierarchy_instance_box.html b/component_catalog/templates/component_catalog/includes/hierarchy_instance_box.html index 3fe1b5a8..984c5198 100644 --- a/component_catalog/templates/component_catalog/includes/hierarchy_instance_box.html +++ b/component_catalog/templates/component_catalog/includes/hierarchy_instance_box.html @@ -28,9 +28,12 @@ {% endif %} + {% if relation.vulnerability_count %} + {% include 'component_catalog/includes/vulnerability_icon_link.html' with url=instance.get_absolute_url count=relation.vulnerability_count %} + {% endif %} {% if relation.package_id and relation.package.declared_dependencies.all %} + href="{{ product.get_absolute_url }}?dependencies-for_package__uuid={{ relation.package.uuid }}#dependencies" data-bs-toggle="tooltip" title="Dependencies" aria-label="Dependencies"> {{ relation.package.declared_dependencies.all|length }} {% endif %} @@ -41,6 +44,9 @@ {% endif %} + {% if relation.vulnerability_count %} + {% include 'component_catalog/includes/vulnerability_icon_link.html' with url=instance.get_absolute_url count=relation.vulnerability_count %} + {% endif %} {% elif not instance %} diff --git a/component_catalog/templates/component_catalog/includes/package_list_table.html b/component_catalog/templates/component_catalog/includes/package_list_table.html index c8f987f2..52f124a9 100644 --- a/component_catalog/templates/component_catalog/includes/package_list_table.html +++ b/component_catalog/templates/component_catalog/includes/package_list_table.html @@ -23,9 +23,9 @@ R {% endif %} - {% if object.plain_package_url in vulnerable_purls %} + {% if object.vulnerability_count %}
  • - {% include 'component_catalog/includes/vulnerability_icon_link.html' %} + {% include 'component_catalog/includes/vulnerability_icon_link.html' with count=object.vulnerability_count %}
  • {% endif %} diff --git a/component_catalog/templates/component_catalog/includes/vulnerability_icon_link.html b/component_catalog/templates/component_catalog/includes/vulnerability_icon_link.html index e360ab29..7507eaec 100644 --- a/component_catalog/templates/component_catalog/includes/vulnerability_icon_link.html +++ b/component_catalog/templates/component_catalog/includes/vulnerability_icon_link.html @@ -1,3 +1 @@ - - - \ No newline at end of file +{% if count %}{{ count }}{% endif %} \ No newline at end of file diff --git a/component_catalog/templates/component_catalog/tabs/tab_vulnerabilities.html b/component_catalog/templates/component_catalog/tabs/tab_vulnerabilities.html new file mode 100644 index 00000000..9f29dea3 --- /dev/null +++ b/component_catalog/templates/component_catalog/tabs/tab_vulnerabilities.html @@ -0,0 +1,67 @@ +{% load i18n %} + + + + + + + + + + + {% for vulnerability in values.vulnerabilities %} + + + + + + + {% endfor %} + +
    + + {% trans 'Affected by' %} + + + + {% trans 'Summary' %} + + + + {% trans 'Aliases' %} + + + + {% trans 'Fixed packages' %} + +
    + + {{ vulnerability.vulnerability_id }} + + + + {{ vulnerability.summary }} + + {% for alias in vulnerability.aliases %} + {% if alias|slice:":3" == "CVE" %} + {{ alias }} + + + {% elif alias|slice:":4" == "GHSA" %} + {{ alias }} + + + {% elif alias|slice:":3" == "NPM" %} + {{ alias }} + + + {% else %} + {{ alias }} + {% endif %} +
    + {% endfor %} +
    + {% if vulnerability.fixed_packages_html %} + {{ vulnerability.fixed_packages_html }} + {% endif %} +
    \ No newline at end of file diff --git a/component_catalog/tests/__init__.py b/component_catalog/tests/__init__.py index ea856412..8b27d87e 100644 --- a/component_catalog/tests/__init__.py +++ b/component_catalog/tests/__init__.py @@ -2,24 +2,65 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # +import random +import string + from component_catalog.models import Component from component_catalog.models import Package +from component_catalog.models import Vulnerability + + +def make_string(length): + return "".join(random.choices(string.ascii_letters, k=length)) -def make_package(dataspace, package_url=None, **data): +def make_package(dataspace, package_url=None, is_vulnerable=False, **data): + """Create a package for test purposes.""" + if not package_url and "filename" not in data: + data["filename"] = make_string(10) + package = Package(dataspace=dataspace, **data) if package_url: package.set_package_url(package_url) package.save() + + if is_vulnerable: + make_vulnerability(dataspace, affecting=package) + return package -def make_component(dataspace, **data): - return Component.objects.create( +def make_component(dataspace, is_vulnerable=False, **data): + """Create a component for test purposes.""" + if "name" not in data: + data["name"] = make_string(10) + + component = Component.objects.create( dataspace=dataspace, **data, ) + + if is_vulnerable: + make_vulnerability(dataspace, affecting=component) + + return component + + +def make_vulnerability(dataspace, affecting=None, **data): + """Create a vulnerability for test purposes.""" + if "vulnerability_id" not in data: + data["vulnerability_id"] = f"VCID-0000-{random.randint(1, 9999):04}" + + vulnerability = Vulnerability.objects.create( + dataspace=dataspace, + **data, + ) + + if affecting: + vulnerability.add_affected(affecting) + + return vulnerability diff --git a/component_catalog/tests/test_admin.py b/component_catalog/tests/test_admin.py index 75c51fe2..5d70ace2 100644 --- a/component_catalog/tests/test_admin.py +++ b/component_catalog/tests/test_admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -1980,6 +1980,25 @@ def test_package_changeform_filename_validation(self): self.assertEqual(200, response.status_code) self.assertEqual(errors, response.context_data["adminform"].form.errors) + @mock.patch("component_catalog.models.VulnerabilityMixin.fetch_vulnerabilities") + def test_package_changeform_fetch_vulnerabilities(self, mock_fetch_vulnerabilities): + mock_fetch_vulnerabilities.return_value = None + self.dataspace1.enable_vulnerablecodedb_access = True + self.dataspace1.save() + self.client.login(username="test", password="secret") + add_url = reverse("admin:component_catalog_package_add") + + data = { + "filename": "package.zip", + "componentassignedpackage_set-TOTAL_FORMS": 0, + "componentassignedpackage_set-INITIAL_FORMS": 0, + "dje-externalreference-content_type-object_id-TOTAL_FORMS": 0, + "dje-externalreference-content_type-object_id-INITIAL_FORMS": 0, + } + + self.client.post(add_url, data) + mock_fetch_vulnerabilities.assert_called() + def test_component_changelist_advanced_search_on_empty_space(self): self.client.login(username="test", password="secret") changelist_url = reverse("admin:component_catalog_component_changelist") diff --git a/component_catalog/tests/test_api.py b/component_catalog/tests/test_api.py index 9504e889..7ea34d3e 100644 --- a/component_catalog/tests/test_api.py +++ b/component_catalog/tests/test_api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/tests/test_command.py b/component_catalog/tests/test_command.py index f2c9d94d..3d0def24 100644 --- a/component_catalog/tests/test_command.py +++ b/component_catalog/tests/test_command.py @@ -2,11 +2,12 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # from io import StringIO +from unittest import mock from django.core import management from django.core.management.base import CommandError @@ -125,3 +126,26 @@ def test_componentfrompackage_management_command(self): expected = "Error: the following arguments are required: dataspace, username" self.assertEqual(expected, str(error.exception)) + + @mock.patch("component_catalog.vulnerabilities.fetch_from_vulnerablecode") + @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.is_configured") + def test_fetchvulnerabilities_management_command(self, mock_is_configured, mock_fetch): + mock_is_configured.return_value = False + self.assertFalse(self.dataspace.enable_vulnerablecodedb_access) + + options = [self.dataspace.name] + with self.assertRaises(CommandError) as error: + management.call_command("fetchvulnerabilities", *options) + expected = "VulnerableCode is not enabled on this Dataspace." + self.assertEqual(expected, str(error.exception)) + + self.dataspace.enable_vulnerablecodedb_access = True + self.dataspace.save() + with self.assertRaises(CommandError) as error: + management.call_command("fetchvulnerabilities", *options) + expected = "VulnerableCode is not configured." + self.assertEqual(expected, str(error.exception)) + + mock_is_configured.return_value = True + management.call_command("fetchvulnerabilities", *options) + mock_fetch.assert_called_once() diff --git a/component_catalog/tests/test_filters.py b/component_catalog/tests/test_filters.py index 5406718c..1899b7b4 100644 --- a/component_catalog/tests/test_filters.py +++ b/component_catalog/tests/test_filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -199,6 +199,21 @@ def test_component_filterset_related_only_values_filter(self): [("Python", "Python")], list(filterset.filters["primary_language"].field.choices) ) + def test_component_filterset_is_vulnerable_filter(self): + component1 = make_component(self.dataspace, is_vulnerable=True) + self.assertTrue(component1.is_vulnerable) + + filterset = ComponentFilterSet(dataspace=self.dataspace) + self.assertQuerySetEqual(filterset.qs, [component1]) + + data = {"is_vulnerable": "yes"} + filterset = ComponentFilterSet(dataspace=self.dataspace, data=data) + self.assertQuerySetEqual(filterset.qs, [component1]) + + data = {"is_vulnerable": "no"} + filterset = ComponentFilterSet(dataspace=self.dataspace, data=data) + self.assertQuerySetEqual(filterset.qs, []) + class ComponentFilterSearchTestCase(TestCase): testfiles_location = join(dirname(__file__), "testfiles") @@ -363,3 +378,20 @@ def test_package_filterset_search_match_order_on_purl_fields(self): filterset = PackageFilterSet(dataspace=self.dataspace, data=data) expected = ["pkg:pypi/django@4.0", "pkg:pypi/django@5.0"] self.assertEqual(sorted(expected), self.sorted_results(filterset.qs)) + + def test_package_filterset_is_vulnerable_filter(self): + package1 = make_package( + self.dataspace, package_url="pkg:pypi/django@5.0", is_vulnerable=True + ) + self.assertTrue(package1.is_vulnerable) + + filterset = PackageFilterSet(dataspace=self.dataspace) + self.assertIn(package1, filterset.qs) + + data = {"is_vulnerable": "yes"} + filterset = PackageFilterSet(dataspace=self.dataspace, data=data) + self.assertQuerySetEqual(filterset.qs, [package1]) + + data = {"is_vulnerable": "no"} + filterset = PackageFilterSet(dataspace=self.dataspace, data=data) + self.assertNotIn(package1, filterset.qs) diff --git a/component_catalog/tests/test_importers.py b/component_catalog/tests/test_importers.py index 0edaacc6..515a5271 100644 --- a/component_catalog/tests/test_importers.py +++ b/component_catalog/tests/test_importers.py @@ -2,13 +2,14 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # import datetime import os from os.path import dirname +from unittest import mock from django.contrib.contenttypes.models import ContentType from django.core.exceptions import NON_FIELD_ERRORS @@ -1396,6 +1397,18 @@ def test_package_import_add_to_product(self): self.assertContains(response, expected3) self.assertContains(response, expected4) + @mock.patch("component_catalog.importers.fetch_for_queryset") + def test_package_import_fetch_vulnerabilities(self, mock_fetch_for_queryset): + mock_fetch_for_queryset.return_value = None + self.dataspace.enable_vulnerablecodedb_access = True + self.dataspace.save() + + file = os.path.join(TESTFILES_LOCATION, "package_from_scancode.json") + importer = PackageImporter(self.super_user, file) + importer.save_all() + self.assertEqual(2, len(importer.results["added"])) + mock_fetch_for_queryset.assert_called() + class SubcomponentImporterTestCase(TestCase): def setUp(self): diff --git a/component_catalog/tests/test_license_expression_dje.py b/component_catalog/tests/test_license_expression_dje.py index 85ff65d7..fbf3f940 100644 --- a/component_catalog/tests/test_license_expression_dje.py +++ b/component_catalog/tests/test_license_expression_dje.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/tests/test_models.py b/component_catalog/tests/test_models.py index 9fd882cc..e942ca93 100644 --- a/component_catalog/tests/test_models.py +++ b/component_catalog/tests/test_models.py @@ -2,13 +2,14 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # import json import uuid from operator import attrgetter +from pathlib import Path from unittest import mock from django.contrib.contenttypes.models import ContentType @@ -37,9 +38,14 @@ from component_catalog.models import Package from component_catalog.models import PackageAlreadyExistsWarning from component_catalog.models import Subcomponent +from component_catalog.models import Vulnerability +from component_catalog.tests import make_component +from component_catalog.tests import make_package +from component_catalog.tests import make_vulnerability from dejacode_toolkit import download from dejacode_toolkit.download import DataCollectionException from dejacode_toolkit.download import collect_package_data +from dejacode_toolkit.vulnerablecode import VulnerableCode from dje.copier import copy_object from dje.models import Dataspace from dje.models import History @@ -58,6 +64,8 @@ class ComponentCatalogModelsTestCase(TestCase): + data = Path(__file__).parent / "testfiles" + def setUp(self): self.dataspace = Dataspace.objects.create(name="nexB") self.other_dataspace = Dataspace.objects.create(name="other") @@ -2567,3 +2575,129 @@ def test_package_model_update_from_purldb(self, mock_get_purldb_entries): for field_name in updated_fields: self.assertEqual(purldb_entry[field_name], getattr(package1, field_name)) + + def test_package_model_vulnerability_queryset_mixin(self): + package1 = make_package(self.dataspace, is_vulnerable=True) + package2 = make_package(self.dataspace) + + qs = Package.objects.with_vulnerability_count() + self.assertEqual(1, qs.get(pk=package1.pk).vulnerability_count) + self.assertEqual(0, qs.get(pk=package2.pk).vulnerability_count) + + qs = Package.objects.vulnerable() + self.assertQuerySetEqual(qs, [package1]) + + def test_vulnerability_mixin_is_vulnerable_property(self): + package1 = make_package(self.dataspace, is_vulnerable=True) + package2 = make_package(self.dataspace) + self.assertTrue(package1.is_vulnerable) + self.assertFalse(package2.is_vulnerable) + + @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.request_get") + def test_vulnerability_mixin_get_entry_for_package(self, mock_request_get): + vulnerablecode = VulnerableCode(self.dataspace) + package1 = make_package(self.dataspace, package_url="pkg:composer/guzzlehttp/psr7@1.9.0") + response_file = self.data / "vulnerabilities" / "idna_3.6_response.json" + mock_request_get.return_value = json.loads(response_file.read_text()) + + affected_by_vulnerabilities = package1.get_entry_for_package(vulnerablecode) + self.assertEqual(1, len(affected_by_vulnerabilities)) + self.assertEqual("VCID-j3au-usaz-aaag", affected_by_vulnerabilities[0]["vulnerability_id"]) + + @mock.patch("component_catalog.models.VulnerabilityMixin.get_entry_for_package") + @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.is_configured") + def test_vulnerability_mixin_get_entry_from_vulnerablecode( + self, mock_is_configured, mock_get_entry_for_package + ): + package1 = make_package(self.dataspace, package_url="pkg:pypi/idna@3.6") + self.assertIsNone(package1.get_entry_from_vulnerablecode()) + + mock_get_entry_for_package.return_value = None + self.dataspace.enable_vulnerablecodedb_access = True + self.dataspace.save() + mock_is_configured.return_value = True + package1.get_entry_from_vulnerablecode() + mock_get_entry_for_package.assert_called_once() + + @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.request_get") + @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.is_configured") + def test_vulnerability_mixin_fetch_vulnerabilities(self, mock_is_configured, mock_request_get): + mock_is_configured.return_value = True + self.dataspace.enable_vulnerablecodedb_access = True + self.dataspace.save() + response_file = self.data / "vulnerabilities" / "idna_3.6_response.json" + mock_request_get.return_value = json.loads(response_file.read_text()) + + package1 = make_package(self.dataspace, package_url="pkg:pypi/idna@3.6") + package1.fetch_vulnerabilities() + + self.assertEqual(1, Vulnerability.objects.scope(self.dataspace).count()) + self.assertEqual(1, package1.affected_by_vulnerabilities.count()) + vulnerability = package1.affected_by_vulnerabilities.get() + self.assertEqual("VCID-j3au-usaz-aaag", vulnerability.vulnerability_id) + + def test_vulnerability_mixin_create_vulnerabilities(self): + response_file = self.data / "vulnerabilities" / "idna_3.6_response.json" + response_json = json.loads(response_file.read_text()) + vulnerabilities_data = response_json["results"][0]["affected_by_vulnerabilities"] + + package1 = make_package(self.dataspace, package_url="pkg:pypi/idna@3.6") + package1.create_vulnerabilities(vulnerabilities_data) + + self.assertEqual(1, Vulnerability.objects.scope(self.dataspace).count()) + self.assertEqual(1, package1.affected_by_vulnerabilities.count()) + vulnerability = package1.affected_by_vulnerabilities.get() + self.assertEqual("VCID-j3au-usaz-aaag", vulnerability.vulnerability_id) + + def test_vulnerability_model_affected_packages_m2m(self): + package1 = make_package(self.dataspace) + vulnerablity1 = make_vulnerability(dataspace=self.dataspace, affecting=package1) + self.assertEqual(package1, vulnerablity1.affected_packages.get()) + self.assertEqual(vulnerablity1, package1.affected_by_vulnerabilities.get()) + + def test_vulnerability_model_add_affected(self): + vulnerablity1 = make_vulnerability(dataspace=self.dataspace) + package1 = make_package(self.dataspace) + package2 = make_package(self.dataspace) + vulnerablity1.add_affected(package1) + vulnerablity1.add_affected([package2]) + self.assertEqual(2, vulnerablity1.affected_packages.count()) + + vulnerablity2 = make_vulnerability(dataspace=self.dataspace) + component1 = make_component(self.dataspace) + vulnerablity2.add_affected([component1, package1]) + self.assertQuerySetEqual(vulnerablity2.affected_packages.all(), [package1]) + self.assertQuerySetEqual(vulnerablity2.affected_components.all(), [component1]) + + def test_vulnerability_model_create_from_data(self): + package1 = make_package(self.dataspace) + vulnerability_data = { + "vulnerability_id": "VCID-q4q6-yfng-aaag", + "summary": "In Django 3.2 before 3.2.25, 4.2 before 4.2.11, and 5.0.", + "aliases": ["CVE-2024-27351", "GHSA-vm8q-m57g-pff3", "PYSEC-2024-47"], + "references": [ + { + "reference_url": "https://access.redhat.com/hydra/rest/" + "securitydata/cve/CVE-2024-27351.json", + "reference_id": "", + "scores": [ + { + "value": "7.5", + "scoring_system": "cvssv3", + "scoring_elements": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H", + } + ], + }, + ], + } + + vulnerability1 = Vulnerability.create_from_data( + dataspace=self.dataspace, + data=vulnerability_data, + affecting=package1, + ) + self.assertEqual(vulnerability_data["vulnerability_id"], vulnerability1.vulnerability_id) + self.assertEqual(vulnerability_data["summary"], vulnerability1.summary) + self.assertEqual(vulnerability_data["aliases"], vulnerability1.aliases) + self.assertEqual(vulnerability_data["references"], vulnerability1.references) + self.assertQuerySetEqual(vulnerability1.affected_packages.all(), [package1]) diff --git a/component_catalog/tests/test_scancodeio.py b/component_catalog/tests/test_scancodeio.py index dae1135e..e0b0c66f 100644 --- a/component_catalog/tests/test_scancodeio.py +++ b/component_catalog/tests/test_scancodeio.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/tests/test_urn.py b/component_catalog/tests/test_urn.py index f10f940e..d53f8023 100644 --- a/component_catalog/tests/test_urn.py +++ b/component_catalog/tests/test_urn.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/tests/test_views.py b/component_catalog/tests/test_views.py index d376c3ea..e6b21573 100644 --- a/component_catalog/tests/test_views.py +++ b/component_catalog/tests/test_views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -39,9 +39,9 @@ from component_catalog.models import ComponentType from component_catalog.models import Package from component_catalog.models import Subcomponent +from component_catalog.tests import make_vulnerability from component_catalog.views import ComponentAddView from component_catalog.views import ComponentListView -from component_catalog.views import PackageDetailsView from component_catalog.views import PackageTabScanView from dejacode_toolkit.scancodeio import get_webhook_url from dejacode_toolkit.vulnerablecode import VulnerableCode @@ -1037,31 +1037,8 @@ def test_component_catalog_details_view_acceptable_linkages_in_policy_tab(self): ) self.assertContains(response, expected, html=True) - @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.get_vulnerabilities_by_cpe") - @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.is_configured") - def test_component_details_view_tab_vulnerabilities( - self, mock_is_configured, mock_get_vulnerabilities_by_cpe - ): - mock_is_configured.return_value = True - - self.nexb_dataspace.enable_vulnerablecodedb_access = True - self.nexb_dataspace.save() - - self.component1.cpe = "cpe:2.3:a:djangoproject:django:0.95:*:*:*:*:*:*:*" - self.component1.save() - - mock_get_vulnerabilities_by_cpe.return_value = [ - { - "vulnerability_id": "VULCOID-5U6", - "summary": "django.contrib.sessions in Django before 1.2.7", - "references": [ - { - "reference_url": "https://nvd.nist.gov/vuln/detail/CVE-2011-4136", - "reference_id": "CVE-2011-4136", - } - ], - } - ] + def test_component_details_view_tab_vulnerabilities(self): + vulnerability1 = make_vulnerability(self.nexb_dataspace, affecting=self.component1) self.client.login(username="nexb_user", password="t3st") response = self.client.get(self.component1.details_url) @@ -1073,11 +1050,7 @@ def test_component_details_view_tab_vulnerabilities( ) self.assertContains(response, expected) self.assertContains(response, 'id="tab_vulnerabilities"') - expected = ( - '' - "CVE-2011-4136" - ) - self.assertContains(response, expected) + self.assertContains(response, vulnerability1.vcid) def test_component_catalog_component_create_ajax_view(self): component_create_ajax_url = reverse("component_catalog:component_add_ajax") @@ -1144,6 +1117,7 @@ def setUp(self): self.package1.save() self.package1_tab_scan_url = self.package1.get_url("tab_scan") + self.vulnerability1 = make_vulnerability(self.dataspace, affecting=self.package1) ComponentAssignedPackage.objects.create( component=self.component1, package=self.package1, dataspace=self.dataspace ) @@ -1155,7 +1129,7 @@ def setUp(self): def test_package_list_view_num_queries(self): self.client.login(username=self.super_user.username, password="secret") - with self.assertNumQueries(17): + with self.assertNumQueries(16): self.client.get(reverse("component_catalog:package_list")) def test_package_views_urls(self): @@ -1265,7 +1239,7 @@ def test_package_list_multi_send_about_files_view(self): def test_package_details_view_num_queries(self): self.client.login(username=self.super_user.username, password="secret") - with self.assertNumQueries(27): + with self.assertNumQueries(29): self.client.get(self.package1.get_absolute_url()) def test_package_details_view_content(self): @@ -1757,6 +1731,18 @@ def test_package_create_ajax_view(self): Package.objects.filter(download_url=collected_data["download_url"]).exists() ) + @mock.patch("component_catalog.models.VulnerabilityMixin.fetch_vulnerabilities") + def test_package_create_ajax_view_fetch_vulnerabilities(self, mock_fetch_vulnerabilities): + mock_fetch_vulnerabilities.return_value = None + package_add_url = reverse("component_catalog:package_add_urls") + self.dataspace.enable_vulnerablecodedb_access = True + self.dataspace.save() + self.client.login(username=self.super_user.username, password="secret") + + data = {"download_urls": "pkg:pypi/django@2.1"} + self.client.post(package_add_url, data) + mock_fetch_vulnerabilities.assert_called() + def test_package_details_view_add_to_product(self): self.client.login(username=self.basic_user.username, password="secret") @@ -2973,128 +2959,22 @@ def test_send_scan_data_as_file_view(self, mock_fetch_scan_data): 'attachment; filename="package1_scan.zip"', response["content-disposition"] ) - @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.get_vulnerable_purls") - @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.is_configured") - def test_package_list_view_vulnerabilities(self, mock_is_configured, mock_vulnerable_purls): - purl = "pkg:pypi/django@2.1" - mock_is_configured.return_value = True - - self.package1.set_package_url(purl) - self.package1.save() - - self.dataspace.enable_vulnerablecodedb_access = True - self.dataspace.save() - - mock_vulnerable_purls.return_value = [purl] - + def test_package_list_view_vulnerabilities(self): self.client.login(username=self.super_user.username, password="secret") package_list_url = reverse("component_catalog:package_list") response = self.client.get(package_list_url) self.assertContains(response, self.package1.identifier) self.assertContains(response, "#vulnerabilities") - expected = '' + expected = f""" + + 1 + + """ self.assertContains(response, expected, html=True) - def test_package_details_view_get_vulnerability_fields(self): - self.package1.set_package_url("pkg:nginx/nginx@1.11.1") - self.package1.save() - copy_object(self.package1, Dataspace.objects.create(name="Other"), self.basic_user) - - get_vulnerability_fields = PackageDetailsView.get_vulnerability_fields - fields = get_vulnerability_fields(vulnerability={}, dataspace=self.dataspace) - self.assertEqual(fields[0], ("Summary", None, "Summary of the vulnerability")) - - vulnerability_url = "http://public.vulnerablecode.io/vulnerabilities/VCID-pk3r-ga7k-aaap" - vulnerability = { - "vulnerability_id": "VCID-pk3r-ga7k-aaap", - "summary": "SQL Injection", - "resource_url": vulnerability_url, - "references": [ - { - "reference_id": "", - "reference_url": "http://www.openwall.com/lists/oss-security/2022/01/18/4", - "scores": [], - }, - { - "reference_id": "CVE-2022-23305", - "reference_url": "https://nvd.nist.gov/vuln/detail/CVE-2022-23305", - "scores": [], - }, - ], - "fixed_packages": [ - {"purl": "pkg:nginx/nginx@1.11.1"}, - {"purl": "pkg:nginx/nginx@1.10.1"}, - ], - } - fields = get_vulnerability_fields( - vulnerability=vulnerability, - dataspace=self.dataspace, - ) - self.assertEqual(fields[0], ("Summary", "SQL Injection", "Summary of the vulnerability")) - self.assertEqual(fields[1][0], "VulnerableCode URL") - url_as_link = f'{vulnerability_url}' - self.assertEqual(fields[1][1], url_as_link) - self.assertEqual(fields[2][0], "Fixed packages") - fixed_package_values = fields[2][1] - self.assertIn("nginx/nginx@1.10.1", fixed_package_values) - self.assertIn( - 'pkg:nginx/nginx@1.11.1', - fixed_package_values, - ) - self.assertEqual( - fields[3][0:2], - ( - "Reference IDs", - '' - "CVE-2022-23305" - "", - ), - ) - self.assertEqual( - fields[4][0:2], - ( - "Reference URLs", - 'http://www.openwall.com/lists/oss-security/2022/01/18/4', - ), - ) - - @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.get_vulnerabilities_by_purl") - @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.is_configured") - def test_package_details_view_tab_vulnerabilities( - self, mock_is_configured, mock_get_vulnerabilities_by_purl - ): - purl = "pkg:pypi/django@2.1" - mock_is_configured.return_value = True - - self.package1.set_package_url(purl) - self.package1.save() - - self.dataspace.enable_vulnerablecodedb_access = True - self.dataspace.save() - - mock_get_vulnerabilities_by_purl.return_value = [ - { - "purl": "pkg:pypi/django@2.1", - "affected_by_vulnerabilities": [ - { - "summary": "SQL Injection", - "references": [ - { - "reference_id": "CVE-2022-23305", - "reference_url": "https://nvd.nist.gov/vuln/detail/CVE-2022-23305", - } - ], - }, - ], - } - ] - + def test_package_details_view_tab_vulnerabilities(self): self.client.login(username=self.super_user.username, password="secret") response = self.client.get(self.package1.details_url) @@ -3105,12 +2985,7 @@ def test_package_details_view_tab_vulnerabilities( ) self.assertContains(response, expected) self.assertContains(response, 'id="tab_vulnerabilities"') - expected = ( - '' - "CVE-2022-23305" - "" - ) - self.assertContains(response, expected) + self.assertContains(response, self.vulnerability1.vcid) def test_vulnerablecode_get_plain_purls(self): purls = get_plain_purls(packages=[]) @@ -3521,6 +3396,23 @@ def test_component_catalog_package_add_view_initial_data( } self.assertEqual(expected, response.context["form"].initial) + @mock.patch("component_catalog.models.VulnerabilityMixin.fetch_vulnerabilities") + def test_component_catalog_package_add_view_fetch_vulnerabilities( + self, mock_fetch_vulnerabilities + ): + mock_fetch_vulnerabilities.return_value = None + self.client.login(username=self.super_user.username, password="secret") + add_url = reverse("component_catalog:package_add") + self.dataspace.enable_vulnerablecodedb_access = True + self.dataspace.save() + + data = { + "filename": "name.zip", + "submit": "Add Package", + } + self.client.post(add_url, data) + mock_fetch_vulnerabilities.assert_called() + @mock.patch("dje.tasks.scancodeio_submit_scan.delay") @mock.patch("dejacode_toolkit.scancodeio.ScanCodeIO.is_configured") def test_component_catalog_package_add_view_create_with_submit_scan( @@ -3957,7 +3849,7 @@ def setUp(self): def test_component_catalog_list_view_num_queries(self): self.client.login(username="nexb_user", password="t3st") - with self.assertNumQueries(18): + with self.assertNumQueries(17): self.client.get(reverse("component_catalog:component_list")) def test_component_catalog_list_view_default(self): diff --git a/component_catalog/tests/test_vulnerabilities.py b/component_catalog/tests/test_vulnerabilities.py new file mode 100644 index 00000000..7ea637de --- /dev/null +++ b/component_catalog/tests/test_vulnerabilities.py @@ -0,0 +1,62 @@ +# +# Copyright (c) nexB Inc. and others. All rights reserved. +# DejaCode is a trademark of nexB Inc. +# SPDX-License-Identifier: AGPL-3.0-only +# See https://github.com/aboutcode-org/dejacode for support or download. +# See https://aboutcode.org for more information about AboutCode FOSS projects. +# + +import io +import json +from pathlib import Path +from unittest import mock + +from django.test import TestCase + +from component_catalog.models import Package +from component_catalog.tests import make_package +from component_catalog.vulnerabilities import fetch_for_queryset +from component_catalog.vulnerabilities import fetch_from_vulnerablecode +from dje.models import Dataspace + + +class VulnerabilitiesTestCase(TestCase): + data = Path(__file__).parent / "testfiles" + + def setUp(self): + self.dataspace = Dataspace.objects.create(name="nexB") + + @mock.patch("component_catalog.vulnerabilities.fetch_for_queryset") + @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.is_configured") + def test_vulnerabilities_fetch_from_vulnerablecode( + self, mock_is_configured, mock_fetch_for_queryset + ): + buffer = io.StringIO() + make_package(self.dataspace, package_url="pkg:pypi/idna@3.6") + make_package(self.dataspace, package_url="pkg:pypi/idna@2.0") + mock_is_configured.return_value = True + mock_fetch_for_queryset.return_value = 2 + fetch_from_vulnerablecode(self.dataspace, batch_size=1, timeout=None, log_func=buffer.write) + expected = "2 Packages in the queue.+ Created 2 vulnerabilitiesCompleted in 0 seconds" + self.assertEqual(expected, buffer.getvalue()) + self.dataspace.refresh_from_db() + self.assertIsNotNone(self.dataspace.vulnerabilities_updated_at) + + @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.bulk_search_by_purl") + def test_vulnerabilities_fetch_for_queryset(self, mock_bulk_search_by_purl): + buffer = io.StringIO() + package1 = make_package(self.dataspace, package_url="pkg:pypi/idna@3.6") + make_package(self.dataspace, package_url="pkg:pypi/idna@2.0") + queryset = Package.objects.scope(self.dataspace) + response_file = self.data / "vulnerabilities" / "idna_3.6_response.json" + response_json = json.loads(response_file.read_text()) + mock_bulk_search_by_purl.return_value = response_json["results"] + + created_vulnerabilities = fetch_for_queryset( + queryset, self.dataspace, batch_size=1, log_func=buffer.write + ) + self.assertEqual(1, created_vulnerabilities) + self.assertEqual("Progress: 1/2Progress: 2/2", buffer.getvalue()) + self.assertEqual(1, package1.affected_by_vulnerabilities.count()) + vulnerability = package1.affected_by_vulnerabilities.get() + self.assertEqual("VCID-j3au-usaz-aaag", vulnerability.vulnerability_id) diff --git a/component_catalog/tests/testfiles/all_licenses.py b/component_catalog/tests/testfiles/all_licenses.py index 27225bce..efef4443 100644 --- a/component_catalog/tests/testfiles/all_licenses.py +++ b/component_catalog/tests/testfiles/all_licenses.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/tests/testfiles/vulnerabilities/idna_3.6_response.json b/component_catalog/tests/testfiles/vulnerabilities/idna_3.6_response.json new file mode 100644 index 00000000..54c2b790 --- /dev/null +++ b/component_catalog/tests/testfiles/vulnerabilities/idna_3.6_response.json @@ -0,0 +1,270 @@ +{ + "count": 1, + "next": null, + "previous": null, + "results": [ + { + "url": "http://public.vulnerablecode.io/api/packages/844821", + "purl": "pkg:pypi/idna@3.6", + "type": "pypi", + "namespace": "", + "name": "idna", + "version": "3.6", + "qualifiers": {}, + "subpath": "", + "is_vulnerable": true, + "next_non_vulnerable_version": "3.7", + "latest_non_vulnerable_version": "3.7", + "affected_by_vulnerabilities": [ + { + "url": "http://public.vulnerablecode.io/api/vulnerabilities/525663", + "vulnerability_id": "VCID-j3au-usaz-aaag", + "summary": "Internationalized Domain Names in Applications (IDNA) vulnerable to denial of service from specially crafted inputs to idna.encode", + "references": [ + { + "reference_url": "https://access.redhat.com/hydra/rest/securitydata/cve/CVE-2024-3651.json", + "reference_id": "", + "reference_type": "", + "scores": [ + { + "value": "6.5", + "scoring_system": "cvssv3", + "scoring_elements": "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H" + } + ], + "url": "https://access.redhat.com/hydra/rest/securitydata/cve/CVE-2024-3651.json" + }, + { + "reference_url": "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-3651", + "reference_id": "", + "reference_type": "", + "scores": [], + "url": "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-3651" + }, + { + "reference_url": "https://ftp.suse.com/pub/projects/security/yaml/suse-cvss-scores.yaml", + "reference_id": "", + "reference_type": "", + "scores": [ + { + "value": "2.1", + "scoring_system": "cvssv2", + "scoring_elements": "AV:N/AC:H/Au:S/C:N/I:P/A:N" + }, + { + "value": "4.4", + "scoring_system": "cvssv3", + "scoring_elements": "CVSS:3.0/AV:L/AC:L/PR:N/UI:R/S:U/C:L/I:N/A:L" + }, + { + "value": "5.5", + "scoring_system": "cvssv3.1", + "scoring_elements": "CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H" + } + ], + "url": "https://ftp.suse.com/pub/projects/security/yaml/suse-cvss-scores.yaml" + }, + { + "reference_url": "https://github.com/kjd/idna", + "reference_id": "", + "reference_type": "", + "scores": [ + { + "value": "6.2", + "scoring_system": "cvssv3.1", + "scoring_elements": "CVSS:3.1/AV:L/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + }, + { + "value": "MODERATE", + "scoring_system": "generic_textual", + "scoring_elements": "" + } + ], + "url": "https://github.com/kjd/idna" + }, + { + "reference_url": "https://github.com/kjd/idna/commit/1d365e17e10d72d0b7876316fc7b9ca0eebdd38d", + "reference_id": "", + "reference_type": "", + "scores": [ + { + "value": "7.5", + "scoring_system": "cvssv3.1", + "scoring_elements": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + }, + { + "value": "MODERATE", + "scoring_system": "generic_textual", + "scoring_elements": "" + } + ], + "url": "https://github.com/kjd/idna/commit/1d365e17e10d72d0b7876316fc7b9ca0eebdd38d" + }, + { + "reference_url": "https://github.com/pypa/advisory-database/tree/main/vulns/idna/PYSEC-2024-60.yaml", + "reference_id": "", + "reference_type": "", + "scores": [ + { + "value": "6.2", + "scoring_system": "cvssv3.1", + "scoring_elements": "CVSS:3.1/AV:L/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + }, + { + "value": "MODERATE", + "scoring_system": "generic_textual", + "scoring_elements": "" + } + ], + "url": "https://github.com/pypa/advisory-database/tree/main/vulns/idna/PYSEC-2024-60.yaml" + }, + { + "reference_url": "https://huntr.com/bounties/93d78d07-d791-4b39-a845-cbfabc44aadb", + "reference_id": "", + "reference_type": "", + "scores": [ + { + "value": "7.5", + "scoring_system": "cvssv3.1", + "scoring_elements": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + }, + { + "value": "MODERATE", + "scoring_system": "generic_textual", + "scoring_elements": "" + } + ], + "url": "https://huntr.com/bounties/93d78d07-d791-4b39-a845-cbfabc44aadb" + }, + { + "reference_url": "https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1069127", + "reference_id": "1069127", + "reference_type": "", + "scores": [], + "url": "https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1069127" + }, + { + "reference_url": "https://bugzilla.redhat.com/show_bug.cgi?id=2274779", + "reference_id": "2274779", + "reference_type": "", + "scores": [], + "url": "https://bugzilla.redhat.com/show_bug.cgi?id=2274779" + }, + { + "reference_url": "https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:kjd:internationalized_domain_names_in_applications:3.6:*:*:*:*:*:*:*", + "reference_id": "cpe:2.3:a:kjd:internationalized_domain_names_in_applications:3.6:*:*:*:*:*:*:*", + "reference_type": "", + "scores": [], + "url": "https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:kjd:internationalized_domain_names_in_applications:3.6:*:*:*:*:*:*:*" + }, + { + "reference_url": "https://nvd.nist.gov/vuln/detail/CVE-2024-3651", + "reference_id": "CVE-2024-3651", + "reference_type": "", + "scores": [ + { + "value": "7.5", + "scoring_system": "cvssv3", + "scoring_elements": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + ], + "url": "https://nvd.nist.gov/vuln/detail/CVE-2024-3651" + }, + { + "reference_url": "https://github.com/advisories/GHSA-jjg7-2v4v-x38h", + "reference_id": "GHSA-jjg7-2v4v-x38h", + "reference_type": "", + "scores": [ + { + "value": "MODERATE", + "scoring_system": "cvssv3.1_qr", + "scoring_elements": "" + } + ], + "url": "https://github.com/advisories/GHSA-jjg7-2v4v-x38h" + }, + { + "reference_url": "https://github.com/kjd/idna/security/advisories/GHSA-jjg7-2v4v-x38h", + "reference_id": "GHSA-jjg7-2v4v-x38h", + "reference_type": "", + "scores": [ + { + "value": "MODERATE", + "scoring_system": "cvssv3.1_qr", + "scoring_elements": "" + } + ], + "url": "https://github.com/kjd/idna/security/advisories/GHSA-jjg7-2v4v-x38h" + }, + { + "reference_url": "https://access.redhat.com/errata/RHSA-2024:3466", + "reference_id": "RHSA-2024:3466", + "reference_type": "", + "scores": [], + "url": "https://access.redhat.com/errata/RHSA-2024:3466" + }, + { + "reference_url": "https://access.redhat.com/errata/RHSA-2024:3543", + "reference_id": "RHSA-2024:3543", + "reference_type": "", + "scores": [], + "url": "https://access.redhat.com/errata/RHSA-2024:3543" + }, + { + "reference_url": "https://access.redhat.com/errata/RHSA-2024:3552", + "reference_id": "RHSA-2024:3552", + "reference_type": "", + "scores": [], + "url": "https://access.redhat.com/errata/RHSA-2024:3552" + }, + { + "reference_url": "https://access.redhat.com/errata/RHSA-2024:3781", + "reference_id": "RHSA-2024:3781", + "reference_type": "", + "scores": [], + "url": "https://access.redhat.com/errata/RHSA-2024:3781" + }, + { + "reference_url": "https://access.redhat.com/errata/RHSA-2024:3846", + "reference_id": "RHSA-2024:3846", + "reference_type": "", + "scores": [], + "url": "https://access.redhat.com/errata/RHSA-2024:3846" + }, + { + "reference_url": "https://access.redhat.com/errata/RHSA-2024:4260", + "reference_id": "RHSA-2024:4260", + "reference_type": "", + "scores": [], + "url": "https://access.redhat.com/errata/RHSA-2024:4260" + }, + { + "reference_url": "https://usn.ubuntu.com/6780-1/", + "reference_id": "USN-6780-1", + "reference_type": "", + "scores": [], + "url": "https://usn.ubuntu.com/6780-1/" + } + ], + "fixed_packages": [ + { + "url": "http://public.vulnerablecode.io/api/packages/810901", + "purl": "pkg:pypi/idna@3.7", + "is_vulnerable": false, + "affected_by_vulnerabilities": [], + "resource_url": "http://public.vulnerablecode.io/packages/pkg:pypi/idna@3.7" + } + ], + "aliases": [ + "CVE-2024-3651", + "GHSA-jjg7-2v4v-x38h", + "PYSEC-2024-60" + ], + "resource_url": "http://public.vulnerablecode.io/vulnerabilities/VCID-j3au-usaz-aaag" + } + ], + "fixing_vulnerabilities": [], + "resource_url": "http://public.vulnerablecode.io/packages/pkg:pypi/idna@3.6" + } + ] +} \ No newline at end of file diff --git a/component_catalog/urls.py b/component_catalog/urls.py index ba9298ee..53ff217a 100644 --- a/component_catalog/urls.py +++ b/component_catalog/urls.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/component_catalog/views.py b/component_catalog/views.py index 7fef7414..22aff7de 100644 --- a/component_catalog/views.py +++ b/component_catalog/views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -95,7 +95,6 @@ from dje.views import DataspacedCreateView from dje.views import DataspacedFilterView from dje.views import DataspacedUpdateView -from dje.views import FieldLastLoop from dje.views import FieldSeparator from dje.views import Header from dje.views import LicenseDataForBuilderMixin @@ -250,169 +249,94 @@ def include_type(view_instance): class TabVulnerabilityMixin: - vulnerability_matching_field = None + template = "component_catalog/tabs/tab_vulnerabilities.html" def tab_vulnerabilities(self): - matching_value = getattr(self.object, self.vulnerability_matching_field) - dataspace = self.object.dataspace - vulnerablecode = VulnerableCode(self.request.user.dataspace) - - # The display of vulnerabilities is controlled by the object Dataspace - display_tab = all( - [ - matching_value, - dataspace.enable_vulnerablecodedb_access, - vulnerablecode.is_configured(), - ] - ) - - if not display_tab: - return - - vulnerability_getter_function = { - "cpe": vulnerablecode.get_vulnerabilities_by_cpe, - "package_url": vulnerablecode.get_vulnerabilities_by_purl, - }.get(self.vulnerability_matching_field) - - vulnerabilities = vulnerability_getter_function(matching_value, timeout=3) - if not vulnerabilities: + vulnerabilities_qs = self.object.affected_by_vulnerabilities.all() + if not vulnerabilities_qs: return - fields, vulnerabilities_count = self.get_vulnerabilities_tab_fields(vulnerabilities) - - if fields: - label = ( - f"Vulnerabilities" - f' {vulnerabilities_count}' - ) - return { - "fields": fields, - "label": format_html(label), - } + vulnerablecode = VulnerableCode(self.object.dataspace) + label = ( + f"Vulnerabilities" + f' {len(vulnerabilities_qs)}' + ) - def get_vulnerabilities_tab_fields(self, vulnerabilities): - raise NotImplementedError + vulnerabilities = [] + for vulnerability in vulnerabilities_qs: + fixed_packages_html = self.get_fixed_packages_html(vulnerability, self.object.dataspace) + vulnerability.fixed_packages_html = fixed_packages_html + vulnerabilities.append(vulnerability) - @staticmethod - def get_vulnerability_fields(vulnerability, dataspace): - include_fixed_packages = "fixed_packages" in vulnerability.keys() - summary = vulnerability.get("summary") - references = vulnerability.get("references", []) - - reference_urls = [] - reference_ids = [] - - for reference in references: - url = reference.get("reference_url") - reference_id = reference.get("reference_id") - if url and reference_id: - reference_ids.append(f'{reference_id}') - elif reference_id: - reference_ids.append(reference_id) - elif url: - reference_urls.append(url) - - reference_urls_joined = "\n".join(reference_urls) - reference_ids_joined = "\n".join(reference_ids) + context = { + "vulnerabilities": vulnerabilities, + "vulnerablecode_url": vulnerablecode.service_url, + } - tab_fields = [ - (_("Summary"), summary, "Summary of the vulnerability"), - ] + return { + "fields": [(None, context, None, self.template)], + "label": format_html(label), + } - if vulnerability_url := vulnerability.get("resource_url"): - vulnerability_url_help = "Link to the VulnerableCode app." - url_as_link = format_html( - '{vulnerability_url}', - vulnerability_url=vulnerability_url, - ) - tab_fields.append((_("VulnerableCode URL"), url_as_link, vulnerability_url_help)) - - if include_fixed_packages: - fixed_packages = vulnerability.get("fixed_packages", []) - fixed_packages_sorted = natsorted(fixed_packages, key=itemgetter("purl")) - add_package_url = reverse("component_catalog:package_add") - vulnerability_icon = ( - '' - '' - "" - ) - no_vulnerabilities_icon = ( - '' - ' ' - ' ' - "" - ) + def get_fixed_packages_html(self, vulnerability, dataspace): + if not vulnerability.fixed_packages: + return - fixed_packages_values = [] - for fixed_package in fixed_packages_sorted: - purl = fixed_package.get("purl") - is_vulnerable = fixed_package.get("is_vulnerable") - package_instances = Package.objects.scope(dataspace).for_package_url(purl) - - for package in package_instances: - absolute_url = package.get_absolute_url() - display_value = package.get_html_link(href=absolute_url) - if is_vulnerable: - display_value += package.get_html_link( - href=f"{absolute_url}#vulnerabilities", - value=format_html(vulnerability_icon), - ) - else: - display_value += no_vulnerabilities_icon - fixed_packages_values.append(display_value) + fixed_packages_sorted = natsorted(vulnerability.fixed_packages, key=itemgetter("purl")) + add_package_url = reverse("component_catalog:package_add") + vulnerability_icon = ( + '' + '' + "" + ) + no_vulnerabilities_icon = ( + '' + ' ' + ' ' + "" + ) - if not package_instances: - display_value = purl.replace("pkg:", "") - if is_vulnerable: - display_value += vulnerability_icon - else: - display_value += no_vulnerabilities_icon - # Warning: do not add spaces between HTML elements as this content - # is displayed in a
    -                    display_value += (
    -                        f''
    -                        f''
    -                        f''
    -                        f""
    -                        f""
    +        fixed_packages_values = []
    +        for fixed_package in fixed_packages_sorted:
    +            purl = fixed_package.get("purl")
    +            is_vulnerable = fixed_package.get("is_vulnerable")
    +            package_instances = Package.objects.scope(dataspace).for_package_url(purl)
    +
    +            for package in package_instances:
    +                absolute_url = package.get_absolute_url()
    +                display_value = package.get_html_link(href=absolute_url)
    +                if is_vulnerable:
    +                    display_value += package.get_html_link(
    +                        href=f"{absolute_url}#vulnerabilities",
    +                        value=format_html(vulnerability_icon),
                         )
    -                    fixed_packages_values.append(display_value)
    -
    -            tab_fields.append(
    -                (
    -                    _("Fixed packages"),
    -                    format_html("\n".join(fixed_packages_values)),
    -                    "The identifiers of Package Versions that have been reported to fix a "
    -                    "specific vulnerability and collected in VulnerableCodeDB.",
    -                ),
    -            )
    +                else:
    +                    display_value += no_vulnerabilities_icon
    +                fixed_packages_values.append(display_value)
     
    -        tab_fields.extend(
    -            [
    -                (
    -                    _("Reference IDs"),
    -                    format_html(reference_ids_joined),
    -                    "Reference IDs to the reported vulnerability, such as a DSA "
    -                    "(Debian Security Advisory) ID or a CVE (Common Vulnerabilities "
    -                    "and Exposures) ID, when available.",
    -                ),
    -                (
    -                    _("Reference URLs"),
    -                    urlize_target_blank(reference_urls_joined),
    -                    "The URLs collected in VulnerableCodeDB that give you quick "
    -                    "access to public websites that provide details about a "
    -                    "vulnerability.",
    -                ),
    -                FieldLastLoop,
    -            ]
    -        )
    +            if not package_instances:
    +                display_value = purl.replace("pkg:", "")
    +                if is_vulnerable:
    +                    display_value += vulnerability_icon
    +                else:
    +                    display_value += no_vulnerabilities_icon
    +                # Warning: do not add spaces between HTML elements as this content
    +                # is displayed in a 
    +                display_value += (
    +                    f''
    +                    f''
    +                    f''
    +                    f""
    +                    f""
    +                )
    +                fixed_packages_values.append(display_value)
     
    -        return tab_fields
    +        return format_html("
    ".join(fixed_packages_values)) class ComponentListView( @@ -432,7 +356,7 @@ class ComponentListView( group_name_version = True table_headers = ( - Header("name", _("Component name")), + Header("name", _("Component name"), filter="is_vulnerable"), Header("version", _("Version")), Header("usage_policy", _("Policy"), filter="usage_policy", condition=include_policy), Header("license_expression", _("Concluded license"), filter="licenses"), @@ -472,30 +396,12 @@ def get_queryset(self): "licenses__usage_policy", ) .with_has_hierarchy() + .with_vulnerability_count() .order_by( "-last_modified_date", ) ) - def get_context_data(self, **kwargs): - context = super().get_context_data(**kwargs) - vulnerablecode = VulnerableCode(self.request.user.dataspace) - - # The display of vulnerabilities is controlled by the objects Dataspace - enable_vulnerabilities = all( - [ - self.dataspace.enable_vulnerablecodedb_access, - vulnerablecode.is_configured(), - ] - ) - - if enable_vulnerabilities: - context["vulnerable_cpes"] = vulnerablecode.get_vulnerable_cpes( - components=context["object_list"], - ) - - return context - class ComponentDetailsView( AcceptAnonymousMixin, @@ -510,7 +416,6 @@ class ComponentDetailsView( add_to_product_perm = "product_portfolio.add_productcomponent" show_previous_and_next_object_links = True include_reference_dataspace = True - vulnerability_matching_field = "cpe" tabset = { "essentials": { "fields": [ @@ -878,18 +783,6 @@ def tab_legal(self): return {"fields": fields} - def get_vulnerabilities_tab_fields(self, vulnerabilities): - dataspace = self.object.dataspace - fields = [] - vulnerabilities_count = 0 - - for vulnerability in vulnerabilities: - vulnerability_fields = self.get_vulnerability_fields(vulnerability, dataspace) - fields.extend(vulnerability_fields) - vulnerabilities_count += 1 - - return fields, vulnerabilities_count - def get_context_data(self, **kwargs): context = super().get_context_data(**kwargs) user = self.request.user @@ -1074,7 +967,12 @@ class PackageListView( include_reference_dataspace = True put_results_in_session = True table_headers = ( - Header("sortable_identifier", _("Identifier"), Package.identifier_help()), + Header( + "sortable_identifier", + _("Identifier"), + Package.identifier_help(), + filter="is_vulnerable", + ), Header("usage_policy", _("Policy"), filter="usage_policy", condition=include_policy), Header("license_expression", _("Concluded license"), filter="licenses"), Header("primary_language", _("Language"), filter="primary_language"), @@ -1105,6 +1003,7 @@ def get_queryset(self): "dataspace", ) .annotate_sortable_identifier() + .with_vulnerability_count() .select_related( "usage_policy", ) @@ -1128,19 +1027,6 @@ def get_context_data(self, **kwargs): if self.request.user.has_perm("component_catalog.change_component"): context["add_to_component_form"] = AddMultipleToComponentForm(self.request) - vulnerablecode = VulnerableCode(self.request.user.dataspace) - # The display of vulnerabilities is controlled by the objects Dataspace - enable_vulnerabilities = all( - [ - self.dataspace.enable_vulnerablecodedb_access, - vulnerablecode.is_configured(), - ] - ) - - if enable_vulnerabilities: - packages = context["object_list"] - context["vulnerable_purls"] = vulnerablecode.get_vulnerable_purls(packages) - return context def post_add_to_component(self, form_class): @@ -1191,7 +1077,6 @@ class PackageDetailsView( template_name = "component_catalog/package_details.html" form_class = PackageAddToProductForm add_to_product_perm = "product_portfolio.add_productpackage" - vulnerability_matching_field = "package_url" tabset = { "essentials": { "fields": [ @@ -1494,20 +1379,6 @@ def tab_aboutcode(self): return {"fields": [(None, context, None, template)]} - def get_vulnerabilities_tab_fields(self, vulnerabilities): - dataspace = self.object.dataspace - fields = [] - vulnerabilities_count = 0 - - for entry in vulnerabilities: - unresolved = entry.get("affected_by_vulnerabilities", []) - for vulnerability in unresolved: - vulnerability_fields = self.get_vulnerability_fields(vulnerability, dataspace) - fields.extend(vulnerability_fields) - vulnerabilities_count += 1 - - return fields, vulnerabilities_count - @staticmethod def readable_date(date): if date: @@ -1633,6 +1504,10 @@ def package_create_ajax_view(request): ) scan_msg = " and submitted to ScanCode.io for scanning" + if dataspace.enable_vulnerablecodedb_access: + for package in created: + package.fetch_vulnerabilities() + if len_created == 1: redirect_url = created[0].get_absolute_url() messages.success(request, "The Package was successfully created.") diff --git a/component_catalog/vulnerabilities.py b/component_catalog/vulnerabilities.py new file mode 100644 index 00000000..a946ebac --- /dev/null +++ b/component_catalog/vulnerabilities.py @@ -0,0 +1,93 @@ +# +# Copyright (c) nexB Inc. and others. All rights reserved. +# DejaCode is a trademark of nexB Inc. +# SPDX-License-Identifier: AGPL-3.0-only +# See https://github.com/aboutcode-org/dejacode for support or download. +# See https://aboutcode.org for more information about AboutCode FOSS projects. +# + +from timeit import default_timer as timer + +from django.contrib.humanize.templatetags.humanize import intcomma +from django.core.management.base import CommandError +from django.utils import timezone + +from component_catalog.models import PACKAGE_URL_FIELDS +from component_catalog.models import Package +from component_catalog.models import Vulnerability +from dejacode_toolkit.vulnerablecode import VulnerableCode +from dje.utils import chunked_queryset +from dje.utils import humanize_time + + +def fetch_from_vulnerablecode(dataspace, batch_size, timeout, log_func=None): + start_time = timer() + vulnerablecode = VulnerableCode(dataspace) + if not vulnerablecode.is_configured(): + return + + available_types = vulnerablecode.get_package_url_available_types() + package_qs = ( + Package.objects.scope(dataspace) + .has_package_url() + .only("dataspace", *PACKAGE_URL_FIELDS) + .filter(type__in=available_types) + .order_by("-last_modified_date") + ) + package_count = package_qs.count() + if log_func: + log_func(f"{package_count} Packages in the queue.") + + created = fetch_for_queryset(package_qs, dataspace, batch_size, timeout, log_func) + run_time = timer() - start_time + if log_func: + log_func(f"+ Created {intcomma(created)} vulnerabilities") + log_func(f"Completed in {humanize_time(run_time)}") + + dataspace.vulnerabilities_updated_at = timezone.now() + dataspace.save(update_fields=["vulnerabilities_updated_at"]) + + +def fetch_for_queryset(queryset, dataspace, batch_size=50, timeout=None, log_func=None): + object_count = queryset.count() + if object_count < 1: + return + + vulnerablecode = VulnerableCode(dataspace) + vulnerability_qs = Vulnerability.objects.scope(dataspace) + created_vulnerabilities = 0 + + for index, batch in enumerate(chunked_queryset(queryset, batch_size), start=1): + if log_func: + progress_count = index * batch_size + if progress_count > object_count: + progress_count = object_count + log_func(f"Progress: {intcomma(progress_count)}/{intcomma(object_count)}") + + vc_entries = vulnerablecode.get_vulnerable_purls(batch, purl_only=False, timeout=timeout) + for vc_entry in vc_entries: + affected_by_vulnerabilities = vc_entry.get("affected_by_vulnerabilities") + if not affected_by_vulnerabilities: + continue + + affected_packages = queryset.filter( + type=vc_entry.get("type"), + namespace=vc_entry.get("namespace") or "", + name=vc_entry.get("name"), + version=vc_entry.get("version") or "", + ) + if not affected_packages: + raise CommandError("Could not find package!") + + for vulnerability_data in affected_by_vulnerabilities: + vulnerability_id = vulnerability_data["vulnerability_id"] + vulnerability = vulnerability_qs.get_or_none(vulnerability_id=vulnerability_id) + if not vulnerability: + vulnerability = Vulnerability.create_from_data( + dataspace=dataspace, + data=vulnerability_data, + ) + created_vulnerabilities += 1 + vulnerability.add_affected_packages(affected_packages) + + return created_vulnerabilities diff --git a/component_catalog/widgets.py b/component_catalog/widgets.py index 31345002..a2d67506 100644 --- a/component_catalog/widgets.py +++ b/component_catalog/widgets.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode/__init__.py b/dejacode/__init__.py index b388da21..68ca48b5 100644 --- a/dejacode/__init__.py +++ b/dejacode/__init__.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -70,6 +70,7 @@ def extract_short_commit(git_describe): # Turn off the warnings for the following modules. warnings.filterwarnings("ignore", module="cyclonedx") +warnings.filterwarnings("ignore", category=FutureWarning, module="rq_scheduler.utils") def command_line(): diff --git a/dejacode/formats/__init__.py b/dejacode/formats/__init__.py index 7aa35de2..f61c4aae 100644 --- a/dejacode/formats/__init__.py +++ b/dejacode/formats/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode/formats/cu/__init__.py b/dejacode/formats/cu/__init__.py index 7aa35de2..f61c4aae 100644 --- a/dejacode/formats/cu/__init__.py +++ b/dejacode/formats/cu/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode/formats/cu/formats.py b/dejacode/formats/cu/formats.py index 797a19a0..90ca1a4b 100644 --- a/dejacode/formats/cu/formats.py +++ b/dejacode/formats/cu/formats.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode/formats/en/__init__.py b/dejacode/formats/en/__init__.py index 7aa35de2..f61c4aae 100644 --- a/dejacode/formats/en/__init__.py +++ b/dejacode/formats/en/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode/formats/en/formats.py b/dejacode/formats/en/formats.py index 797a19a0..90ca1a4b 100644 --- a/dejacode/formats/en/formats.py +++ b/dejacode/formats/en/formats.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode/localization/en/LC_MESSAGES/django.po b/dejacode/localization/en/LC_MESSAGES/django.po index 5fcc3e39..07b8e753 100644 --- a/dejacode/localization/en/LC_MESSAGES/django.po +++ b/dejacode/localization/en/LC_MESSAGES/django.po @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode/localization/en_GB/LC_MESSAGES/django.po b/dejacode/localization/en_GB/LC_MESSAGES/django.po index 707e4181..51fa6a4d 100644 --- a/dejacode/localization/en_GB/LC_MESSAGES/django.po +++ b/dejacode/localization/en_GB/LC_MESSAGES/django.po @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode/settings.py b/dejacode/settings.py index 53f1bd80..953fcc25 100644 --- a/dejacode/settings.py +++ b/dejacode/settings.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -441,10 +441,14 @@ def gettext_noop(s): "HOST": env.str("DEJACODE_REDIS_HOST", default="localhost"), "PORT": env.str("DEJACODE_REDIS_PORT", default="6379"), "PASSWORD": env.str("DEJACODE_REDIS_PASSWORD", default=""), - "DEFAULT_TIMEOUT": env.int("DEJACODE_REDIS_DEFAULT_TIMEOUT", default=360), + "DEFAULT_TIMEOUT": env.int("DEJACODE_RQ_DEFAULT_TIMEOUT", default=360), }, } +# Cron jobs (scheduler) +daily_at_3am = "0 3 * * *" +DEJACODE_VULNERABILITIES_CRON = env.str("DEJACODE_VULNERABILITIES_CRON", default=daily_at_3am) + def enable_rq_eager_mode(): """ diff --git a/dejacode/static/css/dejacode_admin.css b/dejacode/static/css/dejacode_admin.css index f97b9a0a..8b59ac50 100644 --- a/dejacode/static/css/dejacode_admin.css +++ b/dejacode/static/css/dejacode_admin.css @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ diff --git a/dejacode/static/css/dejacode_bootstrap.css b/dejacode/static/css/dejacode_bootstrap.css index 07d374cc..26bf7b52 100644 --- a/dejacode/static/css/dejacode_bootstrap.css +++ b/dejacode/static/css/dejacode_bootstrap.css @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ @@ -64,6 +64,9 @@ a.dropdown-item:hover { .smaller { font-size: 90%; } +.mini { + font-size: .675em; +} code { color: #ac1459; } diff --git a/dejacode/static/js/awesomplete_fields.js b/dejacode/static/js/awesomplete_fields.js index b4a611fe..31d197b3 100644 --- a/dejacode/static/js/awesomplete_fields.js +++ b/dejacode/static/js/awesomplete_fields.js @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ diff --git a/dejacode/static/js/dejacode_main.js b/dejacode/static/js/dejacode_main.js index 1fbfd430..fc065fb0 100644 --- a/dejacode/static/js/dejacode_main.js +++ b/dejacode/static/js/dejacode_main.js @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ diff --git a/dejacode/static/js/edit-autocomplete-link.js b/dejacode/static/js/edit-autocomplete-link.js index ebeef9b5..d5ce02cc 100644 --- a/dejacode/static/js/edit-autocomplete-link.js +++ b/dejacode/static/js/edit-autocomplete-link.js @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ diff --git a/dejacode/static/js/license_expression_builder.js b/dejacode/static/js/license_expression_builder.js index 68246650..030babbe 100644 --- a/dejacode/static/js/license_expression_builder.js +++ b/dejacode/static/js/license_expression_builder.js @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ diff --git a/dejacode/static/js/nexb.js b/dejacode/static/js/nexb.js index 830c94f0..6e814914 100644 --- a/dejacode/static/js/nexb.js +++ b/dejacode/static/js/nexb.js @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ diff --git a/dejacode/static/js/scroll_to.js b/dejacode/static/js/scroll_to.js index 5bbaebd9..5d27c5af 100644 --- a/dejacode/static/js/scroll_to.js +++ b/dejacode/static/js/scroll_to.js @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ diff --git a/dejacode/static/js/widget_autocomplete.js b/dejacode/static/js/widget_autocomplete.js index eee67b94..18383802 100644 --- a/dejacode/static/js/widget_autocomplete.js +++ b/dejacode/static/js/widget_autocomplete.js @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ diff --git a/dejacode/urls.py b/dejacode/urls.py index 13ab6bc4..8bfabfe8 100644 --- a/dejacode/urls.py +++ b/dejacode/urls.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode/wsgi.py b/dejacode/wsgi.py index 0821b82b..d2043cb2 100644 --- a/dejacode/wsgi.py +++ b/dejacode/wsgi.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode_toolkit/__init__.py b/dejacode_toolkit/__init__.py index 0a6e2122..6a84f76e 100644 --- a/dejacode_toolkit/__init__.py +++ b/dejacode_toolkit/__init__.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -47,6 +47,7 @@ def __init__(self, dataspace): if not dataspace: raise ValueError("Dataspace must be provided.") + self.dataspace = dataspace self.service_url = None self.service_api_key = None self.basic_auth_user = None diff --git a/dejacode_toolkit/download.py b/dejacode_toolkit/download.py index 35331db7..3e31cb6d 100644 --- a/dejacode_toolkit/download.py +++ b/dejacode_toolkit/download.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode_toolkit/purldb.py b/dejacode_toolkit/purldb.py index e7b62255..c29e2a28 100644 --- a/dejacode_toolkit/purldb.py +++ b/dejacode_toolkit/purldb.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode_toolkit/scancodeio.py b/dejacode_toolkit/scancodeio.py index 5ce28dac..cd0ecc9f 100644 --- a/dejacode_toolkit/scancodeio.py +++ b/dejacode_toolkit/scancodeio.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode_toolkit/spdx.py b/dejacode_toolkit/spdx.py index 531917fd..63c8adea 100644 --- a/dejacode_toolkit/spdx.py +++ b/dejacode_toolkit/spdx.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode_toolkit/tests/__init__.py b/dejacode_toolkit/tests/__init__.py index 7aa35de2..f61c4aae 100644 --- a/dejacode_toolkit/tests/__init__.py +++ b/dejacode_toolkit/tests/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode_toolkit/tests/test_spdx.py b/dejacode_toolkit/tests/test_spdx.py index 904ce182..3761f090 100644 --- a/dejacode_toolkit/tests/test_spdx.py +++ b/dejacode_toolkit/tests/test_spdx.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode_toolkit/utils.py b/dejacode_toolkit/utils.py index 30517103..7630dcb4 100644 --- a/dejacode_toolkit/utils.py +++ b/dejacode_toolkit/utils.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dejacode_toolkit/vulnerablecode.py b/dejacode_toolkit/vulnerablecode.py index 7cefb243..fb70ba12 100644 --- a/dejacode_toolkit/vulnerablecode.py +++ b/dejacode_toolkit/vulnerablecode.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -69,6 +69,7 @@ def get_vulnerabilities_by_cpe( def bulk_search_by_purl( self, purls, + purl_only, timeout=None, ): """Bulk search of vulnerabilities using the provided list of `purls`.""" @@ -76,7 +77,7 @@ def bulk_search_by_purl( data = { "purls": purls, - "purl_only": True, + "purl_only": purl_only, "plain_purl": True, } @@ -98,7 +99,7 @@ def bulk_search_by_cpes( logger.debug(f"VulnerableCode: url={url} cpes_count={len(cpes)}") return self.request_post(url, json=data, timeout=timeout) - def get_vulnerable_purls(self, packages): + def get_vulnerable_purls(self, packages, purl_only=True, timeout=10): """ Return a list of PURLs for which at least one `affected_by_vulnerabilities` was found in the VulnerableCodeDB for the given list of `packages`. @@ -108,7 +109,11 @@ def get_vulnerable_purls(self, packages): if not plain_purls: return [] - vulnerable_purls = self.bulk_search_by_purl(plain_purls, timeout=5) + vulnerable_purls = self.bulk_search_by_purl( + plain_purls, + purl_only=purl_only, + timeout=timeout, + ) return vulnerable_purls or [] def get_vulnerable_cpes(self, components): @@ -134,6 +139,34 @@ def get_vulnerable_cpes(self, components): return list(set(vulnerable_cpes)) + def get_package_url_available_types(self): + # Replace by fetching the endpoint once available. + # https://github.com/aboutcode-org/vulnerablecode/issues/1561#issuecomment-2298764730 + return [ + "alpine", + "alpm", + "apache", + "cargo", + "composer", + "conan", + "deb", + "gem", + "generic", + "github", + "golang", + "hex", + "mattermost", + "maven", + "mozilla", + "nginx", + "npm", + "nuget", + "openssl", + "pypi", + "rpm", + "ruby", + ] + def get_plain_purl(purl_str): """Remove the PURL qualifiers and subpath from the search lookups.""" diff --git a/dje/__init__.py b/dje/__init__.py index 7aa35de2..f61c4aae 100644 --- a/dje/__init__.py +++ b/dje/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/admin.py b/dje/admin.py index 254c9535..7b5dcefc 100644 --- a/dje/admin.py +++ b/dje/admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/api.py b/dje/api.py index 0a5a9a04..b5b959f7 100644 --- a/dje/api.py +++ b/dje/api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/api_custom.py b/dje/api_custom.py index 8bf84790..0efa759d 100644 --- a/dje/api_custom.py +++ b/dje/api_custom.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/apps.py b/dje/apps.py index 0a20fa47..49e20fbd 100644 --- a/dje/apps.py +++ b/dje/apps.py @@ -2,21 +2,34 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # +import logging + from django.apps import AppConfig from django.utils.translation import gettext_lazy as _ import yaml +logger = logging.getLogger(__name__) + class DejaCodeConfig(AppConfig): name = "dje" verbose_name = _("DejaCode") def ready(self): + """ + Get the apps ready when Django starts. + + WARNING: The ready() method gets triggered multiple times and is not + suitable to handle execution that should only happen once. + - Gunicorn workers: When Gunicorn starts, it spawns multiple worker processes, + each of which calls the ready() method. + - Worker services: Similarly, the workers also trigger the ready() method. + """ from dje.mass_update import action_end from dje.notification import successful_mass_update diff --git a/dje/client_data.py b/dje/client_data.py index 37b920bd..4438534d 100644 --- a/dje/client_data.py +++ b/dje/client_data.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/context_processors.py b/dje/context_processors.py index 67d7bb05..3ee98911 100644 --- a/dje/context_processors.py +++ b/dje/context_processors.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/copier.py b/dje/copier.py index 33f0d13a..758c23cb 100644 --- a/dje/copier.py +++ b/dje/copier.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/dashboard.py b/dje/dashboard.py index b63ca6ae..db887008 100644 --- a/dje/dashboard.py +++ b/dje/dashboard.py @@ -2,10 +2,11 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # +from django.contrib.humanize.templatetags.humanize import naturaltime from django.urls import reverse from django.utils.translation import gettext_lazy as _ @@ -16,6 +17,7 @@ class DejaCodeDashboard(Dashboard): def init_with_context(self, context): user = context["request"].user + dataspace = user.dataspace self.children.append( modules.ModelList( @@ -33,7 +35,7 @@ def init_with_context(self, context): ) administration_models = ["dje.*"] - if user.dataspace.is_reference: + if dataspace.is_reference: administration_models.extend( [ "django.contrib.*", @@ -189,3 +191,23 @@ def init_with_context(self, context): children=imports, ) ) + + data_updates = [] + if dataspace.enable_vulnerablecodedb_access: + updated_at = dataspace.vulnerabilities_updated_at + data_updates.append( + { + "title": _(f"Vulnerabilities: {naturaltime(updated_at)}"), + "description": updated_at, + } + ) + + if data_updates: + self.children.append( + modules.LinkList( + _("Data updates"), + collapsible=True, + column=3, + children=data_updates, + ) + ) diff --git a/dje/decorators.py b/dje/decorators.py index 3ff6ebb9..3639e28c 100644 --- a/dje/decorators.py +++ b/dje/decorators.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/fields.py b/dje/fields.py index fcd6e232..bc88c2c1 100644 --- a/dje/fields.py +++ b/dje/fields.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/filters.py b/dje/filters.py index 43839925..85bbb3ce 100644 --- a/dje/filters.py +++ b/dje/filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -844,16 +844,19 @@ class HasRelationFilter(django_filters.ChoiceFilter): def __init__(self, *args, **kwargs): kwargs["lookup_expr"] = "isnull" kwargs["empty_label"] = "Any" - kwargs["choices"] = ( - ("with", _("With")), - ("without", _("Without")), + kwargs.setdefault( + "choices", + ( + ("with", _("With")), + ("without", _("Without")), + ), ) super().__init__(*args, **kwargs) def filter(self, qs, value): - if value == "with": + if value in ["with", "yes"]: return qs.filter(**{f"{self.field_name}__{self.lookup_expr}": False}).distinct() - elif value == "without": + elif value in ["without", "no"]: return qs.filter(**{f"{self.field_name}__{self.lookup_expr}": True}).distinct() return qs @@ -865,8 +868,8 @@ class HasCountFilter(HasRelationFilter): """ def filter(self, qs, value): - if value == "with": + if value in ["with", "yes"]: return qs.filter(**{f"{self.field_name}_count__gt": 0}).distinct() - elif value == "without": + elif value in ["without", "no"]: return qs.filter(**{f"{self.field_name}_count__exact": 0}).distinct() return qs diff --git a/dje/forms.py b/dje/forms.py index 1ca06a6a..0694cf09 100644 --- a/dje/forms.py +++ b/dje/forms.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/importers.py b/dje/importers.py index 97a4a574..da87f1ba 100644 --- a/dje/importers.py +++ b/dje/importers.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/ldap_backend.py b/dje/ldap_backend.py index c4a0fb8c..c4354312 100644 --- a/dje/ldap_backend.py +++ b/dje/ldap_backend.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/list_display.py b/dje/list_display.py index 81dbabf5..3d23a0ef 100644 --- a/dje/list_display.py +++ b/dje/list_display.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/__init__.py b/dje/management/__init__.py index e75433f0..886b021f 100644 --- a/dje/management/__init__.py +++ b/dje/management/__init__.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/commands/__init__.py b/dje/management/commands/__init__.py index 4a3466f9..000be1ad 100644 --- a/dje/management/commands/__init__.py +++ b/dje/management/commands/__init__.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/commands/checkdata.py b/dje/management/commands/checkdata.py index 4de45e7e..7d812f9c 100644 --- a/dje/management/commands/checkdata.py +++ b/dje/management/commands/checkdata.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/commands/checkmigrations.py b/dje/management/commands/checkmigrations.py index 46faa53b..b3392560 100644 --- a/dje/management/commands/checkmigrations.py +++ b/dje/management/commands/checkmigrations.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/commands/checks.py b/dje/management/commands/checks.py index 8d355344..d6fa78b3 100644 --- a/dje/management/commands/checks.py +++ b/dje/management/commands/checks.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/commands/clonedataset.py b/dje/management/commands/clonedataset.py index b8e06573..26263cd2 100644 --- a/dje/management/commands/clonedataset.py +++ b/dje/management/commands/clonedataset.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/commands/createhistory.py b/dje/management/commands/createhistory.py index e9963b43..a6fa9af0 100644 --- a/dje/management/commands/createhistory.py +++ b/dje/management/commands/createhistory.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/commands/dumpdataset.py b/dje/management/commands/dumpdataset.py index d4cd4448..5b693da6 100644 --- a/dje/management/commands/dumpdataset.py +++ b/dje/management/commands/dumpdataset.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/commands/dumpinitdata.py b/dje/management/commands/dumpinitdata.py index c9199305..0815b9dd 100644 --- a/dje/management/commands/dumpinitdata.py +++ b/dje/management/commands/dumpinitdata.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/commands/flushdataset.py b/dje/management/commands/flushdataset.py index 4ccf36de..b8193060 100644 --- a/dje/management/commands/flushdataset.py +++ b/dje/management/commands/flushdataset.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/management/commands/setupcron.py b/dje/management/commands/setupcron.py new file mode 100644 index 00000000..5b7d2b55 --- /dev/null +++ b/dje/management/commands/setupcron.py @@ -0,0 +1,71 @@ +# +# Copyright (c) nexB Inc. and others. All rights reserved. +# DejaCode is a trademark of nexB Inc. +# SPDX-License-Identifier: AGPL-3.0-only +# See https://github.com/aboutcode-org/dejacode for support or download. +# See https://aboutcode.org for more information about AboutCode FOSS projects. +# + +import sys + +from django.conf import settings +from django.contrib.humanize.templatetags.humanize import naturaltime +from django.core.management.base import BaseCommand + +import django_rq + +from dje.tasks import update_vulnerabilities + +# **Regular RQ jobs** are tasks that run once when you enqueue them. +# **Scheduled RQ jobs** are tasks that are set to run automatically at specific times +# or intervals, like every day or every hour. +# The scheduler keeps track of these jobs and enqueues them when it's time for them to +# run. + + +class Command(BaseCommand): + help = ( + "Sets up the application's scheduled cron jobs. " + "Cron jobs are tasks that run automatically at specified intervals." + ) + + def handle(self, *args, **kwargs): + if not settings.DEJACODE_ASYNC: + self.stdout.write("SYNC mode detected, skipping cron job setup.") + sys.exit(0) + + scheduler = django_rq.get_scheduler("default") + + # Cancel all existing cron jobs in the scheduler. + # This ensures that the cron entries are always up-to-date in case their + # configuration has changed. It also prevents any duplicate or orphaned jobs + # from remaining in the scheduler, maintaining a clean and accurate schedule. + cancel_all_scheduled_jobs(scheduler) + + self.stdout.write("Schedule vulnerabilities update") + forever = None + scheduler.cron( + cron_string=settings.DEJACODE_VULNERABILITIES_CRON, # 3am daily by default + func=update_vulnerabilities, + result_ttl=300, + repeat=forever, + timeout="3h", + use_local_timezone=True, + ) + + self.stdout.write(self.style.SUCCESS("Successfully set up cron jobs.")) + self.stdout.write("Scheduled jobs next execution:") + for job, scheduled_time in scheduler.get_jobs(with_times=True): + msg = f" > {job.description} in {naturaltime(scheduled_time)} ({scheduled_time})" + self.stdout.write(msg) + + +def cancel_all_scheduled_jobs(scheduler): + """ + Cancel all scheduled jobs in the given scheduler. + + This function iterates over all jobs currently scheduled in the scheduler + and cancels each one, effectively clearing the schedule. + """ + for job in scheduler.get_jobs(): + scheduler.cancel(job) diff --git a/dje/mass_update.py b/dje/mass_update.py index f9b37ff4..22ca93f0 100644 --- a/dje/mass_update.py +++ b/dje/mass_update.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/middleware.py b/dje/middleware.py index 45b466fd..932f7d4e 100644 --- a/dje/middleware.py +++ b/dje/middleware.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/migrations/0004_dataspace_vulnerabilities_updated_at.py b/dje/migrations/0004_dataspace_vulnerabilities_updated_at.py new file mode 100644 index 00000000..0739d1c9 --- /dev/null +++ b/dje/migrations/0004_dataspace_vulnerabilities_updated_at.py @@ -0,0 +1,18 @@ +# Generated by Django 5.0.6 on 2024-08-12 14:07 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('dje', '0003_alter_dejacodeuser_homepage_layout'), + ] + + operations = [ + migrations.AddField( + model_name='dataspace', + name='vulnerabilities_updated_at', + field=models.DateTimeField(blank=True, help_text='The date and time when the local vulnerability database was last updated. ', null=True, verbose_name='Last vulnerability data update'), + ), + ] diff --git a/dje/models.py b/dje/models.py index 5bda6463..6d4fa452 100644 --- a/dje/models.py +++ b/dje/models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -345,6 +345,14 @@ class Dataspace(models.Model): ), ) + vulnerabilities_updated_at = models.DateTimeField( + _("Last vulnerability data update"), + auto_now=False, + null=True, + blank=True, + help_text=_("The date and time when the local vulnerability database was last updated. "), + ) + objects = DataspaceManager() class Meta: @@ -743,9 +751,19 @@ def create_from_data(cls, user, data, validate=False): if field_name in model_fields and value not in EMPTY_VALUES } + if isinstance(user, DejacodeUser): + initial_values = { + "dataspace": user.dataspace, + "created_by": user, + } + # Support for providing a Dataspace directly in place of a User + elif isinstance(user, Dataspace): + initial_values = { + "dataspace": user, + } + instance = cls( - dataspace=user.dataspace, - created_by=user, + **initial_values, **cleaned_data, ) @@ -802,7 +820,7 @@ def as_json(self): use_natural_foreign_keys=True, use_natural_primary_keys=True, ) - except SerializationError: + except (SerializationError, ValueError): serialized_data = None return serialized_data diff --git a/dje/notification.py b/dje/notification.py index 8088a584..4371ca4a 100644 --- a/dje/notification.py +++ b/dje/notification.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/outputs.py b/dje/outputs.py index a68f4515..cb21d5ff 100644 --- a/dje/outputs.py +++ b/dje/outputs.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/permissions.py b/dje/permissions.py index a3398052..2b9ed498 100644 --- a/dje/permissions.py +++ b/dje/permissions.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/registration.py b/dje/registration.py index dbee55d3..a2c5cc4b 100644 --- a/dje/registration.py +++ b/dje/registration.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/search.py b/dje/search.py index be1169c7..2de73d0e 100644 --- a/dje/search.py +++ b/dje/search.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tasks.py b/dje/tasks.py index 344e894a..4da86604 100644 --- a/dje/tasks.py +++ b/dje/tasks.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -272,3 +272,17 @@ def improve_packages_from_purldb(product_uuid, user_uuid): recipient=user, description=description, ) + + +@job("default", timeout="3h") +def update_vulnerabilities(): + """Fetch vulnerabilities for all Dataspaces that enable vulnerablecodedb access.""" + from component_catalog.vulnerabilities import fetch_from_vulnerablecode + + logger.info("Entering update_vulnerabilities task") + Dataspace = apps.get_model("dje", "Dataspace") + dataspace_qs = Dataspace.objects.filter(enable_vulnerablecodedb_access=True) + + for dataspace in dataspace_qs: + logger.info(f"fetch_vulnerabilities for datapsace={dataspace}") + fetch_from_vulnerablecode(dataspace, batch_size=50, timeout=60) diff --git a/dje/templates/admin/object_import.html b/dje/templates/admin/object_import.html index 7e3c3ddf..acffb7cc 100644 --- a/dje/templates/admin/object_import.html +++ b/dje/templates/admin/object_import.html @@ -127,7 +127,7 @@
    Help for {{ importer.verbose_name|title }} Import:
    {% if add_to_product_form %}
    - diff --git a/dje/templates/grappelli/dashboard/modules/link_list.html b/dje/templates/grappelli/dashboard/modules/link_list.html new file mode 100644 index 00000000..ccab8bbb --- /dev/null +++ b/dje/templates/grappelli/dashboard/modules/link_list.html @@ -0,0 +1,20 @@ +{% extends "grappelli/dashboard/module.html" %} +{% load i18n %} +{% block module_content %} +
      + {% spaceless %} + {% for child in module.children %} +
    • + {% if child.url %} + {{ child.title }} + {% else %} + {# Added: ability to provide non-link entry #} + + {{ child.title }} + + {% endif %} +
    • + {% endfor %} + {% endspaceless %} +
    +{% endblock %} diff --git a/dje/templates/includes/footer.html b/dje/templates/includes/footer.html index 1ab53da7..1df5875d 100644 --- a/dje/templates/includes/footer.html +++ b/dje/templates/includes/footer.html @@ -6,8 +6,8 @@
  • {% trans 'Documentation' %}
  • {% trans 'Support' %}
  • {% trans 'About' %}
  • -
  • {% trans 'Source Code' %}
  • -
  • {% trans 'Releases' %}
  • +
  • {% trans 'Source Code' %}
  • +
  • {% trans 'Releases' %}
  • {% trans 'License' %}
  • Copyright nexB Inc. DejaCode is a trademark of nexB Inc.

    diff --git a/dje/templates/includes/header.html b/dje/templates/includes/header.html index dbaca667..553e2fbd 100644 --- a/dje/templates/includes/header.html +++ b/dje/templates/includes/header.html @@ -13,8 +13,8 @@ {% trans 'Support' %} {% trans 'About' %} - {% trans 'Source Code' %} - {% trans 'Releases' %} + {% trans 'Source Code' %} + {% trans 'Releases' %} {% trans 'License' %} diff --git a/dje/templates/integrations_status.html b/dje/templates/integrations_status.html index 047963d0..94f71b9d 100644 --- a/dje/templates/integrations_status.html +++ b/dje/templates/integrations_status.html @@ -36,6 +36,13 @@
    {{ label }}
    {% else %}
    Not configured
    {% endif %} + {% if status.extra_info %} +
    +
    + + {{ status.extra_info }} +
    + {% endif %}
    diff --git a/dje/templatetags/__init__.py b/dje/templatetags/__init__.py index 7aa35de2..f61c4aae 100644 --- a/dje/templatetags/__init__.py +++ b/dje/templatetags/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/templatetags/dje_tags.py b/dje/templatetags/dje_tags.py index a5bd61d8..1df93201 100644 --- a/dje/templatetags/dje_tags.py +++ b/dje/templatetags/dje_tags.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/__init__.py b/dje/tests/__init__.py index 64f249cf..6594511b 100644 --- a/dje/tests/__init__.py +++ b/dje/tests/__init__.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_access.py b/dje/tests/test_access.py index f440c759..a99a2223 100644 --- a/dje/tests/test_access.py +++ b/dje/tests/test_access.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_admin.py b/dje/tests/test_admin.py index 1b802a76..d539b95b 100644 --- a/dje/tests/test_admin.py +++ b/dje/tests/test_admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_api.py b/dje/tests/test_api.py index bd83ca80..2d0ba725 100644 --- a/dje/tests/test_api.py +++ b/dje/tests/test_api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -523,7 +523,7 @@ def test_api_external_reference_endpoint_create(self): "content_object": owner_detail_url, "external_source": self.ext_source1.label, "external_id": "dejacode", - "external_url": "https://github.com/nexB/dejacode", + "external_url": "https://github.com/aboutcode-org/dejacode", } response = self.client.post(self.external_reference_list_url, data) @@ -534,7 +534,7 @@ def test_api_external_reference_endpoint_create(self): self.assertEqual(new_ext_ref.content_type, ContentType.objects.get_for_model(new_owner)) self.assertEqual(new_ext_ref.object_id, new_owner.id) self.assertEqual(new_ext_ref.dataspace, self.dataspace) - self.assertEqual(new_ext_ref.external_url, "https://github.com/nexB/dejacode") + self.assertEqual(new_ext_ref.external_url, "https://github.com/aboutcode-org/dejacode") self.assertEqual(new_ext_ref.external_id, "dejacode") # Non-supported object_type diff --git a/dje/tests/test_command.py b/dje/tests/test_command.py index 14c2763d..96d85869 100644 --- a/dje/tests/test_command.py +++ b/dje/tests/test_command.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_external_reference.py b/dje/tests/test_external_reference.py index 22f5a284..0b7a030c 100644 --- a/dje/tests/test_external_reference.py +++ b/dje/tests/test_external_reference.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_fields.py b/dje/tests/test_fields.py index 3a5c6766..96c5e747 100644 --- a/dje/tests/test_fields.py +++ b/dje/tests/test_fields.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_forms.py b/dje/tests/test_forms.py index add6d9f3..668c90ca 100644 --- a/dje/tests/test_forms.py +++ b/dje/tests/test_forms.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -111,7 +111,7 @@ def test_tabs_permission_formset_load_perms(self): def test_copy_defaults_form_get_all_dataspaced_models(self): dataspaced_models = CopyDefaultsForm.get_all_dataspaced_models() self.assertEqual(9, len(dataspaced_models)) - self.assertEqual(7, len(dataspaced_models.get("Component Catalog"))) + self.assertEqual(8, len(dataspaced_models.get("Component Catalog"))) self.assertIn("Subcomponent", str(dataspaced_models.get("Component Catalog"))) def test_copy_defaults_formset_serialize_perms(self): diff --git a/dje/tests/test_history.py b/dje/tests/test_history.py index d0feffc1..b9182bd2 100644 --- a/dje/tests/test_history.py +++ b/dje/tests/test_history.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_html_validation.py b/dje/tests/test_html_validation.py index 22a08c2c..09f1caf2 100644 --- a/dje/tests/test_html_validation.py +++ b/dje/tests/test_html_validation.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_ldap.py b/dje/tests/test_ldap.py index 0b12317b..f7a17716 100644 --- a/dje/tests/test_ldap.py +++ b/dje/tests/test_ldap.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_models.py b/dje/tests/test_models.py index 2f3cbc41..a8950e57 100644 --- a/dje/tests/test_models.py +++ b/dje/tests/test_models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_notification.py b/dje/tests/test_notification.py index e351e010..85ef43f2 100644 --- a/dje/tests/test_notification.py +++ b/dje/tests/test_notification.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_outputs.py b/dje/tests/test_outputs.py index 2bf73628..29694881 100644 --- a/dje/tests/test_outputs.py +++ b/dje/tests/test_outputs.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_permissions.py b/dje/tests/test_permissions.py index a3c7237c..b0a8b1ca 100644 --- a/dje/tests/test_permissions.py +++ b/dje/tests/test_permissions.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_registration.py b/dje/tests/test_registration.py index 6f38278b..3f1e2f9d 100644 --- a/dje/tests/test_registration.py +++ b/dje/tests/test_registration.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_two_factor.py b/dje/tests/test_two_factor.py index 85f5c3c9..52e679dc 100644 --- a/dje/tests/test_two_factor.py +++ b/dje/tests/test_two_factor.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_urn.py b/dje/tests/test_urn.py index a150b2ef..01c4de29 100644 --- a/dje/tests/test_urn.py +++ b/dje/tests/test_urn.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_user.py b/dje/tests/test_user.py index 4fe66020..622b54ff 100644 --- a/dje/tests/test_user.py +++ b/dje/tests/test_user.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_utils.py b/dje/tests/test_utils.py index e63675d4..e3f6f982 100644 --- a/dje/tests/test_utils.py +++ b/dje/tests/test_utils.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_validators.py b/dje/tests/test_validators.py index 846e4c7b..a62e9fc1 100644 --- a/dje/tests/test_validators.py +++ b/dje/tests/test_validators.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/test_views.py b/dje/tests/test_views.py index 20671ac1..2e229134 100644 --- a/dje/tests/test_views.py +++ b/dje/tests/test_views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/tests/testfiles/test_dataset_cc_only.json b/dje/tests/testfiles/test_dataset_cc_only.json index cb14fe0a..63648859 100644 --- a/dje/tests/testfiles/test_dataset_cc_only.json +++ b/dje/tests/testfiles/test_dataset_cc_only.json @@ -100,7 +100,8 @@ "acceptable_linkages": null, "export_restrictions": "", "approved_download_location": "", - "approved_community_interaction": "" + "approved_community_interaction": "", + "affected_by_vulnerabilities": [] } }, { @@ -170,7 +171,8 @@ "acceptable_linkages": null, "export_restrictions": "", "approved_download_location": "", - "approved_community_interaction": "" + "approved_community_interaction": "", + "affected_by_vulnerabilities": [] } }, { @@ -311,7 +313,8 @@ "api_data_url": "", "datasource_id": "", "file_references": [], - "parties": [] + "parties": [], + "affected_by_vulnerabilities": [] } }, { diff --git a/dje/tests/testfiles/test_dataset_pp_only.json b/dje/tests/testfiles/test_dataset_pp_only.json index fab48776..a71b26c2 100644 --- a/dje/tests/testfiles/test_dataset_pp_only.json +++ b/dje/tests/testfiles/test_dataset_pp_only.json @@ -47,7 +47,8 @@ "api_data_url": "", "datasource_id": "", "file_references": [], - "parties": [] + "parties": [], + "affected_by_vulnerabilities": [] } }, { diff --git a/dje/tests/testfiles/test_dataset_user_only.json b/dje/tests/testfiles/test_dataset_user_only.json index 4dbae4c3..555ab40c 100644 --- a/dje/tests/testfiles/test_dataset_user_only.json +++ b/dje/tests/testfiles/test_dataset_user_only.json @@ -23,7 +23,8 @@ "enable_package_scanning": false, "update_packages_from_scan": false, "enable_purldb_access": false, - "enable_vulnerablecodedb_access": false + "enable_vulnerablecodedb_access": false, + "vulnerabilities_updated_at": null } }, { diff --git a/dje/tests/tests.py b/dje/tests/tests.py index 57693ad8..6247cde9 100644 --- a/dje/tests/tests.py +++ b/dje/tests/tests.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/two_factor.py b/dje/two_factor.py index ef4d189c..11cbdb8a 100644 --- a/dje/two_factor.py +++ b/dje/two_factor.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/urn.py b/dje/urn.py index ac178ce4..bf514b74 100644 --- a/dje/urn.py +++ b/dje/urn.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/urn_resolver.py b/dje/urn_resolver.py index 40604d45..76ba51b1 100644 --- a/dje/urn_resolver.py +++ b/dje/urn_resolver.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/utils.py b/dje/utils.py index cbf738fd..98bfe3f8 100644 --- a/dje/utils.py +++ b/dje/utils.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -76,6 +76,12 @@ def chunked(iterable, chunk_size): yield iterable[index:end] +def chunked_queryset(queryset, chunk_size): + """Yield chunks of data from the queryset.""" + for start in range(0, queryset.count(), chunk_size): + yield list(queryset[start : start + chunk_size]) + + def extract_name_version(name_version_str): """ Return a name and a version extracted from the following syntax: 'name:version' @@ -642,3 +648,17 @@ def remove_empty_values(input_dict): `0` and `False` values are kept. """ return {key: value for key, value in input_dict.items() if value not in EMPTY_VALUES} + + +def humanize_time(seconds): + """Convert the provided ``seconds`` number into human-readable time.""" + message = f"{seconds:.0f} seconds" + + if seconds > 86400: + message += f" ({seconds / 86400:.1f} days)" + if seconds > 3600: + message += f" ({seconds / 3600:.1f} hours)" + elif seconds > 60: + message += f" ({seconds / 60:.1f} minutes)" + + return message diff --git a/dje/validators.py b/dje/validators.py index f5e4754b..b2ebd827 100644 --- a/dje/validators.py +++ b/dje/validators.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/views.py b/dje/views.py index ab3ad300..e235e6e8 100644 --- a/dje/views.py +++ b/dje/views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -33,6 +33,7 @@ from django.contrib.auth.mixins import UserPassesTestMixin from django.contrib.auth.models import Group from django.contrib.contenttypes.models import ContentType +from django.contrib.humanize.templatetags.humanize import naturaltime from django.contrib.messages.views import SuccessMessageMixin from django.contrib.staticfiles import finders from django.core.exceptions import ObjectDoesNotExist @@ -681,7 +682,7 @@ def home_view(request): } support_urls = { - "Report an issue": "https://github.com/nexB/dejacode/issues/new/", + "Report an issue": "https://github.com/aboutcode-org/dejacode/issues/new/", } sections = { @@ -2308,6 +2309,7 @@ def get_integration_status(self, integration_class): is_configured = False is_available = False error_log = "" + extra_info = None integration = integration_class(dataspace=self.request.user.dataspace) @@ -2318,10 +2320,16 @@ def get_integration_status(self, integration_class): except Exception as exception: error_log = str(exception) + if integration_class is VulnerableCode: + updated_at = self.request.user.dataspace.vulnerabilities_updated_at + + extra_info = f"Vulnerability data update: {naturaltime(updated_at)}" + status = { "is_configured": is_configured, "is_available": is_available, "error_log": error_log, + "extra_info": extra_info, } if self.request.user.is_superuser: @@ -2338,11 +2346,7 @@ def get_context_data(self, **kwargs): for integration_class in self.integrations } - context.update( - { - "integrations_status": integrations_status, - } - ) + context["integrations_status"] = integrations_status return context diff --git a/dje/views_formset.py b/dje/views_formset.py index 5d39a305..9c1ed9ff 100644 --- a/dje/views_formset.py +++ b/dje/views_formset.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/dje/widgets.py b/dje/widgets.py index de74b1f5..f6358089 100644 --- a/dje/widgets.py +++ b/dje/widgets.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -23,15 +23,19 @@ class DropDownWidget(LinkWidget): {menu} """ - def __init__(self, attrs=None, choices=(), anchor=None, right_align=False, label=None): + def __init__( + self, attrs=None, choices=(), anchor=None, right_align=False, label=None, link_content=None + ): self.anchor = anchor self.right_align = right_align self.label = label + self.link_content = link_content super().__init__(attrs, choices) def render(self, name, value, attrs=None, renderer=None, choices=()): @@ -58,6 +62,7 @@ def render(self, name, value, attrs=None, renderer=None, choices=()): menu=menu, active="active" if value else "", label=self.label if self.label else name.title(), + link_content=format_html(self.link_content or ""), ) def render_option(self, name, selected_choices, option_value, option_label): diff --git a/docker-compose.yml b/docker-compose.yml index 820e4bff..916e5943 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -51,7 +51,7 @@ services: build: . # Ensure that potential db migrations run first by waiting until "web" is up command: wait-for-it --strict --timeout=180 web:8000 -- sh -c " - ./manage.py rqworker --verbosity 1" + ./manage.py rqworker-pool --num-workers 2 --verbosity 1" env_file: - docker.env volumes: @@ -64,6 +64,21 @@ services: - db - web + scheduler: + build: . + command: wait-for-it web:8000 -- sh -c " + ./manage.py setupcron && + ./manage.py rqscheduler --interval=600" + env_file: + - docker.env + volumes: + - .env:/opt/dejacode/.env + - /etc/dejacode/:/etc/dejacode/ + depends_on: + - redis + - db + - web + nginx: image: nginx:alpine ports: diff --git a/docs/contributing.rst b/docs/contributing.rst index b36ac948..3e41fed9 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -20,9 +20,9 @@ Do Your Homework ================ Before adding a contribution or create a new issue, take a look at the project’s -`README `_, read through our +`README `_, read through our `documentation `_, -and browse existing `issues `_, +and browse existing `issues `_, to develop some understanding of the project and confirm whether a given issue/feature has previously been discussed. @@ -37,7 +37,7 @@ First Timers You are here to help, but you're a new contributor! No worries, we always welcome newcomer contributors. We maintain some -`good first issues `_ +`good first issues `_ and encourage new contributors to work on those issues for a smooth start. .. warning:: @@ -58,15 +58,15 @@ and encourage new contributors to work on those issues for a smooth start. Report Issues ------------- -- Report a new `bug `_; just remember to be +- Report a new `bug `_; just remember to be as specific as possible. -- Create a `new issue `_ to request a +- Create a `new issue `_ to request a feature, submit a feedback, or ask a question. -- Look into existing `bugs `_, +- Look into existing `bugs `_, try to reproduce the issue on your side, and discuss solutions in the comments. .. note:: - Make sure to check existing `issues `_, + Make sure to check existing `issues `_, to confirm whether a given issue or a question has previously been discussed. Code Contributions @@ -74,7 +74,7 @@ Code Contributions Code is contributed to the codebase using **pull requests**. A pull request should always be attached to an existing issue. -When there is no existing issues, start by `creating one `_ +When there is no existing issues, start by `creating one `_ to discuss potential solutions and implementation details before writing any code. We use several conventions to ensure code quality regarding format, testing, and @@ -100,7 +100,7 @@ attribution. 3. **Commit messages and Developer Certificate of Origin** Follow the instructions at `Writing good Commit Messages `_ - and `check some examples `_. + and `check some examples `_. **You must include a "Signed-off-by" to your commit messages**:: diff --git a/docs/doc_maintenance.rst b/docs/doc_maintenance.rst index 73d8372e..78db031e 100644 --- a/docs/doc_maintenance.rst +++ b/docs/doc_maintenance.rst @@ -22,7 +22,7 @@ To get started, create or identify a working directory on your local machine. Open that directory and execute the following command in a terminal session:: - git clone https://github.com/nexB/dejacode.git + git clone https://github.com/aboutcode-org/dejacode.git That will create a /dejacode directory in your working directory. Now you can install the dependencies in a virtualenv:: diff --git a/docs/installation.rst b/docs/installation.rst index e6245004..f9ac2692 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -47,10 +47,10 @@ create the Docker images and to manage the services (database, cache, webserver) Clone the DejaCode repository ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Open your terminal and clone the `DejaCode repo `_ +Open your terminal and clone the `DejaCode repo `_ with the following command:: - git clone --depth=1 https://github.com/nexB/dejacode.git + git clone --depth=1 https://github.com/aboutcode-org/dejacode.git Build the Docker image ^^^^^^^^^^^^^^^^^^^^^^ @@ -189,9 +189,9 @@ Before you install DejaCode, make sure you have the following prerequisites: Clone and Configure ------------------- -#. Clone the `DejaCode GitHub repository `_:: +#. Clone the `DejaCode GitHub repository `_:: - git clone https://github.com/nexB/dejacode.git && cd dejacode + git clone https://github.com/aboutcode-org/dejacode.git && cd dejacode #. Install the dependencies:: diff --git a/docs/tutorial-1.rst b/docs/tutorial-1.rst index 091bccc9..edbde63c 100755 --- a/docs/tutorial-1.rst +++ b/docs/tutorial-1.rst @@ -27,7 +27,7 @@ Software Bill of Materials (SBOMs). Alternatively, you can conveniently download one of the provided examples from the following -`GitHub repository `_. +`GitHub repository `_. On the Product details page, from the :guilabel:`Scan` dropdown, select :guilabel:`Load Packages from SBOMs`: diff --git a/etc/git_hooks/pre-commit b/etc/git_hooks/pre-commit index 44c66b3c..9f798bd4 100755 --- a/etc/git_hooks/pre-commit +++ b/etc/git_hooks/pre-commit @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/__init__.py b/license_library/__init__.py index 7aa35de2..f61c4aae 100644 --- a/license_library/__init__.py +++ b/license_library/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/admin.py b/license_library/admin.py index 0da612fc..6df8211c 100644 --- a/license_library/admin.py +++ b/license_library/admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/api.py b/license_library/api.py index 6aca9055..7c4c19c7 100644 --- a/license_library/api.py +++ b/license_library/api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/apps.py b/license_library/apps.py index 2f599914..d684b34c 100644 --- a/license_library/apps.py +++ b/license_library/apps.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/filters.py b/license_library/filters.py index 144a61f5..e693ba54 100644 --- a/license_library/filters.py +++ b/license_library/filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/forms.py b/license_library/forms.py index 7a472cbd..c5b5a1b2 100644 --- a/license_library/forms.py +++ b/license_library/forms.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/languages.py b/license_library/languages.py index fb654a48..fc379c30 100644 --- a/license_library/languages.py +++ b/license_library/languages.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/models.py b/license_library/models.py index 8bf0be4b..ae82e058 100644 --- a/license_library/models.py +++ b/license_library/models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/tests/__init__.py b/license_library/tests/__init__.py index 7aa35de2..f61c4aae 100644 --- a/license_library/tests/__init__.py +++ b/license_library/tests/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/tests/test_admin.py b/license_library/tests/test_admin.py index 2317e0dd..28d5cad5 100644 --- a/license_library/tests/test_admin.py +++ b/license_library/tests/test_admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/tests/test_api.py b/license_library/tests/test_api.py index 1cea3221..c287c0cf 100644 --- a/license_library/tests/test_api.py +++ b/license_library/tests/test_api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/tests/test_copy.py b/license_library/tests/test_copy.py index b100009d..e2f74fb6 100644 --- a/license_library/tests/test_copy.py +++ b/license_library/tests/test_copy.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/tests/test_mass_update.py b/license_library/tests/test_mass_update.py index 53e61b5a..2e8d1d5c 100644 --- a/license_library/tests/test_mass_update.py +++ b/license_library/tests/test_mass_update.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/tests/test_models.py b/license_library/tests/test_models.py index 4ae4726e..cdfa91ed 100644 --- a/license_library/tests/test_models.py +++ b/license_library/tests/test_models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/tests/test_search_quality.py b/license_library/tests/test_search_quality.py index 102e1112..cdfe5d4c 100644 --- a/license_library/tests/test_search_quality.py +++ b/license_library/tests/test_search_quality.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/tests/test_translation.py b/license_library/tests/test_translation.py index 955ede8c..8078f0a8 100644 --- a/license_library/tests/test_translation.py +++ b/license_library/tests/test_translation.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/tests/test_urn.py b/license_library/tests/test_urn.py index cdf47e52..036cce35 100644 --- a/license_library/tests/test_urn.py +++ b/license_library/tests/test_urn.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/tests/test_views.py b/license_library/tests/test_views.py index 00b67250..7e7b1791 100644 --- a/license_library/tests/test_views.py +++ b/license_library/tests/test_views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/urls.py b/license_library/urls.py index d1a52e07..7a3c7ddb 100644 --- a/license_library/urls.py +++ b/license_library/urls.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/license_library/views.py b/license_library/views.py index 15f6b1f2..761dcd5c 100644 --- a/license_library/views.py +++ b/license_library/views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/notification/__init__.py b/notification/__init__.py index 7aa35de2..f61c4aae 100644 --- a/notification/__init__.py +++ b/notification/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/notification/admin.py b/notification/admin.py index 8595d09b..c3111c1d 100644 --- a/notification/admin.py +++ b/notification/admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/notification/apps.py b/notification/apps.py index 088cb0dc..2b8b7e94 100644 --- a/notification/apps.py +++ b/notification/apps.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/notification/models.py b/notification/models.py index 615bf759..c0f37be8 100644 --- a/notification/models.py +++ b/notification/models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/notification/tasks.py b/notification/tasks.py index 63e5db9c..f8f443fc 100644 --- a/notification/tasks.py +++ b/notification/tasks.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/notification/tests/test_models.py b/notification/tests/test_models.py index 450e246c..5c63a4b8 100644 --- a/notification/tests/test_models.py +++ b/notification/tests/test_models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/notification/tests/test_tasks.py b/notification/tests/test_tasks.py index 7a8f9c48..e46c1581 100644 --- a/notification/tests/test_tasks.py +++ b/notification/tests/test_tasks.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/__init__.py b/organization/__init__.py index 7aa35de2..f61c4aae 100644 --- a/organization/__init__.py +++ b/organization/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/admin.py b/organization/admin.py index a936f4ad..ee8c9145 100644 --- a/organization/admin.py +++ b/organization/admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/api.py b/organization/api.py index 9f71afb2..63bd12e1 100644 --- a/organization/api.py +++ b/organization/api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/apps.py b/organization/apps.py index faaa1247..4ce99c5d 100644 --- a/organization/apps.py +++ b/organization/apps.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/filters.py b/organization/filters.py index 35e0e325..c18e9428 100644 --- a/organization/filters.py +++ b/organization/filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/forms.py b/organization/forms.py index a775e394..e9f3dac1 100644 --- a/organization/forms.py +++ b/organization/forms.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/importers.py b/organization/importers.py index 90b912c8..fec7ff2d 100644 --- a/organization/importers.py +++ b/organization/importers.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/models.py b/organization/models.py index 86f9a58b..233c97d9 100644 --- a/organization/models.py +++ b/organization/models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/tests/__init__.py b/organization/tests/__init__.py index 7aa35de2..f61c4aae 100644 --- a/organization/tests/__init__.py +++ b/organization/tests/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/tests/test_admin.py b/organization/tests/test_admin.py index dde68237..de565f20 100644 --- a/organization/tests/test_admin.py +++ b/organization/tests/test_admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/tests/test_api.py b/organization/tests/test_api.py index 25769ec6..45fe0b30 100644 --- a/organization/tests/test_api.py +++ b/organization/tests/test_api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/tests/test_filters.py b/organization/tests/test_filters.py index 31b0a5f7..5eefb4d0 100644 --- a/organization/tests/test_filters.py +++ b/organization/tests/test_filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/tests/test_importers.py b/organization/tests/test_importers.py index 4aa96807..f53f8b87 100644 --- a/organization/tests/test_importers.py +++ b/organization/tests/test_importers.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/tests/test_models.py b/organization/tests/test_models.py index 6ac67623..b6d34982 100644 --- a/organization/tests/test_models.py +++ b/organization/tests/test_models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/tests/test_views.py b/organization/tests/test_views.py index d6fb6dd5..4ee02880 100644 --- a/organization/tests/test_views.py +++ b/organization/tests/test_views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/urls.py b/organization/urls.py index 7506013d..f03af930 100644 --- a/organization/urls.py +++ b/organization/urls.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/organization/views.py b/organization/views.py index c73eb027..9d6363f0 100644 --- a/organization/views.py +++ b/organization/views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/policy/__init__.py b/policy/__init__.py index 7aa35de2..f61c4aae 100755 --- a/policy/__init__.py +++ b/policy/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/policy/admin.py b/policy/admin.py index 2b8822b5..214fa1b2 100755 --- a/policy/admin.py +++ b/policy/admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/policy/api.py b/policy/api.py index 41acfe0c..27b9ddb1 100644 --- a/policy/api.py +++ b/policy/api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/policy/apps.py b/policy/apps.py index 5a7f2c75..e67945dc 100755 --- a/policy/apps.py +++ b/policy/apps.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/policy/forms.py b/policy/forms.py index 3df52f60..4049d4b3 100644 --- a/policy/forms.py +++ b/policy/forms.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/policy/models.py b/policy/models.py index e76155cb..e6e33fde 100755 --- a/policy/models.py +++ b/policy/models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/policy/tests/__init__.py b/policy/tests/__init__.py index 7aa35de2..f61c4aae 100644 --- a/policy/tests/__init__.py +++ b/policy/tests/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/policy/tests/test_admin.py b/policy/tests/test_admin.py index 7ec0ecca..0b41f628 100644 --- a/policy/tests/test_admin.py +++ b/policy/tests/test_admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/policy/tests/test_api.py b/policy/tests/test_api.py index 77ad5946..2dee1923 100644 --- a/policy/tests/test_api.py +++ b/policy/tests/test_api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/policy/tests/test_models.py b/policy/tests/test_models.py index 00823f9b..ab265dca 100644 --- a/policy/tests/test_models.py +++ b/policy/tests/test_models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/__init__.py b/product_portfolio/__init__.py index 7aa35de2..f61c4aae 100644 --- a/product_portfolio/__init__.py +++ b/product_portfolio/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/admin.py b/product_portfolio/admin.py index cc98d8f6..15be02cb 100644 --- a/product_portfolio/admin.py +++ b/product_portfolio/admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/api.py b/product_portfolio/api.py index ddee9a06..6d1f9d50 100644 --- a/product_portfolio/api.py +++ b/product_portfolio/api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/apps.py b/product_portfolio/apps.py index 494a4432..6174c626 100644 --- a/product_portfolio/apps.py +++ b/product_portfolio/apps.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/filters.py b/product_portfolio/filters.py index 59d54e69..6bc89b39 100644 --- a/product_portfolio/filters.py +++ b/product_portfolio/filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -200,6 +200,17 @@ class ProductComponentFilterSet(BaseProductRelationFilterSet): "is_modified", ], ) + is_vulnerable = HasRelationFilter( + label=_("Is Vulnerable"), + field_name="component__affected_by_vulnerabilities", + choices=( + ("yes", _("Affected by vulnerabilities")), + ("no", _("No vulnerabilities found")), + ), + widget=DropDownWidget( + anchor="#inventory", right_align=True, link_content='' + ), + ) class Meta: model = ProductComponent @@ -240,6 +251,17 @@ class ProductPackageFilterSet(BaseProductRelationFilterSet): "is_modified", ], ) + is_vulnerable = HasRelationFilter( + label=_("Is Vulnerable"), + field_name="package__affected_by_vulnerabilities", + choices=( + ("yes", _("Affected by vulnerabilities")), + ("no", _("No vulnerabilities found")), + ), + widget=DropDownWidget( + anchor="#inventory", right_align=True, link_content='' + ), + ) @staticmethod def do_nothing(queryset, name, value): diff --git a/product_portfolio/forms.py b/product_portfolio/forms.py index 8ecabbab..cc356f97 100644 --- a/product_portfolio/forms.py +++ b/product_portfolio/forms.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -581,6 +581,10 @@ def save(self, product): raise transaction.savepoint_commit(sid) + + if self.dataspace.enable_vulnerablecodedb_access: + product.fetch_vulnerabilities() + return warnings, created_counts diff --git a/product_portfolio/importers.py b/product_portfolio/importers.py index d2601d25..f402fa42 100644 --- a/product_portfolio/importers.py +++ b/product_portfolio/importers.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -644,6 +644,9 @@ def save(self): if self.scan_all_packages: transaction.on_commit(lambda: self.product.scan_all_packages_task(self.user)) + if self.user.dataspace.enable_vulnerablecodedb_access: + self.product.fetch_vulnerabilities() + return dict(self.created), dict(self.existing), dict(self.errors) def import_packages(self): @@ -655,6 +658,9 @@ def import_dependencies(self): self.import_dependency(dependency_data) def import_package(self, package_data): + # Vulnerabilities are fetched post import. + package_data.pop("affected_by_vulnerabilities", None) + unique_together_lookups = { field: value for field in self.unique_together_fields diff --git a/product_portfolio/inlines.py b/product_portfolio/inlines.py index 27fd68b6..c5f073c4 100644 --- a/product_portfolio/inlines.py +++ b/product_portfolio/inlines.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/models.py b/product_portfolio/models.py index c158e3fd..0ecf124e 100644 --- a/product_portfolio/models.py +++ b/product_portfolio/models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -29,6 +29,7 @@ from component_catalog.models import LicenseExpressionMixin from component_catalog.models import Package from component_catalog.models import component_mixin_factory +from component_catalog.vulnerabilities import fetch_for_queryset from dje import tasks from dje.fields import LastModifiedByField from dje.models import DataspacedManager @@ -502,6 +503,10 @@ def improve_packages_from_purldb(self, user): updated_packages.append(package) return updated_packages + def fetch_vulnerabilities(self): + """Fetch and update the vulnerabilties of all the Package of this Product.""" + return fetch_for_queryset(self.all_packages, self.dataspace) + class ProductRelationStatus(BaseStatusMixin, DataspacedModel): class Meta(BaseStatusMixin.Meta): diff --git a/product_portfolio/templates/product_portfolio/includes/productrelation_element.html b/product_portfolio/templates/product_portfolio/includes/productrelation_element.html index d4da4180..a2c3b0be 100644 --- a/product_portfolio/templates/product_portfolio/includes/productrelation_element.html +++ b/product_portfolio/templates/product_portfolio/includes/productrelation_element.html @@ -3,13 +3,10 @@ {% include 'includes/object_icon.html' with object=relation.package only %} - {% if relation.package.package_url in vulnerable_purls %} - - - - {% endif %} - {{ relation.package }} -{% elif relation.component %} + + {{ relation.package }} + + {% elif relation.component %} {% include 'includes/object_icon.html' with object=relation.component only %} @@ -51,6 +48,16 @@ {% endif %} +{% if relation.package.vulnerability_count %} + + {% include 'component_catalog/includes/vulnerability_icon_link.html' with url=relation.package.get_absolute_url count=relation.package.vulnerability_count %} + +{% endif %} +{% if relation.component.vulnerability_count %} + + {% include 'component_catalog/includes/vulnerability_icon_link.html' with url=relation.component.get_absolute_url count=relation.component.vulnerability_count %} + +{% endif %} {% if relation.package_id and relation.package.declared_dependencies.all %} diff --git a/product_portfolio/templates/product_portfolio/tabs/tab_dependencies.html b/product_portfolio/templates/product_portfolio/tabs/tab_dependencies.html index c2bc620b..185294c2 100644 --- a/product_portfolio/templates/product_portfolio/tabs/tab_dependencies.html +++ b/product_portfolio/templates/product_portfolio/tabs/tab_dependencies.html @@ -99,9 +99,12 @@ {% if dependency.for_package %} - + {{ dependency.for_package }} + {% if dependency.for_package.vulnerability_count %} + {% include 'component_catalog/includes/vulnerability_icon_link.html' with url=dependency.for_package.get_absolute_url count=dependency.for_package.vulnerability_count %} + {% endif %}
    {{ dependency.for_package.license_expression|default_if_none:"" }}
    @@ -113,6 +116,11 @@ {% if dependency.resolved_to_package %} {{ dependency.resolved_to_package }} + {% if dependency.resolved_to_package.vulnerability_count %} + + {% include 'component_catalog/includes/vulnerability_icon_link.html' with url=dependency.resolved_to_package.get_absolute_url count=dependency.resolved_to_package.vulnerability_count %} + + {% endif %} {% if dependency.resolved_to_package.declared_dependencies_count %} diff --git a/product_portfolio/templates/product_portfolio/tabs/tab_hierarchy.html b/product_portfolio/templates/product_portfolio/tabs/tab_hierarchy.html index 2fbe351b..e4d22942 100644 --- a/product_portfolio/templates/product_portfolio/tabs/tab_hierarchy.html +++ b/product_portfolio/templates/product_portfolio/tabs/tab_hierarchy.html @@ -5,11 +5,16 @@ Inventory (Deployed=All) + {% if product.dataspace.enable_vulnerablecodedb_access %} + + {% endif %} diff --git a/product_portfolio/templates/product_portfolio/tabs/tab_inventory.html b/product_portfolio/templates/product_portfolio/tabs/tab_inventory.html index d5099e8a..e08c2213 100644 --- a/product_portfolio/templates/product_portfolio/tabs/tab_inventory.html +++ b/product_portfolio/templates/product_portfolio/tabs/tab_inventory.html @@ -36,6 +36,11 @@ {% trans 'Item' %} {{ filter_productcomponent.form.object_type }} + {% if product.dataspace.enable_vulnerablecodedb_access %} +
    + {{ filter_productcomponent.form.is_vulnerable }} +
    + {% endif %} {% trans 'Purpose' %} diff --git a/product_portfolio/templatetags/attribution.py b/product_portfolio/templatetags/attribution.py index 2fe70c2c..b78c2d00 100644 --- a/product_portfolio/templatetags/attribution.py +++ b/product_portfolio/templatetags/attribution.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/tests/__init__.py b/product_portfolio/tests/__init__.py index 7aa35de2..f61c4aae 100644 --- a/product_portfolio/tests/__init__.py +++ b/product_portfolio/tests/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/tests/test_admin.py b/product_portfolio/tests/test_admin.py index 2161d2f4..a1ee4d17 100644 --- a/product_portfolio/tests/test_admin.py +++ b/product_portfolio/tests/test_admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/tests/test_admin_guardian.py b/product_portfolio/tests/test_admin_guardian.py index 4ce444ef..e23a6997 100644 --- a/product_portfolio/tests/test_admin_guardian.py +++ b/product_portfolio/tests/test_admin_guardian.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/tests/test_api.py b/product_portfolio/tests/test_api.py index 65a3bec6..3bf50597 100644 --- a/product_portfolio/tests/test_api.py +++ b/product_portfolio/tests/test_api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/tests/test_attribution.py b/product_portfolio/tests/test_attribution.py index 0625dd57..2fa65296 100644 --- a/product_portfolio/tests/test_attribution.py +++ b/product_portfolio/tests/test_attribution.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/tests/test_importers.py b/product_portfolio/tests/test_importers.py index e1a7d5ee..e2f4b642 100644 --- a/product_portfolio/tests/test_importers.py +++ b/product_portfolio/tests/test_importers.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -1014,3 +1014,15 @@ def test_product_portfolio_import_packages_from_scancodeio_importer( self.assertEqual([purl], [str(package) for package in existing["package"]]) self.assertEqual([dependency_uid], [str(package) for package in existing["dependency"]]) self.assertEqual({}, errors) + + self.dataspace.enable_vulnerablecodedb_access = True + self.dataspace.save() + with mock.patch("product_portfolio.models.Product.fetch_vulnerabilities") as mock_fetch: + mock_fetch.return_value = None + importer = ImportPackageFromScanCodeIO( + user=self.super_user, + project_uuid=uuid.uuid4(), + product=self.product1, + ) + importer.save() + mock_fetch.assert_called() diff --git a/product_portfolio/tests/test_models.py b/product_portfolio/tests/test_models.py index 72518845..7ddb42e2 100644 --- a/product_portfolio/tests/test_models.py +++ b/product_portfolio/tests/test_models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/tests/test_views.py b/product_portfolio/tests/test_views.py index 8257047e..4cd59b9f 100644 --- a/product_portfolio/tests/test_views.py +++ b/product_portfolio/tests/test_views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -28,6 +28,8 @@ from component_catalog.models import ComponentAssignedPackage from component_catalog.models import ComponentKeyword from component_catalog.models import Package +from component_catalog.tests import make_package +from component_catalog.tests import make_vulnerability from dejacode_toolkit import scancodeio from dje.models import Dataspace from dje.models import History @@ -82,7 +84,8 @@ def setUp(self): name="Component1", version="1.0", dataspace=self.dataspace ) - self.package1 = Package.objects.create(filename="package1", dataspace=self.dataspace) + self.package1 = make_package(self.dataspace, filename="package1") + self.vulnerability1 = make_vulnerability(self.dataspace, affecting=self.package1) @override_settings(ANONYMOUS_USERS_DATASPACE="nexB") def test_product_portfolio_security_detail_view_no_cross_dataspace_access(self): @@ -524,31 +527,21 @@ def test_product_portfolio_detail_view_inventory_tab_display_scan_features( self.assertContains(response, expected1) self.assertContains(response, expected2) - @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.get_vulnerable_purls") - @mock.patch("dejacode_toolkit.vulnerablecode.VulnerableCode.is_configured") - def test_product_detail_view_inventory_tab_display_vulnerabilities( - self, mock_is_configured, mock_vulnerable_purls - ): - purl = "pkg:pypi/django@2.1" - mock_is_configured.return_value = True - - self.package1.set_package_url(purl) - self.package1.save() + def test_product_detail_view_inventory_tab_display_vulnerabilities(self): ProductPackage.objects.create( product=self.product1, package=self.package1, dataspace=self.dataspace ) - - self.dataspace.enable_vulnerablecodedb_access = True - self.dataspace.save() - - mock_vulnerable_purls.return_value = [purl] - self.client.login(username=self.super_user.username, password="secret") url = self.product1.get_url("tab_inventory") response = self.client.get(url) - expected = '' - self.assertContains(response, expected) + expected = f""" + + 1 + + """ + self.assertContains(response, expected, html=True) def test_product_portfolio_detail_view_feature_field_grouping_in_inventory_tab(self): self.client.login(username="nexb_user", password="secret") diff --git a/product_portfolio/urls.py b/product_portfolio/urls.py index 1be31ece..02395cba 100644 --- a/product_portfolio/urls.py +++ b/product_portfolio/urls.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/product_portfolio/views.py b/product_portfolio/views.py index d38df7c5..bcb4280f 100644 --- a/product_portfolio/views.py +++ b/product_portfolio/views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -23,9 +23,9 @@ from django.core.exceptions import ImproperlyConfigured from django.core.exceptions import ValidationError from django.core.paginator import Paginator -from django.db import models from django.db import transaction from django.db.models import Count +from django.db.models import Prefetch from django.db.models.functions import Lower from django.forms import modelformset_factory from django.http import Http404 @@ -64,10 +64,10 @@ from dejacode_toolkit.scancodeio import get_package_download_url from dejacode_toolkit.scancodeio import get_scan_results_as_file_url from dejacode_toolkit.utils import sha1 -from dejacode_toolkit.vulnerablecode import VulnerableCode from dje import tasks from dje.client_data import add_client_data from dje.filters import BooleanChoiceFilter +from dje.filters import HasCountFilter from dje.models import DejacodeUser from dje.models import History from dje.templatetags.dje_tags import urlize_target_blank @@ -303,9 +303,7 @@ class ProductDetailsView( } def get_queryset(self): - licenses_prefetch = models.Prefetch( - "licenses", License.objects.select_related("usage_policy") - ) + licenses_prefetch = Prefetch("licenses", License.objects.select_related("usage_policy")) productcomponent_qs = ProductComponent.objects.select_related( "component__dataspace", @@ -340,8 +338,8 @@ def get_queryset(self): "licenses__usage_policy", "scancodeprojects", LicenseAssignedTag.prefetch_for_license_tab(), - models.Prefetch("productcomponents", queryset=productcomponent_qs), - models.Prefetch("productpackages", queryset=productpackage_qs), + Prefetch("productcomponents", queryset=productcomponent_qs), + Prefetch("productpackages", queryset=productpackage_qs), ) ) @@ -402,16 +400,18 @@ def tab_hierarchy(self): ) .annotate( children_count=Count("component__children"), + vulnerability_count=Count("component__affected_by_vulnerabilities"), ) .order_by( "feature", - "component", + "component__name", + "component__version", "name", "version", ) ) - declared_dependencies_prefetch = models.Prefetch( + declared_dependencies_prefetch = Prefetch( "package__declared_dependencies", ProductDependency.objects.product(product) ) @@ -423,9 +423,16 @@ def tab_hierarchy(self): "package__licenses", declared_dependencies_prefetch, ) + .annotate( + vulnerability_count=Count("package__affected_by_vulnerabilities"), + ) .order_by( "feature", - "package", + "package__type", + "package__namespace", + "package__name", + "package__version", + "package__filename", ) ) @@ -435,6 +442,12 @@ def tab_hierarchy(self): productcomponent_qs = is_deployed_filter.filter(productcomponent_qs, is_deployed) productpackage_qs = is_deployed_filter.filter(productpackage_qs, is_deployed) + is_vulnerable = self.request.GET.get("hierarchy-is_vulnerable") + if is_vulnerable: + is_vulnerable_filter = HasCountFilter(field_name="vulnerability") + productcomponent_qs = is_vulnerable_filter.filter(productcomponent_qs, is_vulnerable) + productpackage_qs = is_vulnerable_filter.filter(productpackage_qs, is_vulnerable) + if not (productcomponent_qs or productpackage_qs or is_deployed): return @@ -448,6 +461,7 @@ def tab_hierarchy(self): "verbose_name_plural": self.model._meta.verbose_name_plural, "relations_feature_grouped": dict(sorted(relations_feature_grouped.items())), "is_deployed": is_deployed, + "is_vulnerable": is_vulnerable, } return {"fields": [(None, context, None, template)]} @@ -645,26 +659,32 @@ def get_context_data(self, **kwargs): user = self.request.user dataspace = user.dataspace - context["inventory_count"] = self.object.productinventoryitem_set.count() - licenses_prefetch = models.Prefetch( - "licenses", License.objects.select_related("usage_policy") - ) - declared_dependencies_prefetch = models.Prefetch( - "package__declared_dependencies", ProductDependency.objects.product(self.object) + license_qs = License.objects.select_related("usage_policy") + declared_dependencies_qs = ProductDependency.objects.product(self.object) + package_qs = ( + Package.objects.select_related( + "dataspace", + "usage_policy", + ) + .prefetch_related(Prefetch("declared_dependencies", declared_dependencies_qs)) + .with_vulnerability_count() ) + component_qs = Component.objects.select_related( + "dataspace", + "owner__dataspace", + "usage_policy", + ).with_vulnerability_count() productpackage_qs = ( self.object.productpackages.select_related( - "package__dataspace", - "package__usage_policy", "review_status", "purpose", ) .prefetch_related( - licenses_prefetch, - declared_dependencies_prefetch, + Prefetch("licenses", license_qs), + Prefetch("package", package_qs), ) .order_by( "feature", @@ -686,16 +706,14 @@ def get_context_data(self, **kwargs): productcomponent_qs = ( self.object.productcomponents.select_related( - "component__dataspace", - "component__owner__dataspace", - "component__usage_policy", "review_status", "purpose", ) .prefetch_related( + Prefetch("component", component_qs), + Prefetch("licenses", license_qs), "component__packages", "component__children", - licenses_prefetch, ) .order_by( "feature", @@ -767,24 +785,6 @@ def get_context_data(self, **kwargs): if "error" in compliance_alerts: context["compliance_errors"] = True - # 6. Add vulnerability data - vulnerablecode = VulnerableCode(dataspace) - enable_vulnerabilities = all( - [ - dataspace.enable_vulnerablecodedb_access, - vulnerablecode.is_configured(), - ] - ) - if enable_vulnerabilities: - # Re-use the inventory mapping to prevent duplicated queries - packages = [ - inventory_item.package - for inventory_item in object_list - if isinstance(inventory_item, ProductPackage) - ] - - context["vulnerable_purls"] = vulnerablecode.get_vulnerable_purls(packages) - context.update( { "filter_productcomponent": filter_productcomponent, @@ -969,12 +969,15 @@ def get_context_data(self, **kwargs): context_data = super().get_context_data(**kwargs) product = self.object + for_package_qs = Package.objects.only_rendering_fields().with_vulnerability_count() + resolved_to_package_qs = ( + Package.objects.only_rendering_fields() + .declared_dependencies_count(product) + .with_vulnerability_count() + ) dependency_qs = product.dependencies.prefetch_related( - models.Prefetch("for_package", Package.objects.only_rendering_fields()), - models.Prefetch( - "resolved_to_package", - Package.objects.only_rendering_fields().declared_dependencies_count(product), - ), + Prefetch("for_package", for_package_qs), + Prefetch("resolved_to_package", resolved_to_package_qs), ) filter_dependency = DependencyFilterSet( diff --git a/purldb/__init__.py b/purldb/__init__.py index 7aa35de2..f61c4aae 100644 --- a/purldb/__init__.py +++ b/purldb/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/purldb/filters.py b/purldb/filters.py index 290dec29..a62e618e 100644 --- a/purldb/filters.py +++ b/purldb/filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/purldb/tests/__init__.py b/purldb/tests/__init__.py index 7aa35de2..f61c4aae 100644 --- a/purldb/tests/__init__.py +++ b/purldb/tests/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/purldb/tests/test_purldb_toolkit.py b/purldb/tests/test_purldb_toolkit.py index bd2f4bb9..17ad45b8 100644 --- a/purldb/tests/test_purldb_toolkit.py +++ b/purldb/tests/test_purldb_toolkit.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/purldb/tests/test_views.py b/purldb/tests/test_views.py index ae56ef65..5737c4eb 100644 --- a/purldb/tests/test_views.py +++ b/purldb/tests/test_views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/purldb/urls.py b/purldb/urls.py index d7cb284f..3392d3ec 100644 --- a/purldb/urls.py +++ b/purldb/urls.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/purldb/views.py b/purldb/views.py index ab87f3e8..d568310b 100644 --- a/purldb/views.py +++ b/purldb/views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/__init__.py b/reporting/__init__.py index 7aa35de2..f61c4aae 100644 --- a/reporting/__init__.py +++ b/reporting/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/admin.py b/reporting/admin.py index da1a2c15..f1fe3564 100644 --- a/reporting/admin.py +++ b/reporting/admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/api.py b/reporting/api.py index 63a7b967..5d9bae25 100644 --- a/reporting/api.py +++ b/reporting/api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/apps.py b/reporting/apps.py index ba3e285a..cca174e0 100644 --- a/reporting/apps.py +++ b/reporting/apps.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/fields.py b/reporting/fields.py index a8d7b1ad..d042e41b 100644 --- a/reporting/fields.py +++ b/reporting/fields.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/filters.py b/reporting/filters.py index 094081f3..7ad1bd9e 100644 --- a/reporting/filters.py +++ b/reporting/filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/forms.py b/reporting/forms.py index 70fcd78d..b4f2b512 100644 --- a/reporting/forms.py +++ b/reporting/forms.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -28,6 +28,7 @@ from component_catalog.models import PackageAssignedLicense from component_catalog.models import Subcomponent from component_catalog.models import SubcomponentAssignedLicense +from component_catalog.models import Vulnerability from dje.forms import DataspacedAdminForm from dje.mass_update import DejacodeMassUpdateForm from dje.models import DejacodeUser @@ -118,6 +119,7 @@ DejacodeUser, CodebaseResource, CodebaseResourceUsage, + Vulnerability, ] FIELDS_WHITELIST = { @@ -199,6 +201,7 @@ def get_model_data_for_column_template(dataspace=None): "redistribution_required", "change_tracking_required", "where_used", + "is_vulnerable", ], }, "component_catalog:package": { @@ -217,6 +220,7 @@ def get_model_data_for_column_template(dataspace=None): "short_package_url", "where_used", "inferred_url", + "is_vulnerable", ], }, "license_library:license": { diff --git a/reporting/inlines.py b/reporting/inlines.py index 970f4259..81b16721 100644 --- a/reporting/inlines.py +++ b/reporting/inlines.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/introspection.py b/reporting/introspection.py index a1760dc5..86b203b4 100644 --- a/reporting/introspection.py +++ b/reporting/introspection.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/models.py b/reporting/models.py index 7e6a34ab..5afa87c8 100644 --- a/reporting/models.py +++ b/reporting/models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/static/js/change_columntemplate.js b/reporting/static/js/change_columntemplate.js index 7766a4fa..e0a9aa91 100644 --- a/reporting/static/js/change_columntemplate.js +++ b/reporting/static/js/change_columntemplate.js @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ diff --git a/reporting/static/js/change_query.js b/reporting/static/js/change_query.js index 1e31193b..cd1345b5 100644 --- a/reporting/static/js/change_query.js +++ b/reporting/static/js/change_query.js @@ -3,7 +3,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # */ diff --git a/reporting/tests/__init__.py b/reporting/tests/__init__.py index 7aa35de2..f61c4aae 100644 --- a/reporting/tests/__init__.py +++ b/reporting/tests/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/tests/test_admin.py b/reporting/tests/test_admin.py index bf345fc8..d59a2bd1 100644 --- a/reporting/tests/test_admin.py +++ b/reporting/tests/test_admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/tests/test_api.py b/reporting/tests/test_api.py index 6b47ae37..240959a6 100644 --- a/reporting/tests/test_api.py +++ b/reporting/tests/test_api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/tests/test_introspection.py b/reporting/tests/test_introspection.py index bac7c22a..e1fa0f83 100644 --- a/reporting/tests/test_introspection.py +++ b/reporting/tests/test_introspection.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/tests/test_models.py b/reporting/tests/test_models.py index c1552843..efbe0604 100644 --- a/reporting/tests/test_models.py +++ b/reporting/tests/test_models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # @@ -2031,6 +2031,11 @@ def test_get_model_data_for_component_column_template(self): "label": "website_terms_of_use", "value": "website_terms_of_use", }, + { + "group": "Many to Many Fields", + "label": "affected_by_vulnerabilities", + "value": "affected_by_vulnerabilities", + }, {"group": "Many to Many Fields", "label": "children", "value": "children"}, {"group": "Many to Many Fields", "label": "licenses", "value": "licenses"}, {"group": "Many to Many Fields", "label": "packages", "value": "packages"}, @@ -2091,6 +2096,7 @@ def test_get_model_data_for_component_column_template(self): "value": "change_tracking_required", }, {"group": "Properties", "label": "where_used", "value": "where_used"}, + {"group": "Properties", "label": "is_vulnerable", "value": "is_vulnerable"}, ] self.assertEqual(expected, value["component_catalog:component"]["grouped_fields"]) diff --git a/reporting/tests/test_views.py b/reporting/tests/test_views.py index 5d3fafe0..7949815e 100644 --- a/reporting/tests/test_views.py +++ b/reporting/tests/test_views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/urls.py b/reporting/urls.py index 08f97ef8..c9a0693d 100644 --- a/reporting/urls.py +++ b/reporting/urls.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/utils.py b/reporting/utils.py index 0f0990b5..038a03f6 100644 --- a/reporting/utils.py +++ b/reporting/utils.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/reporting/views.py b/reporting/views.py index a64857e4..afa00f2c 100644 --- a/reporting/views.py +++ b/reporting/views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/setup.cfg b/setup.cfg index 3e1826ff..66de081a 100644 --- a/setup.cfg +++ b/setup.cfg @@ -6,7 +6,7 @@ description = Automate open source license compliance and ensure supply chain in long_description = file:README.rst author = nexB Inc. author_email = info@aboutcode.org -url = https://github.com/nexB/dejacode +url = https://github.com/aboutcode-org/dejacode classifiers = Development Status :: 5 - Production/Stable Intended Audience :: Developers @@ -66,6 +66,7 @@ install_requires = django-hcaptcha-field==1.4.0 django-guardian==2.4.0 django-environ==0.11.2 + django-debug-toolbar==4.3.0 # REST API djangorestframework==3.14.0 # API documentation, `coreapi` and its requirements: @@ -101,6 +102,10 @@ install_requires = rq==1.16.2 django-rq==2.10.2 fakeredis==2.21.1 + # Scheduler + rq-scheduler==0.13.1 + crontab==1.0.1 + freezegun==1.5.1 # Libs certifi==2024.2.2 urllib3==2.2.1 @@ -171,8 +176,6 @@ install_requires = [options.extras_require] dev = - # Development tools - django-debug-toolbar==4.3.0 # Linter and Validation ruff==0.5.2 # Documentation diff --git a/setup.py b/setup.py index 069a8711..914880b8 100755 --- a/setup.py +++ b/setup.py @@ -4,7 +4,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/thirdparty/dist/crontab-1.0.1.tar.gz b/thirdparty/dist/crontab-1.0.1.tar.gz new file mode 100644 index 00000000..097e390a Binary files /dev/null and b/thirdparty/dist/crontab-1.0.1.tar.gz differ diff --git a/thirdparty/dist/freezegun-1.5.1-py3-none-any.whl b/thirdparty/dist/freezegun-1.5.1-py3-none-any.whl new file mode 100644 index 00000000..4955690d Binary files /dev/null and b/thirdparty/dist/freezegun-1.5.1-py3-none-any.whl differ diff --git a/thirdparty/dist/rq_scheduler-0.13.1-py2.py3-none-any.whl b/thirdparty/dist/rq_scheduler-0.13.1-py2.py3-none-any.whl new file mode 100644 index 00000000..938078d0 Binary files /dev/null and b/thirdparty/dist/rq_scheduler-0.13.1-py2.py3-none-any.whl differ diff --git a/workflow/__init__.py b/workflow/__init__.py index 7aa35de2..f61c4aae 100644 --- a/workflow/__init__.py +++ b/workflow/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/admin.py b/workflow/admin.py index 75160d04..f0cbe605 100644 --- a/workflow/admin.py +++ b/workflow/admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/api.py b/workflow/api.py index b4c3fdd7..e9640b1f 100644 --- a/workflow/api.py +++ b/workflow/api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/apps.py b/workflow/apps.py index b5bfbc4f..aaf0d764 100644 --- a/workflow/apps.py +++ b/workflow/apps.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/filters.py b/workflow/filters.py index 0dd5ab1d..b0638da6 100644 --- a/workflow/filters.py +++ b/workflow/filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/forms.py b/workflow/forms.py index 15965013..911f4e58 100644 --- a/workflow/forms.py +++ b/workflow/forms.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/inlines.py b/workflow/inlines.py index aed41d99..861f5b7b 100644 --- a/workflow/inlines.py +++ b/workflow/inlines.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/models.py b/workflow/models.py index e1cc92ca..1312775a 100644 --- a/workflow/models.py +++ b/workflow/models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/notification.py b/workflow/notification.py index 8c117ade..c3b71137 100644 --- a/workflow/notification.py +++ b/workflow/notification.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/tests/__init__.py b/workflow/tests/__init__.py index 7aa35de2..f61c4aae 100644 --- a/workflow/tests/__init__.py +++ b/workflow/tests/__init__.py @@ -2,6 +2,6 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/tests/test_admin.py b/workflow/tests/test_admin.py index 59fc0f0b..51ec485d 100644 --- a/workflow/tests/test_admin.py +++ b/workflow/tests/test_admin.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/tests/test_api.py b/workflow/tests/test_api.py index 0053c9f1..b9da8cbe 100644 --- a/workflow/tests/test_api.py +++ b/workflow/tests/test_api.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/tests/test_filters.py b/workflow/tests/test_filters.py index ba43a953..ee57cb4f 100644 --- a/workflow/tests/test_filters.py +++ b/workflow/tests/test_filters.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/tests/test_models.py b/workflow/tests/test_models.py index dfab6962..cc304d09 100644 --- a/workflow/tests/test_models.py +++ b/workflow/tests/test_models.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/tests/test_views.py b/workflow/tests/test_views.py index e85a8899..c0a542b9 100644 --- a/workflow/tests/test_views.py +++ b/workflow/tests/test_views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/urls.py b/workflow/urls.py index da11be28..05a726f1 100644 --- a/workflow/urls.py +++ b/workflow/urls.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # diff --git a/workflow/views.py b/workflow/views.py index efdc5d77..6b941205 100644 --- a/workflow/views.py +++ b/workflow/views.py @@ -2,7 +2,7 @@ # Copyright (c) nexB Inc. and others. All rights reserved. # DejaCode is a trademark of nexB Inc. # SPDX-License-Identifier: AGPL-3.0-only -# See https://github.com/nexB/dejacode for support or download. +# See https://github.com/aboutcode-org/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. #