Merge main and fix conflicts

Signed-off-by: tdruez <tdruez@nexb.com>

CHANGELOG.rst

@@ -5,41 +5,41 @@ Release notes
 
 - Add visual indicator in hierarchy views, when an object on the far left or far right
   also belong or have a hierarchy (relationship tree).
-  https://github.com/nexB/dejacode/issues/70
+  https://github.com/aboutcode-org/dejacode/issues/70
 
 - Add search and pagination on the Product Inventory tab.
-  https://github.com/nexB/dejacode/issues/3
-  https://github.com/nexB/dejacode/issues/112
+  https://github.com/aboutcode-org/dejacode/issues/3
+  https://github.com/aboutcode-org/dejacode/issues/112
 
 - Fix an issue displaying the "Delete" button in the "Edit Product Relationship"
   modal form.
-  https://github.com/nexB/dejacode/issues/128
+  https://github.com/aboutcode-org/dejacode/issues/128
 
 - Add support for PURL(s) in the "Add Package" modal.
   If the PURL type is supported by the packageurl_python library, a download URL
   will be generated for creating the package and submitting a scan.
-  https://github.com/nexB/dejacode/issues/131
+  https://github.com/aboutcode-org/dejacode/issues/131
 
 - Leverage PurlDB during the "Add Package" process.
   DejaCode will look up the PurlDB to retrieve and fetch all available data to
   create the package.
-  https://github.com/nexB/dejacode/issues/131
+  https://github.com/aboutcode-org/dejacode/issues/131
 
 - Populate the Package notice_text using "*NOTICE*" file content from Scan "key files".
-  https://github.com/nexB/dejacode/issues/136
+  https://github.com/aboutcode-org/dejacode/issues/136
 
 - Added 2 new license related fields on the Component and Package models:
   * declared_license_expression
   * other_license_expression
-  https://github.com/nexB/dejacode/issues/63
+  https://github.com/aboutcode-org/dejacode/issues/63
 
 - Added 2 properties on the Component and Package models:
   * declared_license_expression_spdx (computed from declared_license_expression)
   * other_license_expression_spdx (computed from other_license_expression)
-  https://github.com/nexB/dejacode/issues/63
+  https://github.com/aboutcode-org/dejacode/issues/63
 
 - Removed 2 fields: Package.declared_license and Component.concluded_license
-  https://github.com/nexB/dejacode/issues/63
+  https://github.com/aboutcode-org/dejacode/issues/63
 
 - The new license fields are automatically populated from the Package scan
   "Update packages automatically from scan".
@@ -49,103 +49,138 @@ Release notes
   "Add Component from Package data".
   The license expression values provided in the form for the new field is now
   properly checked and return a validation error when incorrect.
-  https://github.com/nexB/dejacode/issues/63
+  https://github.com/aboutcode-org/dejacode/issues/63
 
 - Use the declared_license_expression_spdx value in SPDX outputs.
-  https://github.com/nexB/dejacode/issues/63
+  https://github.com/aboutcode-org/dejacode/issues/63
 
 - Add new ProductDependency model to support relating Packages in the context of a
   Product.
-  https://github.com/nexB/dejacode/issues/138
+  https://github.com/aboutcode-org/dejacode/issues/138
+
+- Add a task scheduler service to the Docker Compose stack.
+  This service runs a dedicated ``setupcron`` management command to create the
+  application's scheduled cron jobs.
+  The scheduler is configured to run the daily vulnerabilities update task.
+  https://github.com/aboutcode-org/dejacode/issues/94
+
+- Add a new Vulnerability model and all the code logic to fetch and create
+  Vulnerability records and assign those to Package/Component through ManyToMany
+  relationships.
+  A fetchvulnerabilities management command is available to fetch all the relevant
+  data from VulnerableCode for a given Dataspace.
+  The latest vulnerability data refresh date is displayed in the Admin dashboard in a
+  new "Data updates" section in the bottom right corner.
+  It is also available in the "Integration Status" page.
+  The Package/Component views that display vulnerability information (icon or tab)
+  are now using the data from the Vulnerability model in place of calling the
+  VulnerableCode API on each request. This results into much better performances as
+  we do not depend on the VulnerableCode service to render the DejaCode view anymore.
+  Also, this will make Vulnerability data available in the Reporting system.
+  The vulnerability icon is displayed next to the Package/Component identifier in the
+  Product views: "Inventory", "Hierarchy", "Dependencies" tabs.
+  The vulnerability data is available in Reporting either through the is_vulnerable
+  property on Package/Component column template or going through the full
+  affected_by_vulnerabilities m2m field.
+  This is available in both Query and ColumnTemplate.
+  The vulnerabilities are fetched each time a Package is created/modified
+  (note that a purl is required on the package for the lookup).
+  Also, all the Packages of a Product are updated with latest vulnerabilities from
+  the VulnerableCode service following importing data in Product using:
+  - Import data from Scan
+  - Load Packages from SBOMs
+  - Import Packages from manifests
+  - Pull ScanCode.io Project data
+  https://github.com/aboutcode-org/dejacode/issues/94
 
 ### Version 5.1.0
 
 - Upgrade Python version to 3.12 and Django to 5.0.x
-  https://github.com/nexB/dejacode/issues/50
+  https://github.com/aboutcode-org/dejacode/issues/50
 
 - Replace Celery by RQ for async job queue and worker.
-  https://github.com/nexB/dejacode/issues/6
+  https://github.com/aboutcode-org/dejacode/issues/6
 
 - Add support for CycloneDX spec version "1.6".
   In the UI and API, older spe version such as "1.4" and "1.5" are also available as
   download.
-  https://github.com/nexB/dejacode/pull/79
+  https://github.com/aboutcode-org/dejacode/pull/79
 
 - Lookup in PurlDB by purl in Add Package form.
   When a Package URL is available in the context of the "Add Package" form,
   for example when using a link from the Vulnerabilities tab,
   data is fetched from the PurlDB to initialize the form.
-  https://github.com/nexB/dejacode/issues/47
+  https://github.com/aboutcode-org/dejacode/issues/47
 
 - If you select two versions of the same Product in the Product list, or two different
   Products, and click the Compare button, you can now download the results of the
   comparison to a .xlsx file, making it easy to share the information with your
   colleagues.
-  https://github.com/nexB/dejacode/issues/7
+  https://github.com/aboutcode-org/dejacode/issues/7
 
 - Add dark theme support in UI.
-  https://github.com/nexB/dejacode/issues/25
+  https://github.com/aboutcode-org/dejacode/issues/25
 
 - Add "Load Packages from SBOMs", "Import scan results", and
   "Pull ScanCode.io project data" feature as Product action in the REST API.
-  https://github.com/nexB/dejacode/issues/59
+  https://github.com/aboutcode-org/dejacode/issues/59
 
 - Add REST API endpoints to download SBOMs as CycloneDX and SPDX.
-  https://github.com/nexB/dejacode/issues/60
+  https://github.com/aboutcode-org/dejacode/issues/60
 
 - Refactor the "Import manifest" feature as "Load SBOMs".
-  https://github.com/nexB/dejacode/issues/61
+  https://github.com/aboutcode-org/dejacode/issues/61
 
 - Add support to import packages from manifest.
-  https://github.com/nexB/dejacode/issues/65
+  https://github.com/aboutcode-org/dejacode/issues/65
 
 - Add a vulnerability link to the VulnerableCode app in the Vulnerability tab.
-  https://github.com/nexB/dejacode/issues/4
+  https://github.com/aboutcode-org/dejacode/issues/4
 
 - Add a DEJACODE_SUPPORT_EMAIL setting for support email address customization.
-  https://github.com/nexB/dejacode/issues/76
+  https://github.com/aboutcode-org/dejacode/issues/76
 
 - Show the individual PURL fields in the Package details view.
-  https://github.com/nexB/dejacode/issues/83
+  https://github.com/aboutcode-org/dejacode/issues/83
 
 - Fix the logout link of the admin app.
-  https://github.com/nexB/dejacode/issues/89
+  https://github.com/aboutcode-org/dejacode/issues/89
 
 - Display full commit in the version displayed in the UI
-  https://github.com/nexB/dejacode/issues/88
+  https://github.com/aboutcode-org/dejacode/issues/88
 
 - Refine the Product comparison logic for Packages.
   The type and namespace fields are now used along the name field to match similar
   Packages (excluding the version).
-  https://github.com/nexB/dejacode/issues/113
+  https://github.com/aboutcode-org/dejacode/issues/113
 
 - Refactor the implementation of Keywords on forms to allow more flexibilty.
   Existing Keywords are suggested for consistency but any values is now allowed.
-  https://github.com/nexB/dejacode/issues/48
+  https://github.com/aboutcode-org/dejacode/issues/48
 
 - Display Product inventory count on the Product list view.
-  https://github.com/nexB/dejacode/issues/81
+  https://github.com/aboutcode-org/dejacode/issues/81
 
 - Always display the full Package URL in the UI view including the "pkg:" prefix.
-  https://github.com/nexB/dejacode/issues/115
+  https://github.com/aboutcode-org/dejacode/issues/115
 
 - Add a new AboutCode tab in Package details view.
-  https://github.com/nexB/dejacode/issues/42
+  https://github.com/aboutcode-org/dejacode/issues/42
 
 - Enhance Package Import to support modifications.
-  https://github.com/nexB/dejacode/issues/84
+  https://github.com/aboutcode-org/dejacode/issues/84
 
 - Add an option on the "Add to Product" form to to replace any existing relationships
   with a different version of the same object by the selected object.
-  https://github.com/nexB/dejacode/issues/12
+  https://github.com/aboutcode-org/dejacode/issues/12
 
 ### Version 5.0.1
 
 - Improve the stability of the "Check for new Package versions" feature.
-  https://github.com/nexB/dejacode/issues/17
+  https://github.com/aboutcode-org/dejacode/issues/17
 
 - Improve the support for SourgeForge download URLs.
-  https://github.com/nexB/dejacode/issues/26
+  https://github.com/aboutcode-org/dejacode/issues/26
 
 ### Version 5.0.0
 

Dockerfile

@@ -2,13 +2,13 @@
 # Copyright (c) nexB Inc. and others. All rights reserved.
 # DejaCode is a trademark of nexB Inc.
 # SPDX-License-Identifier: AGPL-3.0-only
-# See https://github.com/nexB/dejacode for support or download.
+# See https://github.com/aboutcode-org/dejacode for support or download.
 # See https://aboutcode.org for more information about AboutCode FOSS projects.
 #
 
 FROM python:3.12-slim
 
-LABEL org.opencontainers.image.source="https://github.com/nexB/dejacode"
+LABEL org.opencontainers.image.source="https://github.com/aboutcode-org/dejacode"
 LABEL org.opencontainers.image.description="DejaCode"
 LABEL org.opencontainers.image.licenses="AGPL-3.0-only"
 

Makefile

@@ -2,7 +2,7 @@
 # Copyright (c) nexB Inc. and others. All rights reserved.
 # DejaCode is a trademark of nexB Inc.
 # SPDX-License-Identifier: AGPL-3.0-only
-# See https://github.com/nexB/dejacode for support or download.
+# See https://github.com/aboutcode-org/dejacode for support or download.
 # See https://aboutcode.org for more information about AboutCode FOSS projects.
 #
 

README.rst

@@ -30,7 +30,7 @@ Getting started
 The DejaCode documentation is available here: https://dejacode.readthedocs.io/
 
 If you have questions please ask them in
-`Discussions <https://github.com/nexB/dejacode/discussions>`_.
+`Discussions <https://github.com/aboutcode-org/dejacode/discussions>`_.
 
 If you want to contribute to DejaCode, start with our
 `Contributing <https://dejacode.readthedocs.io/en/latest/contributing.html>`_ page.
@@ -75,8 +75,8 @@ You can learn more about this option by contacting us at
 https://www.nexb.com/contact-us/
 
 
-.. |ci-tests| image:: https://github.com/nexB/dejacode/actions/workflows/ci.yml/badge.svg?branch=main
-    :target: https://github.com/nexB/dejacode/actions/workflows/ci.yml
+.. |ci-tests| image:: https://github.com/aboutcode-org/dejacode/actions/workflows/ci.yml/badge.svg?branch=main
+    :target: https://github.com/aboutcode-org/dejacode/actions/workflows/ci.yml
     :alt: CI Tests Status
 
 .. |docs-rtd| image:: https://readthedocs.org/projects/dejacode/badge/?version=latest

component_catalog/__init__.py

@@ -2,6 +2,6 @@
 # Copyright (c) nexB Inc. and others. All rights reserved.
 # DejaCode is a trademark of nexB Inc.
 # SPDX-License-Identifier: AGPL-3.0-only
-# See https://github.com/nexB/dejacode for support or download.
+# See https://github.com/aboutcode-org/dejacode for support or download.
 # See https://aboutcode.org for more information about AboutCode FOSS projects.
 #

component_catalog/admin.py

@@ -2,7 +2,7 @@
 # Copyright (c) nexB Inc. and others. All rights reserved.
 # DejaCode is a trademark of nexB Inc.
 # SPDX-License-Identifier: AGPL-3.0-only
-# See https://github.com/nexB/dejacode for support or download.
+# See https://github.com/aboutcode-org/dejacode for support or download.
 # See https://aboutcode.org for more information about AboutCode FOSS projects.
 #
 

component_catalog/api.py

@@ -2,7 +2,7 @@
 # Copyright (c) nexB Inc. and others. All rights reserved.
 # DejaCode is a trademark of nexB Inc.
 # SPDX-License-Identifier: AGPL-3.0-only
-# See https://github.com/nexB/dejacode for support or download.
+# See https://github.com/aboutcode-org/dejacode for support or download.
 # See https://aboutcode.org for more information about AboutCode FOSS projects.
 #
 

component_catalog/apps.py

@@ -2,7 +2,7 @@
 # Copyright (c) nexB Inc. and others. All rights reserved.
 # DejaCode is a trademark of nexB Inc.
 # SPDX-License-Identifier: AGPL-3.0-only
-# See https://github.com/nexB/dejacode for support or download.
+# See https://github.com/aboutcode-org/dejacode for support or download.
 # See https://aboutcode.org for more information about AboutCode FOSS projects.
 #
 

component_catalog/filters.py

@@ -2,7 +2,7 @@
 # Copyright (c) nexB Inc. and others. All rights reserved.
 # DejaCode is a trademark of nexB Inc.
 # SPDX-License-Identifier: AGPL-3.0-only
-# See https://github.com/nexB/dejacode for support or download.
+# See https://github.com/aboutcode-org/dejacode for support or download.
 # See https://aboutcode.org for more information about AboutCode FOSS projects.
 #
 
@@ -28,6 +28,21 @@
 from license_library.models import License
 
 
+class IsVulnerableFilter(HasRelationFilter):
+    def __init__(self, *args, **kwargs):
+        kwargs["lookup_expr"] = "isnull"
+        kwargs["empty_label"] = "Any"
+        kwargs.setdefault("label", _("Is Vulnerable"))
+        kwargs.setdefault(
+            "choices",
+            (
+                ("yes", _("Affected by vulnerabilities")),
+                ("no", _("No vulnerabilities found")),
+            ),
+        )
+        super().__init__(*args, **kwargs)
+
+
 class ComponentFilterSet(DataspacedFilterSet):
     related_only = [
         "licenses",
@@ -85,6 +100,10 @@ class ComponentFilterSet(DataspacedFilterSet):
             search_placeholder="Search keywords",
         ),
     )
+    is_vulnerable = IsVulnerableFilter(
+        field_name="affected_by_vulnerabilities",
+        widget=DropDownRightWidget(link_content='<i class="fas fa-bug"></i>'),
+    )
 
     class Meta:
         model = Component
@@ -219,6 +238,10 @@ class PackageFilterSet(DataspacedFilterSet):
         empty_label="Last modified (default)",
         widget=SortDropDownWidget,
     )
+    is_vulnerable = IsVulnerableFilter(
+        field_name="affected_by_vulnerabilities",
+        widget=DropDownRightWidget(link_content='<i class="fas fa-bug"></i>'),
+    )
 
     class Meta:
         model = Package

component_catalog/forms.py

@@ -2,7 +2,7 @@
 # Copyright (c) nexB Inc. and others. All rights reserved.
 # DejaCode is a trademark of nexB Inc.
 # SPDX-License-Identifier: AGPL-3.0-only
-# See https://github.com/nexB/dejacode for support or download.
+# See https://github.com/aboutcode-org/dejacode for support or download.
 # See https://aboutcode.org for more information about AboutCode FOSS projects.
 #
 
@@ -446,6 +446,9 @@ def save(self, *args, **kwargs):
             )
             self.cleaned_data["scan_submitted"] = True
 
+        if self.user.dataspace.enable_vulnerablecodedb_access:
+            instance.fetch_vulnerabilities()
+
         return instance
 
 
@@ -1043,6 +1046,11 @@ def save(self, commit=True):
         self._set_purldb_uuid_on_instance()
         return super().save(commit)
 
+    def _save_m2m(self):
+        super()._save_m2m()
+        if self.dataspace.enable_vulnerablecodedb_access:
+            self.instance.fetch_vulnerabilities()
+
 
 class ComponentMassUpdateForm(
     LicenseExpressionFormMixin,

component_catalog/fuzzy.py

@@ -2,7 +2,7 @@
 # Copyright (c) nexB Inc. and others. All rights reserved.
 # DejaCode is a trademark of nexB Inc.
 # SPDX-License-Identifier: AGPL-3.0-only
-# See https://github.com/nexB/dejacode for support or download.
+# See https://github.com/aboutcode-org/dejacode for support or download.
 # See https://aboutcode.org for more information about AboutCode FOSS projects.
 #