Base implementation of a Vulnerability models #94

[tdruez]

This PR is the first step in the CRAVEX implementation. See #94

• It introduces a new Vulnerability model and all the code logic to fetch and create Vulnerability records and assign those to Package/Component through ManyToMany relationships.
• A new fetchvulnerabilities management command is available to fetch all the relevant data from VulnerableCode for a given Dataspace.
• A scheduler was added to run the vulnerability data update daily (we can discuss and adjust this to the most suitable value, depending on how often VCIO is updated for example).
• The latest vulnerability data refresh date is displayed in the Admin dashboard in a new "Data updates" section in the botton right corner.
• The Package/Component views that display vulnerability information (icon or tab) are now using the data from the Vulnerability model in place of calling the VulnerableCode API on each request. This result into much better performances as we do not depend on the VulnerableCode performances to render the DejaCode view anymore. Also, this will make Vulnerability data available in the Reporting system.
• A filter is available next to the "Identifier" column header in the Package list view, and Product tabs.
• The vulnerability icon is displayed next to the Package/Component identifier in the Product views: "Inventory", "Hierarchy", "Dependencies" tabs.
• The vulnerability data is available in Reporting either through the is_vulnerable property on Package/Component column template or going through the full affected_by_vulnerabilities m2m field. This is available in both Query and ColumnTemplate. Query example: Package > affected_by_vulnerabilities > IS_NULL = False

Scheduler:

• Add a scheduler service to run the vulnerability updates on a daily basis:
  5978612
  https://github.com/rq/rq-scheduler?tab=readme-ov-file#cron-jobs
  https://github.com/rq/django-rq?tab=readme-ov-file#support-for-scheduled-jobs

---

TODO:

• Update the ProductTabInventoryView and all Product related views in general to the new system
• Display the vulnerability icon in Herarchy/Dependency tabs
• Add a "has_vulnerability" property on Package and Component to support Queries and Column Templates and Product Review #2
• Add a scheduler to run the vulnerability updates on a daily basis using RQScheduler.
• Fix and add tests
• On Package/Component addition (Add package, Import, etc...) lookup and create the vulnerabilities
• Complete the management command

[DennisClark]

Hi @tdruez I checked everything you mentioned in Staging, and it all works very nicely.

My one suggestion at this point is to add a little bug icon next to the filter button in the Identifier column header on the Packages list view (also on Components) similar to what you did on the Inventory tab of the Product view.

[tdruez]

@DennisClark I've deployed the latest improvements for you to review. The set of features is now complete.

Changes:

• The issue reporting in Base implementation of a Vulnerability models #94 #148 (comment) is fixed
• The scheduler service is in place (vulnerability data update run daily)
• Display the latest data update on the integration status page
• The vulnerabilities are fetched each time a Package is created/modified (note that a purl is required on the package for the lookup), either through:
  • Add package (providing a purl)
  • Add package form
  • Import package
  • Add package Admin
• Also, the whole Packages of a Product are updated with latest vulnerabilities from VCIO following importing data in Product using:
  • Import data from Scan
  • Load Packages from SBOMs
  • Import Packages from manifests
  • Pull ScanCode.io Project data

[DennisClark]

Fix to the problem creating a new package in a non-reference dataspace confirmed in Staging.

[DennisClark]

@tdruez everything looks good to me, no problems found!