Does scanning of PHP composer work properly?

[scandinaf710]

Hi all,

I am using scancode 2.1.0 and according to the release notes scanning PHP composer is already there. However, even though scancode finds composer.json, I don't see any analysis of dependencies section even when I put only this file as input.

How to run this package check correctly?

[pombredanne]

@scandinaf710 good catch. The package parser is there but somehow not active. I pushed a branch and PR to fix this

[pombredanne]

@scandinaf710 A fix is waiting for you in the lucky #777 PR !
You can test it by downloading this https://github.com/nexB/scancode-toolkit/archive/776-composer-package-detection.tar.gz
Thanks again for fining this.

[scandinaf710]

@pombredanne Well, I downloaded tarball and ran this version of scancode asking to analyze just one single file composer.json with the following "require" section:

"require": {
"php": "7.1.",
"symfony/symfony": "3.2.",
"sensio/distribution-bundle": "^5.0",
"doctrine/orm": "^2.5",
"doctrine/doctrine-bundle": "^1.6",
"doctrine/doctrine-cache-bundle": "^1.2",
"symfony/swiftmailer-bundle": "^2.3.10",
"symfony/monolog-bundle": "^3.0.2",
"symfony/polyfill-apcu": "^1.0",
"sensio/framework-extra-bundle": "^3.0.2",
"incenteev/composer-parameter-handler": "^2.0",
"twig/twig": "^1.0||^2.0",
"doctrine/doctrine-migrations-bundle": "^1.0",
"greenlion/php-sql-parser": "^4.1",
"eightpoints/guzzle-bundle": "^5.3",
"jms/serializer-bundle": "^1.1",
"friendsofsymfony/rest-bundle": "^2.1",
"nelmio/api-doc-bundle": "^2.13",
"snc/redis-bundle": "^2.0",
"predis/predis": "^1.0",
"paragonie/halite": "^3.2",
"knplabs/knp-paginator-bundle": "^2.5",
"egeloen/base64-file-bundle": "^1.0",
"liip/imagine-bundle": "^1.7",
"sonata-project/admin-bundle": "^3.17",
"sonata-project/doctrine-orm-admin-bundle": "^3.1",
"doctrine/doctrine-fixtures-bundle": "^2.3",
"php-amqplib/rabbitmq-bundle": "^1.12",
"mtdowling/cron-expression": "^1.2"

What I expect is that scancode will show the list of licenses for each module. But it gives me the following:

"scancode_version": "2.1.0",
"scancode_options": {
"--copyright": true,
"--license": true,
"--package": true,
"--license-score": 0,
"--format": "json-pp"
},
"files_count": 1,
"files": [
{
"path": "scancode-777/composer.json",
"scan_errors": [],
"licenses": [
{
"key": "proprietary",
"score": 10.0,
"short_name": "Proprietary",
"category": "Proprietary Free",
"owner": "nexB",
"homepage_url": "",
"text_url": "",
"dejacode_url": "https://enterprise.dejacode.com/urn/urn:dje:license:proprietary",
"spdx_license_key": "",
"spdx_url": "",
"start_line": 3,
"end_line": 3,
"matched_rule": {
"identifier": "proprietary_8.RULE",
"license_choice": false,
"licenses": [
"proprietary"
]
}
}
],
"copyrights": [],
"packages": []
}
]
}

Maybe, I run scancode somehow incorrectly?

[pombredanne]

@scandinaf710 Thanks for testing ....
There should be an item in the "package" section in you scan and it is empty. Could you provide a link or attach a zip with your composer.json?

There may be still something that needs to be fixed as there should be a proper "package" item returned when you scan your composer.json file and it looks like this is not the case...

Also here are a couple things for you to consider/understand:

ScanCode collects the information about the files in your local codebase... it never goes out and fetch extra things from the web: it will never query a remote package manager and collect information NOT in the codebase.

For instance for this composer.json you get this JSON result with a proper package item and data that are normalized (e.g. a Maven pom.xml or an NPM package.json end up normalized in the same data structure).

The dependencies are listed but not fetched. They would be scanned ONLY if they are in your codebase.

Therefore fetching (and resolving dependencies) remotely is something that is needs to be done ahead of scanning your code.

In the future this other tool is also meant to do this https://github.com/nexB/dependentcode/ and you can see some of the related discussions there: aboutcode-org/dependency-inspector#1 (and help is welcomed and needed to make it happen!)

I hope this clears things out.

[scandinaf710]

@pombredanne thank you for your explanation. Please, find attached composer file.
composer.zip

[pombredanne]

@scandinaf710 I was able to track the issue thanks to your test file and this has been merged in develop.
You can test it there

[pombredanne]

I am doing some cleanup and review as a prep for the 3.0 release... and since this is fixed in develop, I am closing this!