Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ort scan: No module named 'virtualenv.seed.embed.via_app_data' #4469

Closed
alex4200 opened this issue Sep 16, 2021 · 9 comments
Closed

ort scan: No module named 'virtualenv.seed.embed.via_app_data' #4469

alex4200 opened this issue Sep 16, 2021 · 9 comments
Labels
needs info An issue where further information is required scanner About the scanner tool

Comments

@alex4200
Copy link

I tried to run an ort scan on the output of an analyze file like

./ort/cli/build/install/ort/bin/ort scan -i output/analyzer-result.yml -o output2/

and got the following output:

________ _____________________
\_____  \\______   \__    ___/ the OSS Review Toolkit, version 7bb799b.
 /   |   \|       _/ |    |
/    |    \    |   \ |    |    Running 'scan' under Java 11.0.11 on Linux with
\_______  /____|_  / |____|    8 CPUs and a maximum of 3978 MiB of memory.
        \/       \/
Environment variables:
ORT_CONFIG_DIR = /home/adietz/.ort/config
ORT_DATA_DIR = /home/adietz/.ort
SHELL = /bin/bash
TERM = xterm-256color
JAVA_HOME = /usr/lib/jvm/java-11-openjdk-amd64

Using scan storage 'FileBasedStorage with XZCompressedLocalFileStorage backend'.
Local file storage has 0 scan results file(s).
Using scanner 'ScanCode'.
Exception in thread "main" java.io.IOException: Running '/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/scancode --version' in '/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2' failed with exit code 127:
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz/__main__.py", line 169, in <module>
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz/__main__.py", line 165, in run
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz/virtualenv/__main__.py", line 20, in run
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz/virtualenv/run/__init__.py", line 27, in cli_run
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz/virtualenv/run/__init__.py", line 35, in session_via_cli
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz/virtualenv/run/__init__.py", line 76, in build_parser
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz/virtualenv/run/plugin/seeders.py", line 8, in __init__
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz/virtualenv/run/plugin/base.py", line 39, in options
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz/virtualenv/run/plugin/base.py", line 18, in entry_points_for
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz/virtualenv/run/plugin/base.py", line 18, in <genexpr>
  File "/usr/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'virtualenv.seed.embed.via_app_data'
Traceback (most recent call last):
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/etc/configure.py", line 560, in <module>
    create_virtualenv(standard_python, root_dir, thirdparty_dirs, quiet=quiet)
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/etc/configure.py", line 266, in create_virtualenv
    call(vcmd, root_dir)
  File "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/etc/configure.py", line 158, in call
    subprocess.check_call(cmd, shell=True, env=dict(os.environ), cwd=root_dir)
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '"/usr/bin/python" "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/virtualenv.pyz" --never-download -qq --extra-search-dir="/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty/dev" --extra-search-dir="/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/thirdparty" "/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2"' returned non-zero exit status 1.
/tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/scancode: line 114: /tmp/ort-ScanCode-3.2.1-rc29111017345328723843/scancode-toolkit-3.2.1rc2/bin/scancode: No such file or directory

	at org.ossreviewtoolkit.utils.ProcessCapture.requireSuccess(ProcessCapture.kt:147)
	at org.ossreviewtoolkit.utils.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:75)
	at org.ossreviewtoolkit.scanner.LocalScanner.run(LocalScanner.kt:76)
	at org.ossreviewtoolkit.utils.CommandLineTool$DefaultImpls.getVersion(CommandLineTool.kt:81)
	at org.ossreviewtoolkit.scanner.LocalScanner.getVersion(LocalScanner.kt:76)
	at org.ossreviewtoolkit.scanner.scanners.scancode.ScanCode.getVersion(ScanCode.kt:228)
	at org.ossreviewtoolkit.scanner.LocalScanner$scannerDir$2.invoke(LocalScanner.kt:128)
	at org.ossreviewtoolkit.scanner.LocalScanner$scannerDir$2.invoke(LocalScanner.kt:115)
	at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
	at org.ossreviewtoolkit.scanner.LocalScanner.getScannerDir(LocalScanner.kt:115)
	at org.ossreviewtoolkit.scanner.LocalScanner.access$getScannerDir(LocalScanner.kt:76)
	at org.ossreviewtoolkit.scanner.LocalScanner$version$2.invoke(LocalScanner.kt:164)
	at org.ossreviewtoolkit.scanner.LocalScanner$version$2.invoke(LocalScanner.kt:164)
	at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
	at org.ossreviewtoolkit.scanner.LocalScanner.getVersion(LocalScanner.kt:164)
	at org.ossreviewtoolkit.scanner.LocalScanner$details$2.invoke(LocalScanner.kt:174)
	at org.ossreviewtoolkit.scanner.LocalScanner$details$2.invoke(LocalScanner.kt:174)
	at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
	at org.ossreviewtoolkit.scanner.LocalScanner.getDetails(LocalScanner.kt:174)
	at org.ossreviewtoolkit.scanner.LocalScanner.scan(LocalScanner.kt:269)
	at org.ossreviewtoolkit.scanner.LocalScanner.scanPackages$suspendImpl(LocalScanner.kt:231)
	at org.ossreviewtoolkit.scanner.LocalScanner.scanPackages$scanner(LocalScanner.kt)
	at org.ossreviewtoolkit.scanner.ScannerKt$scanOrtResult$scanResults$1$deferredProjectScan$1.invokeSuspend(Scanner.kt:113)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at org.ossreviewtoolkit.scanner.ScannerKt.scanOrtResult(Scanner.kt:110)
	at org.ossreviewtoolkit.cli.commands.ScannerCommand.run(ScannerCommand.kt:173)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:204)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:213)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:17)
	at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:396)
	at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:393)
	at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:411)
	at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:436)
	at org.ossreviewtoolkit.cli.OrtMainKt.main(OrtMain.kt:108)
@sschuberth
Copy link
Member

The line "/usr/lib/python3.8/subprocess.py" seems to indicate you're using a too new Python version for the outdated ScanCode version ORT (still) uses by default. IIRC ScanCode 3.2.1 supports at most Python 2.7.

@sschuberth sschuberth added needs info An issue where further information is required scanner About the scanner tool labels Sep 17, 2021
@alex4200
Copy link
Author

So I cannot use ort only with python2?

@sschuberth
Copy link
Member

sschuberth commented Sep 17, 2021 •
edited
Loading

On the contrary, you can use ORT with Python 2, and the default version of ScanCode also requires it; but using ORT with Python 3 is a bit of work, currently. You may have luck using the Docker image from this PR.

@alex4200
Copy link
Author

I do not see any instructions on how to use the docker image from the PR you have shared. Maybe you can give the explicit command line instructions to use/download this image?

@sschuberth
Copy link
Member

sschuberth commented Sep 17, 2021 •
edited
Loading

It's not a Docker image, it's a Dockerfile, and it can be build in the same way as described in the README.

@alex4200
Copy link
Author

Yes, sure. How do I checkout this specific pull request?

@sschuberth
Copy link
Member

sschuberth commented Sep 17, 2021 •
edited
Loading

Please consult the GitHub docs for basic GitHub usage questions. I recommend using the GitHub CLI:

image

@alex4200
Copy link
Author

alex4200 commented Sep 17, 2021 •
edited
Loading

Ok, I have to create a bug report with github CLI then.

gh pr checkout 3902 does not seem to exist: Error: No such command 'pr'.
git checkout pr/3902 gives error: pathspec 'pr/3902' did not match any file(s) known to git

Maybe I have to open, copy and paste each file on its own?

@alex4200
Copy link
Author

Ok, I finally was able to create the Dockerfile.

I had the wrong gh installed. It is very complicated. Closing ticket

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
needs info An issue where further information is required scanner About the scanner tool
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sschuberth @alex4200