Unstable connection to nist database

[smiklosovic]

Hi @jeremylong ,

I am from Apache Cassandra project and we would like to use Ant plugin as part of our build pipeline.

It works but every now and then, we are getting this error and it makes it pretty unstable to run the builds.

Are we doing something wrong or nist is unstable?

This is our setup: https://github.com/apache/cassandra/blob/trunk/.build/build-owasp.xml

Thanks

/home/jenkins/jenkins-slave/workspace/Cassandra-devbranch-artifacts/jdk/jdk_1.8_latest/label/cassandra/.build/build-owasp.xml:80: One or more exceptions occurred during analysis:
15:14:52 org.owasp.dependencycheck.exception.ExceptionCollection: One or more exceptions occurred during analysis:
15:14:52 UpdateException: Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta
15:14:52 caused by DownloadFailedException: Download failed, unable to retrieve 'https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta'; Error downloading file https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta; unable to connect.
15:14:52 caused by DownloadFailedException: Error downloading file https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta; unable to connect.
15:14:52 caused by DownloadFailedException: Error retrieving https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta; received response code 503; Service Unavailable
15:14:52 NoDataException: No documents exist
15:14:52 at org.owasp.dependencycheck.Engine.throwFatalExceptionCollection(Engine.java:1103)
15:14:52 at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:612)
15:14:52 at org.owasp.dependencycheck.taskdefs.Check.callExecuteAnalysis(Check.java:1844)
15:14:52 at org.owasp.dependencycheck.taskdefs.Check.execute(Check.java:1801)
15:14:52 at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
15:14:52 at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
15:14:52 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
15:14:52 at java.lang.reflect.Method.invoke(Method.java:498)
15:14:52 at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
15:14:52 at org.apache.tools.ant.Task.perform(Task.java:350)
15:14:52 at org.apache.tools.ant.Target.execute(Target.java:449)
15:14:52 at org.apache.tools.ant.Target.performTasks(Target.java:470)
15:14:52 at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388)
15:14:52 at org.apache.tools.ant.Project.executeTarget(Project.java:1361)
15:14:52 at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
15:14:52 at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
15:14:52 at org.apache.tools.ant.Main.runBuild(Main.java:834)
15:14:52 at org.apache.tools.ant.Main.startAnt(Main.java:223)
15:14:52 at org.apache.tools.ant.launch.Launcher.run(Launcher.java:284)
15:14:52 at org.apache.tools.ant.launch.Launcher.main(Launcher.java:101)

[williamroboly]

I get the same output using the docker approach.

[ERROR] Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2006.meta

Thoughts?

[jeremylong]

This is unfortunately caused by the NVD's rate limiting that was enabled in September. I am currently working on a resolution that will include increased wait times and additional data caching. One of the best options for people using the CLI, Ant, or docker image would be to switch to using the owasp/dependency-check-action:latest docker image. This is built nightly and contains a full up-to-date dependency-check database. Other then having an up-to-date database it works exactly like the standard ODC docker image.

[jeremylong]

We should have a new release within 1-2 days that will resolve the issue with downloading from the NVD.

[jeremylong]

ODC 6.4.1 was released and should resolve the issues you were facing.

[smiklosovic]

We should have a new release within 1-2 days that will resolve the issue with downloading from the NVD.

Awesome news! Thank you very much. Can't wait to hook this stuff into our pipeline again.

[darmbrust]

Just a note - we are still seeing these random download failures with 6.5.3, in a docker build pipeline.
Can you give more specifics on how to get the pre-built DB into our own docker image?

[jeremylong]

Sorry for the delayed response - you can use: https://hub.docker.com/r/owasp/dependency-check-action

It is built nightly and contains the full NVD database.