Confusing response to test session and its result state

[AlexThurston]

environment
DEMO

testSessionId
566292

vsId
2720997 and 2720998

Algorithm registration
N/A

Endpoint in which the error is experienced
https://demo.acvts.nist.gov/acvp/v1/testSessions/566292 GET
https://demo.acvts.nist.gov/acvp/v1/testSessions/566292/results GET

Expected behaviour
I'll start off by saying that the test session in question is expired. This might be the cause of the root issue but I believe it is still an issue none the less.

Reading the test session https://demo.acvts.nist.gov/acvp/v1/testSessions/566292 GET responds with:

{
  "createdOn": "2024-12-05T10:01:45Z",
  "expiresOn": "2025-01-05T07:10:28Z",
  "isSample": true,
  "passed": false,
  "publishable": false,
  "url": "/acvp/v1/testSessions/566292",
  "vectorSetUrls": [
    "/acvp/v1/testSessions/566292/vectorSets/2720997",
    "/acvp/v1/testSessions/566292/vectorSets/2720998"
  ]
}

Notably, ACVTS is reporting that the test session is not passed.

If I read the results of the test session https://demo.acvts.nist.gov/acvp/v1/testSessions/566292/results GET, the response is:

{
  "passed": false,
  "results": [
    {
      "status": "passed",
      "vectorSetUrl": "/acvp/v1/testSessions/566292/vectorSets/2720997"
    },
    {
      "status": "passed",
      "vectorSetUrl": "/acvp/v1/testSessions/566292/vectorSets/2720998"
    }
  ]
}

As you can see both vector sets are being registered as passed but similar to the GET on the test session the overall passed is still false which doesn't seem correct.

As I mentioned above, I'm going to guess that this is because the test session is expired, but even still, I'd expect that passed state to be independent from the expiry.

[cpu]

FWIW I've also run into this with unexpired sessions, and found this previous issue #183 that seemed to indicate it was a server-side data race 🤷

[jbrock24]

I'll start off by saying that the test session in question is expired. This might be the cause of the root issue but I believe it is still an issue none the less.

In this scenario, this is the condition that caused the TS to have passed = false. We are unable to say that a TS expired based on the spec limitations, but when it is expired, it will display the TS as passed=false. Thank you for the question @AlexThurston , it led to some great info exchange and discussion. Please let me know if you have any other questions.