As a paginator, I want endpoints to consistently return no results when I page to the end of results

[nutjob4life]

Various API endpoints like the /collections/{lidvid}/products let you paginate through your results in a fairly direct manner:

1. Set your start to zero and your limit to 20.
2. Get back 20 results, so increment start to 20 and try again.
3. Get back 2 results, so increment start to 22 and try again.
4. Get back zero results; this indicates you've reached the end.

For example, try this:

1. curl --header 'Accept: application/json' 'https://pds-gamma.jpl.nasa.gov/api/collections/urn%3Anasa%3Apds%3Ainsight_documents%3Adocument_hp3rad%3A%3A8.0/products?start=0&limit=20&fields=product_class&only-summary=false' | json_pp and note that the data field has 2 entries in it.
2. Now bump the start to 2 try curl --header 'Accept: application/json' 'https://pds-gamma.jpl.nasa.gov/api/collections/urn%3Anasa%3Apds%3Ainsight_documents%3Adocument_hp3rad%3A%3A8.0/products?start=2&limit=20&fields=product_class&only-summary=false' | json_pp and you get this nice little result:

{
   "summary" : {
      "limit" : 20,
      "properties" : [],
      "sort" : [],
      "start" : 2
   }
}

🍻 Perfect!

This strategy worked with the various other endpoints too, like /bundles/{lidvid}/collections. Or it used to. Now (possibly related to the fix for NASA-PDS/pds-api#73) this no longer works for /bundles/{lidvid}/collections. Instead of returning zero results, it gives a 500 Internal Server Error.

For example, try this:

1. curl --header 'Accept: application/json' 'https://pds-gamma.jpl.nasa.gov/api/bundles/urn%3Anasa%3Apds%3Ainsight_documents%3A%3A2.0/collections?start=0&limit=20&fields=product_class&only-summary=false' | json_pp. Note the data parameter has 5 entries.
2. Now bump the start to 5 and do curl --header 'Accept: application/json' 'https://pds-gamma.jpl.nasa.gov/api/bundles/urn%3Anasa%3Apds%3Ainsight_documents%3A%3A2.0/collections?start=5&limit=20&fields=product_class&only-summary=false' | json_pp. Now you get this:

{
   "error" : "Internal Server Error",
   "message" : "",
   "path" : "/bundles/urn:nasa:pds:insight_documents::2.0/collections",
   "status" : 500,
   "timestamp" : 1631137572150
}

😭 What happened? This used to work.

On a possibly related note, asking for zero items (limit=0) also used to work; you'd get back a data entry that was an empty array, which is what you'd expect. Now you get 500 Internal Server Error—and on multiple endpoints. I think you should be able to ask for nothing and get nothing, which is not an error condition 😅

[jordanpadams]

@tdddblog we can talk about this at tomorrow's breakout, but I would like to add this to the top of the queue since we can't really ship this API next week without being able to paginate results