CSW harvester ignores numberOfRecordsReturned attribute

[etj]

If the numberOfRecordsReturned is smaller than the requested maxRecords, not all records will be retrieved.

e.g.:

2015-10-20 15:55:19,081 INFO  [XYZ] - Searching on : XYZ (61..81)
2015-10-20 15:55:19,465 DEBUG [XYZ] - Sent request GET /XXX/srv/de/csw202?request=GetRecords&service=CSW&version=2.0.2&resultType=results&namespace=xmlns(csw=http://www.opengis.net/cat/csw/2.0.2),xmlns(gmd=http://www.isotc211.org/2005/gmd)&outputFormat=application/xml&outputSchema=http://www.isotc211.org/2005/gmd&startPosition=61&maxRecords=20&elementSetName=summary&constraintLanguage=CQL_TEXT&constraint_language_version=1.1.0&typeNames=gmd:MD_Metadata

2015-10-20 15:55:19,469 DEBUG [XYZ] - Search results:
<csw:GetRecordsResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
  <csw:SearchStatus timestamp="2015-10-20T15:55:05" />
  <csw:SearchResults numberOfRecordsMatched="825" numberOfRecordsReturned="10" elementSet="summary" nextRecord="71">
--
2015-10-20 15:55:19,483 INFO  [XYZ] - Searching on : XYZ (81..101)
2015-10-20 15:55:19,895 DEBUG [XYZ] - Sent request GET /XXX/srv/de/csw202?request=GetRecords&service=CSW&version=2.0.2&resultType=results&namespace=xmlns(csw=http://www.opengis.net/cat/csw/2.0.2),xmlns(gmd=http://www.isotc211.org/2005/gmd)&outputFormat=application/xml&outputSchema=http://www.isotc211.org/2005/gmd&startPosition=81&maxRecords=20&elementSetName=summary&constraintLanguage=CQL_TEXT&constraint_language_version=1.1.0&typeNames=gmd:MD_Metadata

2015-10-20 15:55:19,898 DEBUG [XYZ] - Search results:
<csw:GetRecordsResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
  <csw:SearchStatus timestamp="2015-10-20T15:55:05" />
  <csw:SearchResults numberOfRecordsMatched="825" numberOfRecordsReturned="10" elementSet="summary" nextRecord="91">

[etj]

The harvester now uses the nextRecord attribute for retrieving the next chunk of metadata, and to find out when there are no more records to be retrieved.
numberOfRecordsReturned is only used for internal checks and logging.

[etj]

Also check these commits and related comments:

• 92f8a77 (may skip records at the end)
• 6c4b4d1 (may skip records in loop)