[FDS-1725] Missing entityId handling testing

[BWMac]

Description:
This PR follows up on #1456, where the filenameExists validation rule was implemented. This rule handles cases where a file path present in a manifest does not exist in a dataset and cases where the entityId provided in a manifest row does not match with its corresponding file path.

This PR adds additional error handling for situations including:

• The entityId field in a manifest row is empty
• The entityId provided in a manifest row does not exist in the dataset

Previously, an outer join was used in the validation rule followed by a line that handled cases where there were more rows in the dataset than the manifest. The outer join implementation did not always maintain the order of manifest rows resulting in incorrect errors on manifest rows. I changed this outer join to a left join on the manifest to handle both issues.

The row number indexing for filename_validation is also updated to match the other validation rules for consistency.

I have also updated generate_filename_error to handle the new error cases.

Testing:

• New integration test cases for filename_validation have been added as needed.
• Unit tests for filename_validation and generate_filename_error have been added

[BryanFauble]

🔥

[GiaJordan]

For the new cases where we're checking if entityIds don't exist at all or are missing from the manifest we should also add validation error messages to generate_filename_error that specify to the user what the issue with their metadata was. And then the validation integration test should be updated to check for those error messages for the test cases you add to the manifest

[BWMac]

Thanks for the comments @GiaJordan! I am going to revert this back to being a draft for now and work on those updates.

[linglp]

This is looking very good! I left a few comments. But I have no problems having it pre-approved

[linglp]

@BWMac sorry one more comment. Since there's no test related to CLI, I could test manually. Give me some time :) I will report back shortly.

[linglp]

@BWMac thanks for your patience. I tested the validation CLI with my own test manifest and it works. And I also realized that we don't have "dataset scope" parameter added for manifest submission so we can't use the file name validation during submission. But this issue is definitely outside of your PR! Thanks for your hard work. After the tests are all passing, I think you could merge 👍🏼

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

Approved by other code owners