Guarantee dataset properties in order in dataset-diff

[ktuite]

Dataset properties are returned and shown in alphabetical order on the frontend, but it might be clearer for form designers if these properties match the order in which they are found in the form.

This is related to Issue #675 about guaranteeing property order in the CSV export. Whatever ordering we end up using there, we should use in the dataset-diff endpoint as well.

[sadiqkhoja]

Have to fix it for Dataset Metadata endpoint as well

[lognaturel]

How much does the actual order matter?

Could sorting first by publishedAt and then by database id work? Are properties inserted in form order currently?

[ktuite]

See this comment about possibly removing schemaId from ds_property_fields in this refactor. Didn't remove it here but added it as an issue #943

[ktuite]

@getodk/testers there is more detail about this issue in this PR: #933

the three places you'll see these properties in their new order (matching the order in which they appear in the form) are

1. form draft tab
2. form publish modal
3. form detail page

The weird edge cases include:

• moving fields around in your form
• adding/removing or changing the property names of savetos to these fields
• deleting a saveto from a field or deleting the field overall and having it not appear in the list of properties on the form, but having older versions of the form still able to create entities with those properties

[dbemke]

I’m not sure what should be the expected result in those situations connected with moving fields in a form

• if I moved a property field in a form (e.g it was the last property in the previous version of a form, then I moved it as the first one in the following version of the form) should the order of the properties change in the CSV of the dataset?
• If I moved the property field should there be an information about the fact that the field was moved in the publish draft dialog?

[ktuite]

if I moved a property field in a form (e.g it was the last property in the previous version of a form, then I moved it as the first one in the following version of the form) should the order of the properties change in the CSV of the dataset?

Overall, like in the CSV export and in the dataset property page, the order of the properties should not change if a property gets moved around in later versions of the form. We're trying to make the order of the properties as consistent over time as possible.

However... in this property list on a form status page, I'm trying to make it match the order in the current version form. It may not be the same order as the export order, but the properties here may not contain the full list of dataset properties anyway. Is that the right thing to do? I'm not sure, but feels better than alphabetical order, and I'm hoping it might help someone who is checking the properties in the latest version of the form.

If I moved the property field should there be an information about the fact that the field was moved in the publish draft dialog?

Because the properties within a dataset can come from multiple forms, it's more complex than fields within a form, and it doesn't really matter if the order changes in later versions of different forms. We'll order the properties by

• order of how they were in the first published version of a form
• additive, so forms that introduce new properties will have those added at the end, and forms that reference existing properties won't change the order of those properties
• don't remove properties from the dataset even if they get removed from a form

[dbemke]

Testes with success!

[srujner]

Testes with success!