File Level Metadata - Tabular Metadata

[djbrooke]

Tabular files have additional metadata that is created during the ingest process. Some of this metadata should be shown, along with a preview, on the File Level Landing Page (#2465).

[djbrooke]

• Need to check with @landreev - the metadata is generated from a tabular file and outputted in a standard format
• We should check with DSS about what the most valuable items are to show here

[djbrooke]

@scolapasta and I met with DSS yesterday. They were very helpful in providing information about ways we could improve the display of tabular metadata on the File Landing Page (#2465). We walked away with three pieces of feedback:

• It would be helpful to provide the Variable Metadata in a human-readable format. It is currently downloaded as DDI XML and is decidedly not human-readable. :)
• Information about missing data on a per-variable basis will allow users to easily see if specific variables within the file have the information they are looking for. @scolapasta mentioned that in 3.x we had the concept of "Invalid" and "Valid" data that could be investigate for use here.
• A "representation" or "overview" of the files contents (DSS encouraged me not to use the term "preview") similar to the attached image.

• Variables on left side instead of across the top (to prevent the endless horizontal scroll
• A label or other descriptor next to each variable
• The first ~5 values for each variable

I thanked the DSS folks and told them that this will provide a good basis for generating more feedback from the community.

[mercecrosas]

Consider also embedding the DDI Explorer (renamed Data Explorer) developed by ScholarsPortal in the file page for a tabular file. This would give a good preview/summary of the data without having to download the data values.

[raprasad]

(tangentially metadata related)

basic tabular preview example using file id 2966861:

• basic/basic table: https://services.dataverse.harvard.edu/miniverse/datafile/table-preview-html/2966861
• json: https://services.dataverse.harvard.edu/miniverse/datafile/table-preview-json/2966861

Add any id of a published file to the urls above

---

[pdurbin]

The comments above were all made before external tools were shipped in PR #4298.

These days, we very likely say that this should be an external tool, a preview tool. So I'd suggest anyone who is interested in this should create an issue at https://github.com/gdcc/dataverse-previewers

Meanwhile, it's probably safe to close this issue. (I do like the idea and think it would be useful!) 😄

[pdurbin]

Related:

• Custom field per dataset and also dataset file #6455
• Customized Metadata at Datafile Level #7930
• Feature Request/Idea: custom metadata fields for files #8240

[sbarbosadataverse]

Closed per Phil's followup above

[qqmyers]

FWIW: There is a tabular previewer these days. I don't think it uses the extracted file-level metadata though (it just shows a tabular view of the file contents), so that could be an improvement.