Skip to content

Commit

Permalink
Upgrade to Data Package 2.0
Browse files Browse the repository at this point in the history
* Add $schema attributes
* Bump frictionless to 5.18

See #60.
  • Loading branch information
ezwelty committed Feb 7, 2025
1 parent f7d5b8a commit acde3fb
Show file tree
Hide file tree
Showing 5 changed files with 32 additions and 17 deletions.
22 changes: 16 additions & 6 deletions datapackage.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -143,20 +143,22 @@ contributors:
- name: Department of Geosciences, University of Fribourg (UniFr), Switzerland
role: DataCollector
status: creator
profile: tabular-data-package
$schema: https://datapackage.org/profiles/2.0/datapackage.json
resources:
- name: source
description: Sources of information considered in the compilation of this database. Column names and categorical values closely follow the Citation Style Language (CSL) 1.0.2 specification. Names of people in non-Latin scripts are followed by a latinization in square brackets (e.g. В. С. Загороднов [V. S. Zagorodnov]) and non-English titles are followed by a translation in square brackets. The family name of Latin-script names is wrapped in curly braces when it is not the last word of the name (e.g. Emmanuel {Le Meur}, e.g. {Duan} Keqin) or the name ends in two or more unabbreviated words (e.g. Jon Ove {Hagen}). The family name of a Chinese name (and of the latinization) is wrapped in curly braces when it is not the first character.
path: data/source.csv
format: csv
dialect:
$schema: https://datapackage.org/profiles/2.0/tabledialect.json
header: true
delimiter: ','
lineTerminator: "\n"
quoteChar: '"'
doubleQuote: true
profile: tabular-data-resource
$schema: https://datapackage.org/profiles/2.0/dataresource.json
schema:
$schema: https://datapackage.org/profiles/2.0/tableschema.json
fields:
- name: id
description: Unique identifier constructed from the first author's lowercase, latinized, family name and the publication year, followed as needed by a lowercase letter to ensure uniqueness (e.g. Загороднов 1981 → zagorodnov1981a).
Expand Down Expand Up @@ -295,13 +297,15 @@ resources:
path: data/borehole.csv
format: csv
dialect:
$schema: https://datapackage.org/profiles/2.0/tabledialect.json
header: true
delimiter: ','
lineTerminator: "\n"
quoteChar: '"'
doubleQuote: true
profile: tabular-data-resource
$schema: https://datapackage.org/profiles/2.0/dataresource.json
schema:
$schema: https://datapackage.org/profiles/2.0/tableschema.json
fields:
- name: id
description: Unique identifier.
Expand Down Expand Up @@ -465,13 +469,15 @@ resources:
- data/bravo2015-pascua-lama/profile.csv
format: csv
dialect:
$schema: https://datapackage.org/profiles/2.0/tabledialect.json
header: true
delimiter: ','
lineTerminator: "\n"
quoteChar: '"'
doubleQuote: true
profile: tabular-data-resource
$schema: https://datapackage.org/profiles/2.0/dataresource.json
schema:
$schema: https://datapackage.org/profiles/2.0/tableschema.json
fields:
- name: borehole_id
description: Borehole identifier.
Expand Down Expand Up @@ -559,13 +565,15 @@ resources:
- data/bravo2015-pascua-lama/measurement.csv
format: csv
dialect:
$schema: https://datapackage.org/profiles/2.0/tabledialect.json
header: true
delimiter: ','
lineTerminator: "\n"
quoteChar: '"'
doubleQuote: true
profile: tabular-data-resource
$schema: https://datapackage.org/profiles/2.0/dataresource.json
schema:
$schema: https://datapackage.org/profiles/2.0/tableschema.json
fields:
- name: borehole_id
description: Borehole identifier.
Expand Down Expand Up @@ -601,13 +609,15 @@ resources:
path: data/person.csv
format: csv
dialect:
$schema: https://datapackage.org/profiles/2.0/tabledialect.json
header: true
delimiter: ','
lineTerminator: "\n"
quoteChar: '"'
doubleQuote: true
profile: tabular-data-resource
$schema: https://datapackage.org/profiles/2.0/dataresource.json
schema:
$schema: https://datapackage.org/profiles/2.0/tableschema.json
fields:
- name: titles
description: Author titles matching this person, as a pipe-delimited list.
Expand Down
2 changes: 1 addition & 1 deletion environment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ channels:

dependencies:
- python=3.9
- frictionless=5.13
- frictionless=5.18
- pandas=2.2
- pytest=7.3
- markdown=3.4
Expand Down
11 changes: 5 additions & 6 deletions glenglat.py
Original file line number Diff line number Diff line change
Expand Up @@ -248,11 +248,7 @@ def write_submission_yaml() -> None:
field.false_values = ['False', 'false', 'FALSE']
# --- Strip path prefixes ---
for resource in package.resources:
if isinstance(resource.path, list):
path = resource.path[0]
else:
path = resource.path
resource.path = Path(path).name
resource.path = Path(resource.path).name
resource.extrapaths = None
# --- Overwrite package metadata ---
package.name = 'glenglat-submission'
Expand All @@ -263,7 +259,10 @@ def write_submission_yaml() -> None:
package.licenses = None
package.contributors = None
package.keywords = None
package.custom = {}
package.custom = {
key: value for key, value in package.custom.items()
if key in ['$schema', 'languages']
}
# --- Validate metadata ----
metadata = package.to_dict()
# HACK: Remove spurious csv key from dialects introduced by frictionless
Expand Down
12 changes: 9 additions & 3 deletions submission/datapackage.yaml
Original file line number Diff line number Diff line change
@@ -1,19 +1,18 @@
name: glenglat-submission
description: Submission format for the global englacial temperature database.
homepage: https://github.com/mjacqu/glenglat
profile: tabular-data-package
version: 1.0.0-rc4
resources:
- name: borehole
type: table
description: Metadata about each borehole.
profile: tabular-data-resource
path: borehole.csv
scheme: file
format: csv
mediatype: text/csv
dialect:
header: true
$schema: https://datapackage.org/profiles/2.0/tabledialect.json
delimiter: ','
lineTerminator: '
Expand Down Expand Up @@ -142,16 +141,18 @@ resources:
- ''
primaryKey:
- id
$schema: https://datapackage.org/profiles/2.0/tableschema.json
$schema: https://datapackage.org/profiles/2.0/dataresource.json
- name: measurement
type: table
description: Temperature measurements with depth.
profile: tabular-data-resource
path: measurement.csv
scheme: multipart
format: csv
mediatype: text/csv
dialect:
header: true
$schema: https://datapackage.org/profiles/2.0/tabledialect.json
delimiter: ','
lineTerminator: '
Expand Down Expand Up @@ -217,3 +218,8 @@ resources:
resource: borehole
fields:
- id
$schema: https://datapackage.org/profiles/2.0/tableschema.json
$schema: https://datapackage.org/profiles/2.0/dataresource.json
languages:
- en
$schema: https://datapackage.org/profiles/2.0/datapackage.json
2 changes: 1 addition & 1 deletion submission/environment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ channels:

dependencies:
- python=3.9
- frictionless=5.13
- frictionless=5.18

0 comments on commit acde3fb

Please sign in to comment.