Skip to content

Commit

Permalink
Run frictionless validate as pytest test
Browse files Browse the repository at this point in the history
Frictionless Repository has been failing to run in GitHub CI for an unknown reason.
This avoids the problem and simplifies testing.
  • Loading branch information
ezwelty committed Feb 6, 2025
1 parent f7c7765 commit f7d5b8a
Show file tree
Hide file tree
Showing 4 changed files with 23 additions and 34 deletions.
20 changes: 0 additions & 20 deletions .github/workflows/frictionless.yaml

This file was deleted.

19 changes: 7 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
| :- | :- |
[Tutorial notebook](tutorial.ipynb) | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mjacqu/glenglat/blob/main/tutorial.ipynb)
Version history | [![GitHub Release](https://img.shields.io/github/v/release/mjacqu/glenglat?label=latest)](https://github.com/mjacqu/glenglat/releases)
Tests | [![Frictionless](https://github.com/mjacqu/glenglat/actions/workflows/frictionless.yaml/badge.svg)](https://github.com/mjacqu/glenglat/actions/workflows/frictionless.yaml) [![Pytest](https://github.com/mjacqu/glenglat/actions/workflows/pytest.yaml/badge.svg)](https://github.com/mjacqu/glenglat/actions/workflows/pytest.yaml)
Tests | [![Pytest](https://github.com/mjacqu/glenglat/actions/workflows/pytest.yaml/badge.svg)](https://github.com/mjacqu/glenglat/actions/workflows/pytest.yaml)

_Our paper is currently in public review at [Earth System Science Data](https://essd.copernicus.org). Preview it and consider reviewing it at https://essd.copernicus.org/preprints/essd-2024-249/!_

Expand Down Expand Up @@ -146,31 +146,26 @@ cp .env.example .env

### Run tests

Run the basic (`frictionless`) tests.
Run all the tests in the [`tests`](tests) folder.

```sh
frictionless validate datapackage.yaml
pytest
```

Run the custom (`pytest`) tests in the [`tests`](tests) folder.
Run only fast tests with the `--fast` option and only slow tests with the `--slow` option.

```sh
pytest
pytest --fast
pytest --slow
```

An optional test checks that `borehole.glims_id` is consistent with borehole coordinates. To run, install `geopandas` and `pyarrow` and set the `GLIMS_PATH` environment variable before calling `pytest`.
An optional (slow) test checks that `borehole.glims_id` is consistent with borehole coordinates. To run, install `geopandas` and `pyarrow` and set the `GLIMS_PATH` environment variable before calling `pytest`.

```sh
conda install -c conda-forge geopandas=0.13 pyarrow
pytest
```

Slow tests can be skipped by using the `--fast` option.

```sh
pytest --fast
```

### Maintain the repository

The [`glenglat.py`](glenglat.py) module contains functions used to maintain the repository. They can be run from the command line as `python glenglat.py {function}`.
Expand Down
10 changes: 8 additions & 2 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@


def pytest_addoption(parser: pytest.Parser) -> None:
parser.addoption('--fast', action='store_true', default=False, help='Skip slow tests')
parser.addoption('--fast', action='store_true', default=False, help='Run fast tests')
parser.addoption('--slow', action='store_true', default=False, help='Run slow tests')


def pytest_configure(config: pytest.Config) -> None:
Expand All @@ -13,8 +14,13 @@ def pytest_collection_modifyitems(
config: pytest.Config,
items: list[pytest.Item]
) -> None:
if config.getoption('--fast'):
if config.getoption('--fast') and not config.getoption('--slow'):
skip_slow = pytest.mark.skip(reason='--fast skips slow tests')
for item in items:
if 'slow' in item.keywords:
item.add_marker(skip_slow)
elif config.getoption('--slow') and not config.getoption('--fast'):
skip_fast = pytest.mark.skip(reason='--slow skips fast tests')
for item in items:
if 'slow' not in item.keywords:
item.add_marker(skip_fast)
8 changes: 8 additions & 0 deletions tests/test_data.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import frictionless
import numpy as np
import pandas as pd
import pytest
Expand All @@ -12,6 +13,13 @@
import glenglat


@pytest.mark.slow
def test_datapackage_is_valid() -> None:
"""Frictionless datapackage is valid."""
report = frictionless.validate('datapackage.yaml')
assert report.valid, report.to_summary()


# ---- source ----

def test_profile_date_min_less_than_date_max() -> None:
Expand Down

0 comments on commit f7d5b8a

Please sign in to comment.