Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assemble AD2CP timestamp with nanosecond precision #1436

Merged
merged 3 commits into from
Feb 2, 2025

Conversation

leewujung
Copy link
Member

@leewujung leewujung commented Feb 2, 2025

Previously when assembling timestamp for AD2CP files, the precision was not explicitly set and defaulted to us. This causes issues with encoding times in xarray: When the time is encoded with us, coding.times.encode_cf_datetime returns negative values, which are then converted to NaT in coding.times.decode_cf_datetime and causes whole bunch tests failing seen in here.

This PR addresses this issue by explicitly set precision to ns when assembling timestamp in parse_ad2cp.py::Ad2cpDataPacket.timestamp.

The encode-decode code block:

else:
# fmt: off
encoded_data, _, _ = coding.times.encode_cf_datetime(
da, **{
"units": DEFAULT_TIME_ENCODING["units"],
"calendar": DEFAULT_TIME_ENCODING["calendar"],
}
)
# fmt: on
return coding.times.decode_cf_datetime(
encoded_data,
**{
"units": DEFAULT_TIME_ENCODING["units"],
"calendar": DEFAULT_TIME_ENCODING["calendar"],
},
)

@codecov-commenter
Copy link

codecov-commenter commented Feb 2, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.66%. Comparing base (9f56124) to head (ec15e95).
Report is 166 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1436      +/-   ##
==========================================
- Coverage   83.52%   80.66%   -2.86%     
==========================================
  Files          64       19      -45     
  Lines        5686     3108    -2578     
==========================================
- Hits         4749     2507    -2242     
+ Misses        937      601     -336     
Flag Coverage Δ
unittests 80.66% <ø> (-2.86%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@leewujung leewujung merged commit 9132303 into OSOceanAcoustics:main Feb 2, 2025
8 checks passed
leewujung added a commit that referenced this pull request Feb 2, 2025
* Drop Ping Time Duplicates (#1382)

* init commit

* revert change to fix merge conflict

* test only one file

* use other file

* move test duplicate to test convert ek

* add extra line

* move test back to ek80 convert

* pin zarr and add check unique ping time duplicates and tests

* fix test message

* test remove zarr pin

* add back zarr pin

* Fix tests that fail from new Xarray variable and attribute assignment updates (#1433)

* add .values to change water level scalar value

* add .values to fillna

* fix assign attributes

* remove test1

* update echodata properly

* remove print statement

* Assemble AD2CP timestamp with nanosecond precision (#1436)

* assemble ad2cp timestamp at ns instead of us

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add noqa

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Use `import_resources.files` instead of the legacy `open_text` (#1434)

* use import_resources.files to open yaml files

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* import from importlib.resources

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* add exception for "anc"

---------

Co-authored-by: Caesar Tuguinay <87830138+ctuguinay@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
oftfrfbf pushed a commit to oftfrfbf/echopype that referenced this pull request Feb 5, 2025
…1436)

* assemble ad2cp timestamp at ns instead of us

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add noqa

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
leewujung added a commit that referenced this pull request Feb 20, 2025
…ll tests ci] (#1447)

* EP-1420 resolved path issue with ek60_missing_channel_power

* EP-1420 fixed path to use string concatenation

* formatting

* adding conditionals for checking platform nmea and exchange from time1 to time_nmea

* working datatree for v0.9.0 w o calibration

* Drop Ping Time Duplicates (#1382)

* init commit

* revert change to fix merge conflict

* test only one file

* use other file

* move test duplicate to test convert ek

* add extra line

* move test back to ek80 convert

* pin zarr and add check unique ping time duplicates and tests

* fix test message

* test remove zarr pin

* add back zarr pin

* Fix tests that fail from new Xarray variable and attribute assignment updates (#1433)

* add .values to change water level scalar value

* add .values to fillna

* fix assign attributes

* remove test1

* update echodata properly

* remove print statement

* Assemble AD2CP timestamp with nanosecond precision (#1436)

* assemble ad2cp timestamp at ns instead of us

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add noqa

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Use `import_resources.files` instead of the legacy `open_text` (#1434)

* use import_resources.files to open yaml files

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* import from importlib.resources

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* bump codespell version, add exceptions (#1438)

* Pin `zarr` and `netcdf4` temporarily [all tests ci] (#1429)

* pin zarr and netcdf4 in requirements.txt

* change np.alltrue to np.all

* use np.argmin(da.data) instead of da.argmin()

* use S instead s for MVBS ping_time_bin

* extract single element in computing nsamples and L in tapered_chirp

* change from S to s in testing.py

* Add `type-extensions` to requirements.txt (#1440)

* add type-extensions

* sort requirements.txt

* add manual trigger to pypi workflow (#1442)

* update cff with ices paper data (#1443)

* add assign actual range utility function (#1435)

* chore(deps): bump actions/setup-python from 5.3.0 to 5.4.0 (#1445)

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.3.0 to 5.4.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@v5.3.0...v5.4.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [pre-commit.ci] pre-commit autoupdate (#1446)

* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/PyCQA/isort: 5.13.2 → 6.0.0](PyCQA/isort@5.13.2...6.0.0)
- [github.com/psf/black: 24.10.0 → 25.1.0](psf/black@24.10.0...25.1.0)
- [github.com/codespell-project/codespell: v2.4.0 → v2.4.1](codespell-project/codespell@v2.4.0...v2.4.1)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* added pooch and bound the test registry to the echodata test

* EP-1420 added legacy test files >=v0.8.4, created pooch registry, parameterized test

* EP-1420 checked tests

* EP-1420 debugging TestEchoData test failures, test_nbytes, test_setattr, & test_setitem

* EP-1447 troubleshooting bug still

* EP-1420 removed pooch and added normal test path resources

* EP-1420 fixed test_nbytes, commented out test_setattr

* EP-1420 changed "time_nmea" to "nmea_time", added ek80 legacy_datatree tests, fixed readme

* EP-1420 removed import

* trying to add sonar channel_all

* EP-1420 checked sonar for channel and renamed to channel_all

* check legacy data differently

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove v0.9.2 zarr test files

* remove test_setattr that is no longer needed, see #1457 that removes EchoData.__setattr__

* Update echopype/tests/echodata/test_echodata.py

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Caesar Tuguinay <87830138+ctuguinay@users.noreply.github.com>
Co-authored-by: Wu-Jung Lee <leewujung@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@leewujung leewujung added this to the v0.10.0 milestone Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants