Added initial version of an MSG BUFR reader and TOZ product yaml file #769

ColinDuff · 2019-05-16T20:48:54Z

I have added a BUFR reader , currently for MSG BUFR products.

No test case for now as its still in the initial stages of development

ColinDuff · 2019-05-16T21:08:27Z

Please note this reader requires the installation of the python-eccodes package

…- project header added

…heck

codecov · 2019-05-17T16:19:03Z

Codecov Report

Merging #769 into master will increase coverage by 0.02%.
The diff coverage is 91.89%.

@@            Coverage Diff             @@
##           master     #769      +/-   ##
==========================================
+ Coverage   86.61%   86.63%   +0.02%     
==========================================
  Files         175      177       +2     
  Lines       26822    26948     +126     
==========================================
+ Hits        23231    23346     +115     
- Misses       3591     3602      +11

Impacted Files	Coverage Δ
satpy/tests/reader_tests/__init__.py	`98.27% <100%> (+0.03%)`	⬆️
satpy/readers/seviri_l2_bufr.py	`91.07% <91.07%> (ø)`
satpy/tests/reader_tests/test_seviri_l2_bufr.py	`92.59% <92.59%> (ø)`
satpy/tests/writer_tests/test_cf.py	`98.37% <0%> (-0.2%)`	⬇️
satpy/writers/cf_writer.py	`91.24% <0%> (-0.2%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 283ca90...35b7441. Read the comment docs.

coveralls · 2019-05-17T16:19:07Z

Coverage increased (+0.03%) to 86.634% when pulling 35b7441 on ColinDuff:msg_bufr_reader into 283ca90 on pytroll:master.

mraspaud · 2019-09-18T22:34:14Z

@ColinDuff could you add some tests to this ?

ColinDuff · 2019-09-19T07:15:24Z

@mraspaud , hi , been busy with a lot of MPEF MSG work but now have the time to look at testing this change - hope to have it completed soon

mraspaud · 2019-09-19T07:16:41Z

Sounds great, no rush :)

ColinDuff · 2019-10-01T12:19:42Z

i have added some more seviri L2 Bufr products

sjoro

More yaml-file clean-up needed. Wrong reference documents given for seviri_l2_bufr-reader.
Also, the standard names, now mostly having count in YAML-files need to be checked against:
http://cfconventions.org/Data/cf-standard-names/68/build/cf-standard-name-table.html

satpy/readers/seviri_l2_bufr.py

satpy/etc/readers/seviri_l2_bufr_asr.yaml

mraspaud

Thanks for putting the effort of reading bufr files, it's no small feat!

Here are my first comments/questions on top of the inline review:

Do we really need that many readers ? One should suffice...
You should add the reader(s) to the reader table in doc/index.rst
Please run pre-commit on the PR and fix the styling issues

satpy/readers/seviri_l2_bufr.py

satpy/tests/reader_tests/test_seviri_l2_bufr.py

satpy/readers/seviri_l2_bufr.py

…into msg_bufr_reader

ColinDuff · 2019-10-24T15:15:58Z

based on the review comments , in particular the opening of the file twice i realised the get_attributes function was working as i had hoped. As it was only used to determine the segment size a dictionary can be used instead.

sjoro

resolution for the the products is in my opinion wrong, all of the products are segmented, resolution should reflect the segment size.

satpy/etc/readers/seviri_l2_bufr.yaml

…dard names changed.

…into msg_bufr_reader

sjoro

looks good to me!

mraspaud

LGTM. Great job on adding all these products in one got !
Maybe the next step is a PR on pytroll-examples to show the users how to use this ?

…ror in setup.py, python-eccodes is the correct dependency

…dependency

… values

… no mapping/sampling takes place

sjoro

seg_area_dict is not needed in seviri_l2_bufr.py, please remove. otherwise LGTM.

sjoro

LGTM

mraspaud

LGTM

Added initial version of an MSG BUFR reader and TOZ product yaml file

e6f6213

ColinDuff requested review from djhoese and mraspaud as code owners May 16, 2019 20:48

mraspaud assigned ColinDuff May 16, 2019

mraspaud added component:readers enhancement code enhancements, features, improvements labels May 16, 2019

Colin.Duff@eumetsat.int added 4 commits May 16, 2019 23:14

Added initial version of an MSG BUFR reader and TOZ product yaml file

5207aef

Added initial version of an MSG BUFR reader and TOZ product yaml file

186be9d

Added initial version of an MSG BUFR reader and TOZ product yaml file…

d9778fe

…- project header added

Updated seviri_l2_bufr_toz.yaml due to some errors raised during PR c…

3823289

…heck

mraspaud added the PCW Pytroll Contributors' Week label May 21, 2019

Colin.Duff@eumetsat.int added 5 commits September 26, 2019 14:27

Added test script for seviri l2 bufr reader

e30e5ca

Added test script for seviri l2 bufr reader

c8bbb52

Added new seviri bufr product yaml files and updated toz yaml file

bf0ea1e

Added new seviri bufr product yaml files and updated toz yaml file

f9a9154

Added new seviri bufr product yaml files and updated toz yaml file

529504a

mraspaud requested a review from sjoro October 16, 2019 07:33

Seviri Bufr reader, code cleanup

3846eb6

sjoro requested changes Oct 16, 2019

View reviewed changes

satpy/readers/seviri_l2_bufr.py Outdated Show resolved Hide resolved

satpy/etc/readers/seviri_l2_bufr_asr.yaml Outdated Show resolved Hide resolved

mraspaud requested changes Oct 17, 2019

View reviewed changes

Updated seviri l2 bufr reader in response to reader comments

f925ac9

ColinDuff and others added 4 commits October 24, 2019 16:57

Delete seviri_l2_bufr_toz.yaml

8f651ef

Updated seviri l2 bufr reader in response to reader comments

88c005e

Merge branch 'msg_bufr_reader' of https://github.com/ColinDuff/satpy …

4d86b9f

…into msg_bufr_reader

Merge branch 'master' into msg_bufr_reader

abec73b

sjoro requested changes Oct 25, 2019

View reviewed changes

satpy/etc/readers/seviri_l2_bufr.yaml Outdated Show resolved Hide resolved

satpy/etc/readers/seviri_l2_bufr.yaml Outdated Show resolved Hide resolved

satpy/etc/readers/seviri_l2_bufr.yaml Outdated Show resolved Hide resolved

Colin.Duff@eumetsat.int added 2 commits October 25, 2019 15:06

Updates made to resolve comments, fill values updated, units and stan…

ba5d28d

…dard names changed.

Merge branch 'msg_bufr_reader' of https://github.com/ColinDuff/satpy …

2f1fab9

…into msg_bufr_reader

sjoro approved these changes Oct 25, 2019

View reviewed changes

mraspaud approved these changes Oct 25, 2019

View reviewed changes

mraspaud and others added 10 commits October 27, 2019 21:31

Add the severi-l2-bufr test suite the main suite

96f7da7

Merge branch 'master' into msg_bufr_reader

fbd41e6

Don't test seviri l2 on windows (eccodes)

becebe0

Fix seviri l2 test

d363d82

Updated to add seviri l2 bufr test to main suite and to correct an er…

09eb851

…ror in setup.py, python-eccodes is the correct dependency

Updated to add seviri l2 bufr test to main suite and to correct an er…

1d7bce0

…ror in setup.py, python-eccodes is the correct dependency

Updated setup.py to only include the pip eccodes-python package as a …

e72b84f

…dependency

Updated reader to warn user of missing eccodes module if import fails

1c6fafa

Udated test with a larger error margin for checking random array mean…

b2ecded

… values

Updated so that the bufr reader only returns data/lat/lon as vectors,…

9f36202

… no mapping/sampling takes place

sjoro requested changes Nov 1, 2019

View reviewed changes

Removed unused dict seg_area_dict

35b7441

sjoro approved these changes Nov 4, 2019

View reviewed changes

mraspaud approved these changes Nov 4, 2019

View reviewed changes

mraspaud merged commit e958d7d into pytroll:master Nov 4, 2019

ColinDuff deleted the msg_bufr_reader branch November 11, 2020 08:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added initial version of an MSG BUFR reader and TOZ product yaml file #769

Added initial version of an MSG BUFR reader and TOZ product yaml file #769

ColinDuff commented May 16, 2019

ColinDuff commented May 16, 2019

codecov bot commented May 17, 2019 •

edited

Loading

coveralls commented May 17, 2019 •

edited

Loading

mraspaud commented Sep 18, 2019

ColinDuff commented Sep 19, 2019

mraspaud commented Sep 19, 2019

ColinDuff commented Oct 1, 2019

sjoro left a comment •

edited

Loading

mraspaud left a comment

ColinDuff commented Oct 24, 2019

sjoro left a comment

sjoro left a comment

mraspaud left a comment

sjoro left a comment

sjoro left a comment

mraspaud left a comment

Added initial version of an MSG BUFR reader and TOZ product yaml file #769

Added initial version of an MSG BUFR reader and TOZ product yaml file #769

Conversation

ColinDuff commented May 16, 2019

ColinDuff commented May 16, 2019

codecov bot commented May 17, 2019 • edited Loading

Codecov Report

coveralls commented May 17, 2019 • edited Loading

mraspaud commented Sep 18, 2019

ColinDuff commented Sep 19, 2019

mraspaud commented Sep 19, 2019

ColinDuff commented Oct 1, 2019

sjoro left a comment • edited Loading

Choose a reason for hiding this comment

mraspaud left a comment

Choose a reason for hiding this comment

ColinDuff commented Oct 24, 2019

sjoro left a comment

Choose a reason for hiding this comment

sjoro left a comment

Choose a reason for hiding this comment

mraspaud left a comment

Choose a reason for hiding this comment

sjoro left a comment

Choose a reason for hiding this comment

sjoro left a comment

Choose a reason for hiding this comment

mraspaud left a comment

Choose a reason for hiding this comment

codecov bot commented May 17, 2019 •

edited

Loading

coveralls commented May 17, 2019 •

edited

Loading

sjoro left a comment •

edited

Loading