Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NUCAPS time format of data from CLASS #1387

Closed
joleenf opened this issue Oct 2, 2020 · 1 comment · Fixed by #1388
Closed

NUCAPS time format of data from CLASS #1387

joleenf opened this issue Oct 2, 2020 · 1 comment · Fixed by #1388

Comments

@joleenf
Copy link
Contributor

joleenf commented Oct 2, 2020

Describe the bug
The Scene object is not able to load data from recently downloaded NUCAPS data from NOAA Class. There seems to have been a time format string change within the files. There is a format change of the time_coverage_start and time_coverage_end strings. I believe the format of the time variable is unchanged.

To Reproduce
Get NUCAPS EDR data from NOAA Class: (https://www.avl.class.noaa.gov/saa/products/search?datatype_family=JPSS_SND)

from datetime import datetime
from satpy import find_files_and_readers, Scene
from satpy.utils import debug_on; debug_on()

files = find_files_and_readers(start_time=datetime(2020,10,1,8,29), end_time=datetime(2020,10,1,8,32))
scn = Scene(files)

Expected behavior
The Scene object should load the files without error.

Actual results
[DEBUG: 2020-10-02 12:24:14 : satpy.scene] Setting 'PPP_CONFIG_DIR' to '/Users/joleenf/code/satpy/satpy/etc'
[DEBUG: 2020-10-02 12:24:14 : satpy.readers.yaml_reader] Reading ('/Users/joleenf/code/satpy/satpy/etc/readers/nucaps.yaml',)
[DEBUG: 2020-10-02 12:24:14 : satpy.readers.yaml_reader] Assigning to nucaps: ['NUCAPS-EDR_v2r0_npp_s202010010831030_e202010010831330_c202010011004460.nc', 'NUCAPS-EDR_v2r0_npp_s202010010829270_e202010010829570_c202010011018340.nc', 'NUCAPS-EDR_v2r0_npp_s202010010831350_e202010010832050_c202010011018180.nc', 'NUCAPS-EDR_v2r0_npp_s202010010828550_e202010010829250_c202010011017490.nc', 'NUCAPS-EDR_v2r0_npp_s202010010830310_e202010010831010_c202010011003140.nc', 'NUCAPS-EDR_v2r0_npp_s202010010829590_e202010010830290_c202010011018170.nc']
Traceback (most recent call last):
File "", line 1, in
File "/Users/joleenf/code/satpy/satpy/scene.py", line 151, in init
self._readers = self._create_reader_instances(filenames=filenames,
File "/Users/joleenf/code/satpy/satpy/scene.py", line 200, in _create_reader_instances
return load_readers(filenames=filenames,
File "/Users/joleenf/code/satpy/satpy/readers/init.py", line 495, in load_readers
reader_instance.create_filehandlers(loadables, fh_kwargs=reader_kwargs_without_filter)
File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 604, in create_filehandlers
filehandlers = self._new_filehandlers_for_filetype(filetype_info,
File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 592, in _new_filehandlers_for_filetype
return list(filtered_iter)
File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 561, in filter_fh_by_metadata
filehandler.metadata['start_time'] = filehandler.start_time
File "/Users/joleenf/code/satpy/satpy/readers/nucaps.py", line 90, in start_time
return self._parse_datetime(self['/attr/time_coverage_start'])
File "/Users/joleenf/code/satpy/satpy/readers/nucaps.py", line 84, in _parse_datetime
return datetime.strptime(datestr, "%Y-%m-%dT%H:%M:%S.%fZ")
File "/Users/joleenf/miniconda3/envs/satpy-dev/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
tt, fraction, gmtoff_fraction = _strptime(data_string, format)
File "/Users/joleenf/miniconda3/envs/satpy-dev/lib/python3.8/_strptime.py", line 349, in _strptime
raise ValueError("time data %r does not match format %r" %
ValueError: time data '2020-10-01T08:31:03Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'

Environment Info:

  • OS: OSX 10.15.6 (Darwin simple.local 19.6.0 Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT 2020; root:xnu-6153.141.1~1/RELEASE_X86_64 x86_64)

  • Satpy Version: 0.23

  • PyResample Version: 1.16.0

  • Readers and writers dependencies Readers
    =======
    abi_l1b: ok
    abi_l1b_scmi: ok
    abi_l2_nc: ok
    acspo: ok
    agri_l1: ok
    ahi_hrit: ok
    ahi_hsd: ok
    ahi_l1b_gridded_bin: ok
    ami_l1b: ok
    amsr2_l1b: ok
    amsr2_l2: ok
    avhrr_l1b_aapp: ok
    avhrr_l1b_eps: ok
    avhrr_l1b_gaclac: cannot find module 'satpy.readers.avhrr_l1b_gaclac' (No module named 'pygac')
    avhrr_l1b_hrpt: cannot find module 'satpy.readers.hrpt' (No module named 'pygac')
    caliop_l2_cloud: cannot find module 'satpy.readers.caliop_l2_cloud' (No module named 'pyhdf')
    clavrx: cannot find module 'satpy.readers.clavrx' (No module named 'pyhdf')
    cmsaf-claas2_l2_nc: ok
    electrol_hrit: ok
    fci_l1c_fdhsi: ok
    fci_l2_nc: ok
    generic_image: ok
    geocat: ok
    ghrsst_l3c_sst: cannot find module 'satpy.readers.ghrsst_l3c_sst' (cannot import name 'Dataset' from 'satpy.dataset' (/Users/joleenf/miniconda3/envs/satpy/lib/python3.8/site-packages/satpy/dataset/init.py))
    glm_l2: ok
    goes-imager_hrit: ok
    goes-imager_nc: ok
    gpm_imerg: ok
    grib: cannot find module 'satpy.readers.grib' (No module named 'pygrib')
    hsaf_grib: cannot find module 'satpy.readers.hsaf_grib' (No module named 'pygrib')
    hy2_scat_l2b_h5: ok
    iasi_l2: ok
    iasi_l2_so2_bufr: cannot find module 'satpy.readers.iasi_l2_so2_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n Error: ', ModuleNotFoundError("No module named 'eccodes'")))
    jami_hrit: ok
    li_l2: cannot find module 'satpy.readers.li_l2' (No module named 'h5netcdf')
    maia: ok
    mersi2_l1b: ok
    mimicTPW2_comp: ok
    modis_l1b: cannot find module 'satpy.readers.modis_l1b' (No module named 'pyhdf')
    modis_l2: cannot find module 'satpy.readers.modis_l2' (No module named 'pyhdf')
    msi_safe: cannot find module 'satpy.readers.msi_safe' (No module named 'glymur')
    mtsat2-imager_hrit: ok
    nucaps: ok
    nwcsaf-geo: ok
    nwcsaf-msg2013-hdf5: ok
    nwcsaf-pps_nc: ok
    olci_l1b: ok
    olci_l2: ok
    omps_edr: ok
    safe_sar_l2_ocn: ok
    sar-c_safe: ok
    scatsat1_l2b: cannot find module 'satpy.readers.scatsat1_l2b' (cannot import name 'Dataset' from 'satpy.dataset' (/Users/joleenf/miniconda3/envs/satpy/lib/python3.8/site-packages/satpy/dataset/init.py))
    seviri_l1b_hrit: ok
    seviri_l1b_icare: cannot find module 'satpy.readers.seviri_l1b_icare' (No module named 'pyhdf')
    seviri_l1b_native: ok
    seviri_l1b_nc: ok
    seviri_l2_bufr: cannot find module 'satpy.readers.seviri_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
    seviri_l2_grib: cannot find module 'satpy.readers.seviri_l2_grib' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
    slstr_l1b: ok
    slstr_l2: ok
    smos_l2_wind: ok
    tropomi_l2: ok
    vaisala_gld360: ok
    vii_l1b_nc: ok
    vii_l2_nc: ok
    viirs_compact: ok
    viirs_edr_active_fires: ok
    viirs_edr_flood: cannot find module 'satpy.readers.viirs_edr_flood' (No module named 'pyhdf')
    viirs_l1b: ok
    viirs_sdr: ok
    virr_l1b: ok
    Writers
    =======
    cf: ok
    geotiff: ok
    mitiff: ok
    ninjotiff: cannot find module 'satpy.writers.ninjotiff' (No module named 'pyninjotiff')
    scmi: ok
    simple_image: ok
    Extras
    ======
    cartopy: No module named 'cartopy'
    geoviews: No module named 'geoviews'

@djhoese
Copy link
Member

djhoese commented Oct 2, 2020

Looks like maybe we should do a double check. First with the format with the .%f part and then if that fails, try again without the .%f part (note the . is in the format which makes it a required character).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants