You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
The Scene object is not able to load data from recently downloaded NUCAPS data from NOAA Class. There seems to have been a time format string change within the files. There is a format change of the time_coverage_start and time_coverage_end strings. I believe the format of the time variable is unchanged.
Expected behavior
The Scene object should load the files without error.
Actual results
[DEBUG: 2020-10-02 12:24:14 : satpy.scene] Setting 'PPP_CONFIG_DIR' to '/Users/joleenf/code/satpy/satpy/etc'
[DEBUG: 2020-10-02 12:24:14 : satpy.readers.yaml_reader] Reading ('/Users/joleenf/code/satpy/satpy/etc/readers/nucaps.yaml',)
[DEBUG: 2020-10-02 12:24:14 : satpy.readers.yaml_reader] Assigning to nucaps: ['NUCAPS-EDR_v2r0_npp_s202010010831030_e202010010831330_c202010011004460.nc', 'NUCAPS-EDR_v2r0_npp_s202010010829270_e202010010829570_c202010011018340.nc', 'NUCAPS-EDR_v2r0_npp_s202010010831350_e202010010832050_c202010011018180.nc', 'NUCAPS-EDR_v2r0_npp_s202010010828550_e202010010829250_c202010011017490.nc', 'NUCAPS-EDR_v2r0_npp_s202010010830310_e202010010831010_c202010011003140.nc', 'NUCAPS-EDR_v2r0_npp_s202010010829590_e202010010830290_c202010011018170.nc']
Traceback (most recent call last):
File "", line 1, in
File "/Users/joleenf/code/satpy/satpy/scene.py", line 151, in init
self._readers = self._create_reader_instances(filenames=filenames,
File "/Users/joleenf/code/satpy/satpy/scene.py", line 200, in _create_reader_instances
return load_readers(filenames=filenames,
File "/Users/joleenf/code/satpy/satpy/readers/init.py", line 495, in load_readers
reader_instance.create_filehandlers(loadables, fh_kwargs=reader_kwargs_without_filter)
File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 604, in create_filehandlers
filehandlers = self._new_filehandlers_for_filetype(filetype_info,
File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 592, in _new_filehandlers_for_filetype
return list(filtered_iter)
File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 561, in filter_fh_by_metadata
filehandler.metadata['start_time'] = filehandler.start_time
File "/Users/joleenf/code/satpy/satpy/readers/nucaps.py", line 90, in start_time
return self._parse_datetime(self['/attr/time_coverage_start'])
File "/Users/joleenf/code/satpy/satpy/readers/nucaps.py", line 84, in _parse_datetime
return datetime.strptime(datestr, "%Y-%m-%dT%H:%M:%S.%fZ")
File "/Users/joleenf/miniconda3/envs/satpy-dev/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
tt, fraction, gmtoff_fraction = _strptime(data_string, format)
File "/Users/joleenf/miniconda3/envs/satpy-dev/lib/python3.8/_strptime.py", line 349, in _strptime
raise ValueError("time data %r does not match format %r" %
ValueError: time data '2020-10-01T08:31:03Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'
Environment Info:
OS: OSX 10.15.6 (Darwin simple.local 19.6.0 Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT 2020; root:xnu-6153.141.1~1/RELEASE_X86_64 x86_64)
Satpy Version: 0.23
PyResample Version: 1.16.0
Readers and writers dependencies Readers
=======
abi_l1b: ok
abi_l1b_scmi: ok
abi_l2_nc: ok
acspo: ok
agri_l1: ok
ahi_hrit: ok
ahi_hsd: ok
ahi_l1b_gridded_bin: ok
ami_l1b: ok
amsr2_l1b: ok
amsr2_l2: ok
avhrr_l1b_aapp: ok
avhrr_l1b_eps: ok
avhrr_l1b_gaclac: cannot find module 'satpy.readers.avhrr_l1b_gaclac' (No module named 'pygac')
avhrr_l1b_hrpt: cannot find module 'satpy.readers.hrpt' (No module named 'pygac')
caliop_l2_cloud: cannot find module 'satpy.readers.caliop_l2_cloud' (No module named 'pyhdf')
clavrx: cannot find module 'satpy.readers.clavrx' (No module named 'pyhdf')
cmsaf-claas2_l2_nc: ok
electrol_hrit: ok
fci_l1c_fdhsi: ok
fci_l2_nc: ok
generic_image: ok
geocat: ok
ghrsst_l3c_sst: cannot find module 'satpy.readers.ghrsst_l3c_sst' (cannot import name 'Dataset' from 'satpy.dataset' (/Users/joleenf/miniconda3/envs/satpy/lib/python3.8/site-packages/satpy/dataset/init.py))
glm_l2: ok
goes-imager_hrit: ok
goes-imager_nc: ok
gpm_imerg: ok
grib: cannot find module 'satpy.readers.grib' (No module named 'pygrib')
hsaf_grib: cannot find module 'satpy.readers.hsaf_grib' (No module named 'pygrib')
hy2_scat_l2b_h5: ok
iasi_l2: ok
iasi_l2_so2_bufr: cannot find module 'satpy.readers.iasi_l2_so2_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n Error: ', ModuleNotFoundError("No module named 'eccodes'")))
jami_hrit: ok
li_l2: cannot find module 'satpy.readers.li_l2' (No module named 'h5netcdf')
maia: ok
mersi2_l1b: ok
mimicTPW2_comp: ok
modis_l1b: cannot find module 'satpy.readers.modis_l1b' (No module named 'pyhdf')
modis_l2: cannot find module 'satpy.readers.modis_l2' (No module named 'pyhdf')
msi_safe: cannot find module 'satpy.readers.msi_safe' (No module named 'glymur')
mtsat2-imager_hrit: ok
nucaps: ok
nwcsaf-geo: ok
nwcsaf-msg2013-hdf5: ok
nwcsaf-pps_nc: ok
olci_l1b: ok
olci_l2: ok
omps_edr: ok
safe_sar_l2_ocn: ok
sar-c_safe: ok
scatsat1_l2b: cannot find module 'satpy.readers.scatsat1_l2b' (cannot import name 'Dataset' from 'satpy.dataset' (/Users/joleenf/miniconda3/envs/satpy/lib/python3.8/site-packages/satpy/dataset/init.py))
seviri_l1b_hrit: ok
seviri_l1b_icare: cannot find module 'satpy.readers.seviri_l1b_icare' (No module named 'pyhdf')
seviri_l1b_native: ok
seviri_l1b_nc: ok
seviri_l2_bufr: cannot find module 'satpy.readers.seviri_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
seviri_l2_grib: cannot find module 'satpy.readers.seviri_l2_grib' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
slstr_l1b: ok
slstr_l2: ok
smos_l2_wind: ok
tropomi_l2: ok
vaisala_gld360: ok
vii_l1b_nc: ok
vii_l2_nc: ok
viirs_compact: ok
viirs_edr_active_fires: ok
viirs_edr_flood: cannot find module 'satpy.readers.viirs_edr_flood' (No module named 'pyhdf')
viirs_l1b: ok
viirs_sdr: ok
virr_l1b: ok
Writers
=======
cf: ok
geotiff: ok
mitiff: ok
ninjotiff: cannot find module 'satpy.writers.ninjotiff' (No module named 'pyninjotiff')
scmi: ok
simple_image: ok
Extras
======
cartopy: No module named 'cartopy'
geoviews: No module named 'geoviews'
The text was updated successfully, but these errors were encountered:
Looks like maybe we should do a double check. First with the format with the .%f part and then if that fails, try again without the .%f part (note the . is in the format which makes it a required character).
Describe the bug
The Scene object is not able to load data from recently downloaded NUCAPS data from NOAA Class. There seems to have been a time format string change within the files. There is a format change of the time_coverage_start and time_coverage_end strings. I believe the format of the time variable is unchanged.
To Reproduce
Get NUCAPS EDR data from NOAA Class: (https://www.avl.class.noaa.gov/saa/products/search?datatype_family=JPSS_SND)
Expected behavior
The Scene object should load the files without error.
Actual results
[DEBUG: 2020-10-02 12:24:14 : satpy.scene] Setting 'PPP_CONFIG_DIR' to '/Users/joleenf/code/satpy/satpy/etc'
[DEBUG: 2020-10-02 12:24:14 : satpy.readers.yaml_reader] Reading ('/Users/joleenf/code/satpy/satpy/etc/readers/nucaps.yaml',)
[DEBUG: 2020-10-02 12:24:14 : satpy.readers.yaml_reader] Assigning to nucaps: ['NUCAPS-EDR_v2r0_npp_s202010010831030_e202010010831330_c202010011004460.nc', 'NUCAPS-EDR_v2r0_npp_s202010010829270_e202010010829570_c202010011018340.nc', 'NUCAPS-EDR_v2r0_npp_s202010010831350_e202010010832050_c202010011018180.nc', 'NUCAPS-EDR_v2r0_npp_s202010010828550_e202010010829250_c202010011017490.nc', 'NUCAPS-EDR_v2r0_npp_s202010010830310_e202010010831010_c202010011003140.nc', 'NUCAPS-EDR_v2r0_npp_s202010010829590_e202010010830290_c202010011018170.nc']
Traceback (most recent call last):
File "", line 1, in
File "/Users/joleenf/code/satpy/satpy/scene.py", line 151, in init
self._readers = self._create_reader_instances(filenames=filenames,
File "/Users/joleenf/code/satpy/satpy/scene.py", line 200, in _create_reader_instances
return load_readers(filenames=filenames,
File "/Users/joleenf/code/satpy/satpy/readers/init.py", line 495, in load_readers
reader_instance.create_filehandlers(loadables, fh_kwargs=reader_kwargs_without_filter)
File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 604, in create_filehandlers
filehandlers = self._new_filehandlers_for_filetype(filetype_info,
File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 592, in _new_filehandlers_for_filetype
return list(filtered_iter)
File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 561, in filter_fh_by_metadata
filehandler.metadata['start_time'] = filehandler.start_time
File "/Users/joleenf/code/satpy/satpy/readers/nucaps.py", line 90, in start_time
return self._parse_datetime(self['/attr/time_coverage_start'])
File "/Users/joleenf/code/satpy/satpy/readers/nucaps.py", line 84, in _parse_datetime
return datetime.strptime(datestr, "%Y-%m-%dT%H:%M:%S.%fZ")
File "/Users/joleenf/miniconda3/envs/satpy-dev/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
tt, fraction, gmtoff_fraction = _strptime(data_string, format)
File "/Users/joleenf/miniconda3/envs/satpy-dev/lib/python3.8/_strptime.py", line 349, in _strptime
raise ValueError("time data %r does not match format %r" %
ValueError: time data '2020-10-01T08:31:03Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'
Environment Info:
OS: OSX 10.15.6 (Darwin simple.local 19.6.0 Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT 2020; root:xnu-6153.141.1~1/RELEASE_X86_64 x86_64)
Satpy Version: 0.23
PyResample Version: 1.16.0
Readers and writers dependencies Readers
=======
abi_l1b: ok
abi_l1b_scmi: ok
abi_l2_nc: ok
acspo: ok
agri_l1: ok
ahi_hrit: ok
ahi_hsd: ok
ahi_l1b_gridded_bin: ok
ami_l1b: ok
amsr2_l1b: ok
amsr2_l2: ok
avhrr_l1b_aapp: ok
avhrr_l1b_eps: ok
avhrr_l1b_gaclac: cannot find module 'satpy.readers.avhrr_l1b_gaclac' (No module named 'pygac')
avhrr_l1b_hrpt: cannot find module 'satpy.readers.hrpt' (No module named 'pygac')
caliop_l2_cloud: cannot find module 'satpy.readers.caliop_l2_cloud' (No module named 'pyhdf')
clavrx: cannot find module 'satpy.readers.clavrx' (No module named 'pyhdf')
cmsaf-claas2_l2_nc: ok
electrol_hrit: ok
fci_l1c_fdhsi: ok
fci_l2_nc: ok
generic_image: ok
geocat: ok
ghrsst_l3c_sst: cannot find module 'satpy.readers.ghrsst_l3c_sst' (cannot import name 'Dataset' from 'satpy.dataset' (/Users/joleenf/miniconda3/envs/satpy/lib/python3.8/site-packages/satpy/dataset/init.py))
glm_l2: ok
goes-imager_hrit: ok
goes-imager_nc: ok
gpm_imerg: ok
grib: cannot find module 'satpy.readers.grib' (No module named 'pygrib')
hsaf_grib: cannot find module 'satpy.readers.hsaf_grib' (No module named 'pygrib')
hy2_scat_l2b_h5: ok
iasi_l2: ok
iasi_l2_so2_bufr: cannot find module 'satpy.readers.iasi_l2_so2_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n Error: ', ModuleNotFoundError("No module named 'eccodes'")))
jami_hrit: ok
li_l2: cannot find module 'satpy.readers.li_l2' (No module named 'h5netcdf')
maia: ok
mersi2_l1b: ok
mimicTPW2_comp: ok
modis_l1b: cannot find module 'satpy.readers.modis_l1b' (No module named 'pyhdf')
modis_l2: cannot find module 'satpy.readers.modis_l2' (No module named 'pyhdf')
msi_safe: cannot find module 'satpy.readers.msi_safe' (No module named 'glymur')
mtsat2-imager_hrit: ok
nucaps: ok
nwcsaf-geo: ok
nwcsaf-msg2013-hdf5: ok
nwcsaf-pps_nc: ok
olci_l1b: ok
olci_l2: ok
omps_edr: ok
safe_sar_l2_ocn: ok
sar-c_safe: ok
scatsat1_l2b: cannot find module 'satpy.readers.scatsat1_l2b' (cannot import name 'Dataset' from 'satpy.dataset' (/Users/joleenf/miniconda3/envs/satpy/lib/python3.8/site-packages/satpy/dataset/init.py))
seviri_l1b_hrit: ok
seviri_l1b_icare: cannot find module 'satpy.readers.seviri_l1b_icare' (No module named 'pyhdf')
seviri_l1b_native: ok
seviri_l1b_nc: ok
seviri_l2_bufr: cannot find module 'satpy.readers.seviri_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
seviri_l2_grib: cannot find module 'satpy.readers.seviri_l2_grib' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
slstr_l1b: ok
slstr_l2: ok
smos_l2_wind: ok
tropomi_l2: ok
vaisala_gld360: ok
vii_l1b_nc: ok
vii_l2_nc: ok
viirs_compact: ok
viirs_edr_active_fires: ok
viirs_edr_flood: cannot find module 'satpy.readers.viirs_edr_flood' (No module named 'pyhdf')
viirs_l1b: ok
viirs_sdr: ok
virr_l1b: ok
Writers
=======
cf: ok
geotiff: ok
mitiff: ok
ninjotiff: cannot find module 'satpy.writers.ninjotiff' (No module named 'pyninjotiff')
scmi: ok
simple_image: ok
Extras
======
cartopy: No module named 'cartopy'
geoviews: No module named 'geoviews'
The text was updated successfully, but these errors were encountered: