Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NWCSAF GEO images black with numpy 2 #2877

Closed
gerritholl opened this issue Aug 7, 2024 · 1 comment
Closed

NWCSAF GEO images black with numpy 2 #2877

gerritholl opened this issue Aug 7, 2024 · 1 comment
Assignees
@gerritholl
Labels
bug component:enhancements

Comments

@gerritholl
Copy link
Member

Describe the bug

With numpy 2, images produced from NWCSAF GEO composites such as cloudtype and cloud_top_height appear black.

To Reproduce

from satpy import Scene
from satpy.utils import debug_on; debug_on()
import numpy as np
fns = ["S_NWC_CTTH_MSG4_MSG-N-VISIR_20230222T170000Z.nc", "S_NWC_CT_MSG4_MSG-N-VISIR_20230201T073000Z.nc"]
sc = Scene(filenames=fns, reader=["nwcsaf-geo"])
sc.load(["cloud_top_height", "cloudtype"])
ls = sc.resample("nqeuro3km", radius_of_influence=15000)
ls.save_datasets(writer="geotiff", keep_palette=True)

Expected behavior

I expected coloured palette images.

Actual results

[DEBUG: 2024-08-07 10:25:51 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/nwcsaf-geo.yaml',)
[DEBUG: 2024-08-07 10:25:51 : satpy.readers.yaml_reader] Assigning to nwcsaf-geo: ['/media/nas/x21308/scratch/NWCSAF/202302010730/S_NWC_CT_MSG4_MSG-N-VISIR_20230201T073000Z.nc', '/media/nas/x21308/scratch/NWCSAF/202302221700/S_NWC_CTTH_MSG4_MSG-N-VISIR_20230222T170000Z.nc']
[DEBUG: 2024-08-07 10:25:51 : satpy.composites.config_loader] Looking for composites config file seviri.yaml
[DEBUG: 2024-08-07 10:25:51 : pyorbital.tlefile] Path to the Pyorbital configuration (where e.g. platforms.txt is found): /data/gholl/mambaforge/envs/py312/lib/python3.12/site-packages/pyorbital/etc
[DEBUG: 2024-08-07 10:25:51 : satpy.composites.config_loader] Looking for composites config file visir.yaml
[DEBUG: 2024-08-07 10:25:51 : satpy.readers.nwcsaf_nc] Reading ctth_alti.
[DEBUG: 2024-08-07 10:25:51 : h5py._conv] Creating converter from 7 to 5
[DEBUG: 2024-08-07 10:25:51 : h5py._conv] Creating converter from 5 to 7
[DEBUG: 2024-08-07 10:25:51 : h5py._conv] Creating converter from 7 to 5
[DEBUG: 2024-08-07 10:25:51 : h5py._conv] Creating converter from 5 to 7
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.nwcsaf_nc] Reading ct.
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.nwcsaf_nc] Reading ctth_status_flag.
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.nwcsaf_nc] Reading ctth_alti_pal.
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.yaml_reader] No coordinates found for DataID(name='ctth_alti_pal', resolution=3000, modifiers=())
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.nwcsaf_nc] Reading ct_status_flag.
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.nwcsaf_nc] Reading ctth_conditions.
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.nwcsaf_nc] Reading ct_quality.
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.nwcsaf_nc] Reading ctth_quality.
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.nwcsaf_nc] Reading ct_pal.
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.yaml_reader] No coordinates found for DataID(name='ct_pal', resolution=3000, modifiers=())
[DEBUG: 2024-08-07 10:25:52 : satpy.readers.nwcsaf_nc] Reading ct_conditions.
[DEBUG: 2024-08-07 10:25:52 : satpy.scene] Unloading dataset: DataID(name='ctth_alti', resolution=3000, modifiers=())
[DEBUG: 2024-08-07 10:25:52 : satpy.scene] Unloading dataset: DataID(name='ct', resolution=3000, modifiers=())
[DEBUG: 2024-08-07 10:25:52 : satpy.scene] Resampling DataID(name='cloud_top_height', resolution=3000)
[INFO: 2024-08-07 10:25:53 : satpy.resample] Using default KDTree resampler
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Resampling add-93c45dc6e66ec87f1121b9481a937450
[DEBUG: 2024-08-07 10:25:53 : satpy.scene] Resampling DataID(name='ctth_status_flag', resolution=3000, modifiers=())
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Resampling ctth_status_flag
[DEBUG: 2024-08-07 10:25:53 : satpy.scene] Resampling DataID(name='ctth_conditions', resolution=3000, modifiers=())
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Resampling ctth_conditions
[DEBUG: 2024-08-07 10:25:53 : satpy.scene] Resampling DataID(name='ctth_quality', resolution=3000, modifiers=())
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Resampling ctth_quality
[DEBUG: 2024-08-07 10:25:53 : satpy.scene] Resampling DataID(name='cloudtype', resolution=3000)
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Resampling add-4040886413dba3654a72aeaae5287ccb
[DEBUG: 2024-08-07 10:25:53 : satpy.scene] Resampling DataID(name='ct_status_flag', resolution=3000, modifiers=())
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Resampling ct_status_flag
[DEBUG: 2024-08-07 10:25:53 : satpy.scene] Resampling DataID(name='ct_conditions', resolution=3000, modifiers=())
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Resampling ct_conditions
[DEBUG: 2024-08-07 10:25:53 : satpy.scene] Resampling DataID(name='ct_quality', resolution=3000, modifiers=())
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2024-08-07 10:25:53 : satpy.resample] Resampling ct_quality
[DEBUG: 2024-08-07 10:25:53 : satpy.writers] Reading ['/data/gholl/checkouts/satpy/satpy/etc/writers/geotiff.yaml']
[DEBUG: 2024-08-07 10:25:53 : satpy.writers] Adding enhancement configuration from file: /data/gholl/checkouts/satpy/satpy/etc/enhancements/generic.yaml
[DEBUG: 2024-08-07 10:25:53 : satpy.writers] Adding enhancement configuration from file: /home/gholl/Arbeit/checkouts-perforce/config/enhancements/generic.yaml
[DEBUG: 2024-08-07 10:25:53 : satpy.writers] Adding enhancement configuration from file: /home/gholl/checkouts/pytroll-dwd-config/enhancements/generic.yaml
[DEBUG: 2024-08-07 10:25:53 : satpy.writers] Adding enhancement configuration from file: /data/gholl/checkouts/satpy/satpy/etc/enhancements/seviri.yaml
[DEBUG: 2024-08-07 10:25:53 : satpy.writers] Data for DataID(name='cloud_top_height', resolution=3000) will be enhanced with options:
        [{'name': 'palettize', 'method': <function palettize at 0x7f9ea9a10540>, 'kwargs': {'palettes': [{'dataset': 'ctth_alti_pal', 'color_scale': 255}]}}]
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Entering env context: <rasterio.env.Env object at 0x7f9eaaada690>
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Starting outermost env
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] No GDAL environment exists
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] New GDAL environment <rasterio._env.GDALEnv object at 0x7f9eaab40550> created
[DEBUG: 2024-08-07 10:25:53 : rasterio._filepath] Installing FilePath filesystem handler plugin...
[DEBUG: 2024-08-07 10:25:53 : rasterio._env] GDAL_DATA found in environment.
[DEBUG: 2024-08-07 10:25:53 : rasterio._env] PROJ_DATA found in environment.
[DEBUG: 2024-08-07 10:25:53 : rasterio._env] Started GDALEnv: self=<rasterio._env.GDALEnv object at 0x7f9eaab40550>.
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Entered env context: <rasterio.env.Env object at 0x7f9eaaada690>
[DEBUG: 2024-08-07 10:25:53 : rasterio._io] Path: _UnparsedPath(path='cloud_top_height_20230222_170000.tif'), mode: w, driver: GTiff
[DEBUG: 2024-08-07 10:25:53 : rasterio._base] Nodata success: 0, Nodata value: 0.000000
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Exiting env context: <rasterio.env.Env object at 0x7f9eaaada690>
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Cleared existing <rasterio._env.GDALEnv object at 0x7f9eaab40550> options
[DEBUG: 2024-08-07 10:25:53 : rasterio._env] Stopped GDALEnv <rasterio._env.GDALEnv object at 0x7f9eaab40550>.
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Exiting outermost env
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Exited env context: <rasterio.env.Env object at 0x7f9eaaada690>
[DEBUG: 2024-08-07 10:25:53 : satpy.writers] Data for DataID(name='cloudtype', resolution=3000) will be enhanced with options:
        [{'name': 'palettize', 'method': <function palettize at 0x7f9ea9a10540>, 'kwargs': {'palettes': [{'dataset': 'ct_pal', 'color_scale': 255}]}}]
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Entering env context: <rasterio.env.Env object at 0x7f9ea99fefc0>
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Starting outermost env
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] No GDAL environment exists
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] New GDAL environment <rasterio._env.GDALEnv object at 0x7f9ea99fcb50> created
[DEBUG: 2024-08-07 10:25:53 : rasterio._env] GDAL_DATA found in environment.
[DEBUG: 2024-08-07 10:25:53 : rasterio._env] PROJ_DATA found in environment.
[DEBUG: 2024-08-07 10:25:53 : rasterio._env] Started GDALEnv: self=<rasterio._env.GDALEnv object at 0x7f9ea99fcb50>.
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Entered env context: <rasterio.env.Env object at 0x7f9ea99fefc0>
[DEBUG: 2024-08-07 10:25:53 : rasterio._io] Path: _UnparsedPath(path='cloudtype_20230201_073000.tif'), mode: w, driver: GTiff
[DEBUG: 2024-08-07 10:25:53 : rasterio._base] Nodata success: 0, Nodata value: 0.000000
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Exiting env context: <rasterio.env.Env object at 0x7f9ea99fefc0>
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Cleared existing <rasterio._env.GDALEnv object at 0x7f9ea99fcb50> options
[DEBUG: 2024-08-07 10:25:53 : rasterio._env] Stopped GDALEnv <rasterio._env.GDALEnv object at 0x7f9ea99fcb50>.
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Exiting outermost env
[DEBUG: 2024-08-07 10:25:53 : rasterio.env] Exited env context: <rasterio.env.Env object at 0x7f9ea99fefc0>
[INFO: 2024-08-07 10:25:53 : satpy.writers] Computing and writing results...
/data/gholl/mambaforge/envs/py312/lib/python3.12/site-packages/dask/core.py:127: RuntimeWarning: invalid value encountered in sin
  return func(*(_execute_task(a, cache) for a in args))
/data/gholl/mambaforge/envs/py312/lib/python3.12/site-packages/dask/core.py:127: RuntimeWarning: invalid value encountered in cos
  return func(*(_execute_task(a, cache) for a in args))
/data/gholl/checkouts/trollimage/trollimage/colormap.py:182: RuntimeWarning: All-NaN slice encountered
  outside_range_bin = max(np.nanmax(arr), values.max()) + 1

Screenshots

grafik

Environment Info:

  • OS: openSUSE Leap 15.3
  • Satpy version: main (v0.50.0-38-g6f475f199)
  • trollimage version: main (v1.24.0-7-gc665c4d)

Additional context

I found this when working on #2870. It is related to #2872 and pytroll/trollimage#175, but #2874 (in combination with pytroll/trollimage#176) solves the problem only for cloud type, not for cloud top height.
@gerritholl gerritholl self-assigned this Aug 7, 2024
@gerritholl
Copy link
Member Author

Never mind, there was something wrong with the input file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gerritholl gerritholl

Labels
bug component:enhancements
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@gerritholl