available_dataset_names omits composites depending on more than one reader

[gerritholl]

Describe the bug
Composites that depend on more than one reader, for which all the requirements have been met and which can be loaded, appear to be not listed when calling Scene.available_dataset_names().

To Reproduce

from satpy import Scene
from satpy.utils import debug_on; debug_on()
from satpy.readers import FSFile
from s3fs import S3FileSystem
fs = S3FileSystem(anon=True)
abi_file = FSFile(
        "noaa-goes16/ABI-L1b-RadF/2021/152/18/OR_ABI-L1b-RadF-M6C14_G16_s20211521800193_e20211521809501_c20211521809566.nc",
        fs=fs)
glm_file = '/media/nas/x21308/GLM-processed/F/1min/2021/06/01/18/OR_GLM-L2-GLMF-M3_G16_s20211521800000_e20211521801000_c20211791700300.nc'
comp = "C14_flash_extent_density"
sc = Scene(filenames={"abi_l1b": [abi_file], "glm_l2": [glm_file]})
names = sc.available_dataset_names()
print(names)
print(comp in names)
sc.load([comp])
sc.save_datasets()

Expected behavior

I expect that, since comp is loadable, it should be included in names.

Actual results

comp is loadable, but is not included with available_dataset_names(). Only composites that depend on exactly one reader are shown.

(removed >2000 lines of debug messages from botocore and s3fs)

[DEBUG: 2021-06-28 17:54:23 : asyncio] Using selector: EpollSelector
[DEBUG: 2021-06-28 17:54:23 : satpy.readers.yaml_reader] Reading ('/home/gholl/checkouts/satpy/satpy/etc/readers/abi_l1b.yaml', '/media/nas/o16091/00_MITARBEITER/HOLL/Arbeit/checkouts-perforce/dev_Accso_EBP/config/readers/abi_l1b.yaml')
[DEBUG: 2021-06-28 17:54:23 : satpy.readers.yaml_reader] Assigning to abi_l1b: [<FSFile "noaa-goes16/ABI-L1b-RadF/2021/152/18/OR_ABI-L1b-RadF-M6C14_G16_s20211521800193_e20211521809501_c20211521809566.nc">]
b''
[DEBUG: 2021-06-28 17:54:29 : satpy.readers.yaml_reader] Reading ('/home/gholl/checkouts/satpy/satpy/etc/readers/glm_l2.yaml',)
[DEBUG: 2021-06-28 17:54:29 : satpy.readers.yaml_reader] Assigning to glm_l2: ['/media/nas/x21308/GLM-processed/F/1min/2021/06/01/18/OR_GLM-L2-GLMF-M3_G16_s20211521800000_e20211521801000_c20211791700300.nc']
[DEBUG: 2021-06-28 17:54:29 : satpy.composites.config_loader] Looking for composites config file abi.yaml
[DEBUG: 2021-06-28 17:54:30 : satpy.composites.config_loader] Looking for composites config file visir.yaml
[DEBUG: 2021-06-28 17:54:30 : satpy.composites.config_loader] Looking for composites config file glm.yaml
[DEBUG: 2021-06-28 17:54:30 : satpy.readers.glm_l2] Reading in get_dataset flash_extent_density.
[DEBUG: 2021-06-28 17:54:30 : satpy.readers.abi_l1b] Reading in get_dataset C14.
[DEBUG: 2021-06-28 17:54:30 : satpy.readers.abi_l1b] Calibrating to brightness temperatures
/data/gholl/miniconda3/envs/py39/lib/python3.9/site-packages/pyproj/crs/crs.py:1216: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems
  return self._crs.to_proj4(version=version)
[DEBUG: 2021-06-28 17:54:32 : satpy.writers] Enhancement configuration options: [{'name': 'colorize', 'method': <function colorize at 0x7f25377c4a60>, 'kwargs': {'palettes': [{'colors': 'ylorrd', 'min_value': 0, 'max_value': 20}]}}]
[DEBUG: 2021-06-28 17:54:32 : satpy.writers] Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f253c0eac10>, 'kwargs': {'stretch': 'linear'}}]
[DEBUG: 2021-06-28 17:54:32 : trollimage.xrimage] Applying stretch linear with parameters {}
[DEBUG: 2021-06-28 17:54:32 : trollimage.xrimage] Perform a linear contrast stretch.
[DEBUG: 2021-06-28 17:54:32 : trollimage.xrimage] Calculate the histogram quantiles:
[DEBUG: 2021-06-28 17:54:32 : trollimage.xrimage] Left and right quantiles: 0.005 0.005
/data/gholl/miniconda3/envs/py39/lib/python3.9/site-packages/pyproj/crs/crs.py:1216: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems
  return self._crs.to_proj4(version=version)
[DEBUG: 2021-06-28 17:54:32 : satpy.scene] Unloading dataset: DataID(name='flash_extent_density', resolution=2000, modifiers=())
[DEBUG: 2021-06-28 17:54:32 : satpy.scene] Unloading dataset: DataID(name='C14', wavelength=WavelengthRange(min=10.8, central=11.2, max=11.6, unit='µm'), resolution=2000, calibration=<calibration.brightness_temperature>, modifiers=())
[DEBUG: 2021-06-28 17:54:32 : satpy.writers] Reading ['/home/gholl/checkouts/satpy/satpy/etc/writers/geotiff.yaml']
[DEBUG: 2021-06-28 17:54:32 : satpy.writers] Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f253c0eac10>, 'kwargs': {'stretch': 'crude', 'min_stretch': [0, 0, 0], 'max_stretch': [1, 1, 1]}}]
[DEBUG: 2021-06-28 17:54:32 : trollimage.xrimage] Applying stretch crude with parameters {'min_stretch': [0, 0, 0], 'max_stretch': [1, 1, 1]}
[DEBUG: 2021-06-28 17:54:32 : rasterio.env] Entering env context: <rasterio.env.Env object at 0x7f25378279d0>
[DEBUG: 2021-06-28 17:54:32 : rasterio.env] Starting outermost env
[DEBUG: 2021-06-28 17:54:32 : rasterio.env] No GDAL environment exists
[DEBUG: 2021-06-28 17:54:32 : rasterio.env] New GDAL environment <rasterio._env.GDALEnv object at 0x7f2537827280> created
[DEBUG: 2021-06-28 17:54:32 : rasterio._env] GDAL_DATA found in environment: '/data/gholl/miniconda3/envs/py39/share/gdal'.
[DEBUG: 2021-06-28 17:54:32 : rasterio._env] PROJ_LIB found in environment: '/data/gholl/miniconda3/envs/py39/share/proj'.
[DEBUG: 2021-06-28 17:54:32 : rasterio._env] Started GDALEnv <rasterio._env.GDALEnv object at 0x7f2537827280>.
[DEBUG: 2021-06-28 17:54:32 : rasterio.env] Entered env context: <rasterio.env.Env object at 0x7f25378279d0>
[DEBUG: 2021-06-28 17:54:32 : rasterio._io] Path: UnparsedPath(path='C14_flash_extent_density_20210601_180009.tif'), mode: w, driver: GTiff
[DEBUG: 2021-06-28 17:54:32 : rasterio._io] Option: ('COMPRESS', b'DEFLATE')
[DEBUG: 2021-06-28 17:54:32 : rasterio._io] Option: ('ZLEVEL', b'6')
[DEBUG: 2021-06-28 17:54:32 : rasterio._io] Option: ('PHOTOMETRIC', b'RGB')
[DEBUG: 2021-06-28 17:54:32 : rasterio._base] Nodata success: 0, Nodata value: -10000000000.000000
[DEBUG: 2021-06-28 17:54:32 : rasterio._base] Nodata success: 0, Nodata value: -10000000000.000000
[DEBUG: 2021-06-28 17:54:32 : rasterio._base] Nodata success: 0, Nodata value: -10000000000.000000
[DEBUG: 2021-06-28 17:54:32 : rasterio._base] Nodata success: 0, Nodata value: -10000000000.000000
[DEBUG: 2021-06-28 17:54:32 : rasterio.env] Exiting env context: <rasterio.env.Env object at 0x7f25378279d0>
[DEBUG: 2021-06-28 17:54:32 : rasterio.env] Cleared existing <rasterio._env.GDALEnv object at 0x7f2537827280> options
[DEBUG: 2021-06-28 17:54:32 : rasterio._env] Stopped GDALEnv <rasterio._env.GDALEnv object at 0x7f2537827280>.
[DEBUG: 2021-06-28 17:54:32 : rasterio.env] Exiting outermost env
[DEBUG: 2021-06-28 17:54:32 : rasterio.env] Exited env context: <rasterio.env.Env object at 0x7f25378279d0>
[INFO: 2021-06-28 17:54:32 : satpy.writers] Computing and writing results...
[DEBUG: 2021-06-28 17:54:38 : trollimage.xrimage] Interval: left=<xarray.DataArray (bands: 1)>
array([211.67026326])
Coordinates:
    quantile  float64 0.005
Dimensions without coordinates: bands, right=<xarray.DataArray (bands: 1)>
array([311.27409528])
Coordinates:
    quantile  float64 0.995
Dimensions without coordinates: bands
['C14', 'DQF', 'average_flash_area', 'average_group_area', 'flash_centroid_density', 'flash_extent_density', 'group_centroid_density', 'group_extent_density', 'minimum_flash_area', 'total_energy']
False

Environment Info:

• Satpy Version: v0.29.0-55-gb2af59a5

[djhoese]

available_dataset_names only lists datasets from the reader. You can use available_composite_names to see the composites or pass composites=True to available_dataset_names. You can see in your list that it lists C14 from the ABI reader and the other products are from the GLM reader.

Closing now. Feel free to comment if I missed something and this is a bug.