Does rioxarray allow easy loading of COG overviews into xarray?

[rsignell-usgs]

Does rioxarray allow loading image data from the COG overviews into xarray?
I didn't find it but just wanted to make sure I didn't miss it.

[snowman2]

rioxarray has the rioxarray.open_rasterio() function which uses rasterio.open() under the hood and should be able to open anything handled by rasterio.open(). Do you have an example GOG overlay to try?

[rsignell-usgs]

Yes, here's the rather lame way I'm dealing with overviews now: https://nbviewer.jupyter.org/gist/rsignell-usgs/0f96bb9c0ca34a5dd0fc8131a7bbae1c
I was hoping for something better. 😺

[snowman2]

Nice example. I don't think we have anything better than that at the moment, but it sounds like that would be pretty useful to add.

Currently the open function reads data like so:

rioxarray/rioxarray/_io.py

Line 132 in 83e321c

out = riods.read(band_key, window=window, masked=self.masked)

I think this will need to be thought through a bit as the current implementation uses windowed reading for lazy loading. When using COG overview, it seems that everything needs to be scaled.

My current thoughts on adding support for this is to add support for decimated reading.

Other ideas?

[rsignell-usgs]

Before I knew about the existance of rioxarray, I raised the issue here:
pydata/xarray#3269
with an idea of at least being able to manually request a specific overview via an additional parameter request.

[snowman2]

Maybe three kwargs?

1. overlay_index to select the resampling factor from the overlay index.
2. resampling_factor to pass in resampling read scale factor https://rasterio.readthedocs.io/en/latest/topics/resampling.html
3. resampling_method

Also found this useful for reference:
https://automating-gis-processes.github.io/CSC/notebooks/L5/read-cogs.html

[snowman2]

rasterio/rasterio#1504

[snowman2]

https://gist.github.com/snowman2/9c15df64bd290d1ffd928e0b66855b07
You can try it out with code in #48. It does not allow you to switch between overviews at the moment (it just seems to load in the highest resolution). So, needs more research. But, the first overview read works.

[snowman2]

Issue addressed. It was related to: rasterio/rasterio#1504

[snowman2]

@rsignell-usgs any issues using the URL from your example for an example in rioxarray? Is it a temporary file or a longer term file?

[snowman2]

@vincentsarago do you happen to have a small test-sized COG tif file that could be used for testing?

[vincentsarago]

@snowman2 https://github.com/cogeotiff/rio-cogeo/blob/master/tests/fixtures/image_rgb.tif ?

[snowman2]

Do you happen to have one with overviews?

[snowman2]

rds.overviews(1)

[]

[vincentsarago]

🤦‍♂

[vincentsarago]

Sorry about that, this one should be better https://www.dropbox.com/s/g6etn6o9x3bed62/cog.tif?dl=0

[snowman2]

Perfect, thanks! Mind if I add it to the repo? Also, mind if I add you to the contributor list for testing?

[vincentsarago]

sure, no worry ;-)

[rsignell-usgs]

@snowman2 I intend to leave the Southern California DEM COG there for a few months or so, but after that, it will hopefully move back to sciencebase.gov, so probably not the best to use for long-term examples.

[snowman2]

@snowman2 I intend to leave the Southern California DEM COG there for a few months or so, but after that, it will hopefully move back to sciencebase.gov, so probably not the best to use for long-term examples.

@rsignell-usgs That is good to know. Thanks for letting me know. Are you aware of any COG files that are accessible via URL and are more long term?

[vincentsarago]

@snowman2 on https://openaerialmap.org you should find what you need

[snowman2]

@snowman2 on https://openaerialmap.org you should find what you need

Thanks for the link. I took a quick look and it seems like it provides tiled endpoints, but I couldn't find an endpoint to a COG. I am probably missing something, so please point it out if I am.

[vincentsarago]

@snowman2 you can find the link on the little arrow down

[snowman2]

Nice, thanks for that 👍

[snowman2]

Updated: https://corteva.github.io/rioxarray/html/examples/COG.html

[snowman2]

@rsignell-usgs, this was a good idea and you provided useful examples to help implement this feature. Mind if I add you to the contributors list for ideas?

[rsignell-usgs]

@snowman2 , sure that would be great! BTW, here's my new streamlined hillshade notebook using the overview enhancement of #48 and the CRS help supplied in #49.

[snowman2]

Nice! The notebook looks much simpler to use.

Sounds good, I will add you in a bit.