Implement Presto Feature Extractor UDF

[GriffinBabe]

@kvantricht I suppose this needs to be done once the extraction pipeline (higher priority) is finished

[GriffinBabe]

@kvantricht This is related to the worldcereal-classification repository? Could you complete the details for this task?

[kvantricht]

@kvantricht I suppose this needs to be done once the extraction pipeline (higher priority) is finished

not really, because this is something we'll mainly use for inference. It's something that can (should) be started in parallel to extraction workflow. We need to come up with a first UDF recipe the fetches required inputs, preprocesses according to our needs and then runs Presto to compute the embeddings.

This is related to the worldcereal-classification repository? Could you complete the details for this task?

yes, as discussed the UDF for this is worldcereal-specific and needs to import the gfmap functionality and then launch OpenEO jobs. I think drafting this is something we should do together where you can show how the UDF works now and I can show what we need to do.

[kvantricht]

currently being tackled by @HansVRP

[HansVRP]

A UDF for both the presto as presto+classification were created and tested. Branch: hv_mvp_inferenceUDF

Locally the correct results are obtained on the belgium_good_2020-12-01_2021-11-30.nc dataset.

Remotely the UDF runs but an incorrect output comes out, which could be attributed to lacking METEO data or how the lacking data is being filled in.

In order to verify, the preprocessing pipeline needs to be adapted to include the AGERA5 dataset as well

[kvantricht]

Functional UDF available here: https://github.com/WorldCereal/worldcereal-classification/blob/kvt_mvp_inferenceUDF/minimal_wc_presto/udf_long_worldcereal_inference.py

Next steps:

• Translation to gfmap (comparing to the ref output to make sure it keeps working)
• Addition of meteo input
• Merge the result to main once it's stable and we agree on a package structure
• Optimization of the UDF to lower credit cost