Match product taxons to DFC product types

[mkllnk]

ℹ️ This is a funded feature. Please use the Clockify project with the following name when working on this, including review and test: #9170 OFN DFC Products

Description

Related to:

• Export product data to DFC via the DFC Connector #10781
• DFC Products Part II #10289
• READ Producer Properties via API #11334

Within OFN the instance admin can set up the list of product categories (taxons). A product has a list of taxons and one primary taxon. When we export to another DFC app or import from another DFC app, we want to map these taxons to DFC product types. They are defined in:

• https://github.com/datafoodconsortium/taxonomies/blob/main/productTypes.json
• human readable version : https://showvoc.datafoodconsortium.org/showvoc/#/datasets/ProductTypes/data

The DFC Connector provides these in a tree through methods that look like constants. For example:

connector.PRODUCT_TYPES.VEGETABLE.CABBAGE.BROCCOLI_CABBAGE

Acceptance Criteria & Tests

1. Export OFN products into the DFC prototype.
2. The catalog should display the correct type for each product.

Solution ideas

1. We could try to match by name. This would currently fail in most cases. We can rename categories in OFN to match the DFC name but the DFC standard currently contains only English and French. Other languages would, at least currently, not work.
2. We could add a new field to Spree::Taxon to link to the DFC category. Once instance managers copied the links to DFC categories into OFN then it can be used in the DFC export and import.
3. We could replace all current OFN categories with DFC categories. Instead of instance managers maintaining the categories, we could load the DFC database. We would need to contribute to the DFC to add all languages we support or use Transifex to translate the category names (or both). Instances then need to change the DFC standard if they want to change the categories.

I think that the last option would be the best long-term but it also requires a mapping of current categories to DFC categories for the migration. So solution 2 could be a good intermediate step to achieve the mapping and see if we can map everything to the DFC.

[mkllnk]

From @kirstenalarsen

2 seems like best option to me. Also means that this is just a thing that needs to be dealt with / done by instances using / focused on DFC and therefore not needing to deal with all the languages etc where it is not relevant at the moment. Definitely not 1 as that would be very annoying for users

I think 3 would meet a lot of resistance too

[mkllnk]

From Olivier

for OFN Belgium, we rethought all our product categories so they would have a high compatibility [understand easy mapping] with GS1 Product Categories (GPC), Google Product Categories, INCO (EU) categories. I took ideas from OFN AU, UK, France, USA, ...
This is what we went for (depending on the country/culture, some subcategory can be skipped and the main one is then used) : https://docs.google.com/spreadsheets/d/1YYTPNpK6R63n-fhGzjBD10erK6ZzTIlgBdlaNz9rf2A/edit#gid=1201582111 (mapping can be found in the other sheet tabs).
I thought DFC categories were not finished yet so I didn't integrate them (but I can now 🙂, just need a CSV / spreadsheet version, semantics are hard to process ...) to make the mappings.We limited the amount of categories to 60 ... It's already a lot for the way it's presented to the customer in OFN ...
We were going to infer the OFN BE category from the GS1 category (super precise), you could do the same with DFC just need a bit of compatibility 😄

[mkllnk]

From @RaggedStaff

I was envisaging something like 2. 3 would be awesome but i think that's a longer term process...
We currently have a vocbench instance to manage Taxonomies, instane managers would need to register & get assigned to the projects (Product Type mainly), we can then assign them privileges & they can submit enhancements to the taxon.
I the shorter term, I'm intending to map the OFN-UK product types to the DFC one ( and enhance where necessary). I'm unclear if that has already happened for FR... :thinking_face:

[rioug]

FYI Order of taxon can't be changed : #9386, might be fixed by #11636 , currently being looked at by a contributor

[rioug]

For a start I have been trying to just match the root Product type (Baker, Drink, Dairy product etc..) but I am running into issue with the connector.

DfcLoader.connector.PRODUCT_TYPES.DRINK.class.name
"DataFoodConsortium::Connector::SKOSInstance"

DfcLoader.connector.PRODUCT_TYPES.DRINK.SOFT_DRINK.LEMONADE.class.name
"DataFoodConsortium::Connector::SKOSConcept"

Unfortunately the DfcLoader.connector.export blows up if we pass DataFoodConsortium::Connector::SKOSInstance has a productType.
Looking at the data: https://showvoc.datafoodconsortium.org/showvoc/#/datasets/ProductTypes/data?resId=https:%2F%2Fgithub.com%2Fdatafoodconsortium%2Ftaxonomies%2Freleases%2Flatest%2Fdownload%2FproductTypes.rdf%23drink , it looks like DRINK should also be DataFoodConsortium::Connector::SKOSConcept, or even a DataFoodConsortium::Connector::ProductType (It's an empty subclass of DataFoodConsortium::Connector::SKOSConcept). On top of that none of variable are set (broaders, narrowers, prefLabels ...).

I think it's related to this issue: datafoodconsortium/connector-ruby#13

@mkllnk I am not sure what's the next step , should I try monkey patch the connector so we can move forward (and raise a PR for the connector) ? or wait for the connector to be fixed ?

[mkllnk]

Unfortunately the DfcLoader.connector.export blows up if we pass DataFoodConsortium::Connector::SKOSInstance has a productType.

I wonder if the next version of the connector will solve this problem. I think that it's getting rid of the product type class and everything is just an skos concept. I can't remember properly but it's worth waiting for the next version. Unfortunately, the next version of the connector has a bug that prevents even the loading of product types...

That may be fixed though:

• [Ruby] Loading concepts fails datafoodconsortium/connector-codegen#3 (comment)

I haven't tried that branch yet.

[rioug]

I had a quick look at the branch but it only seems to fix your issue, it doesn't look like they fixed anything else in the parser.
I left a comment on datafoodconsortium/connector-ruby#13 so hopefully it'll get things moving.

[kirstenalarsen]

@gracejam just checking you have found your way to this issue, just re. the analysis of OFN Aus product categories and how we might handle migration

[rioug]

@mkllnk unfortunately the latest version of the connector doesn't unblock this. I created a draft PR : #11817 with my work so far. It also explains the problem and point to the connector code that will need fixing

[mkllnk]

Great work, @rioug. Shall we patch the connector to fix this? I don't think that Maxime has much time for this.

[rioug]

👍 we can chat about it tomorrow.

[gracejam]

Noting for future work of changing OFN categories to align with DFC:
My attempt of mapping current OFN product categories to DFC product categories is in this Airtable base - first 5 tables are applicable. Descriptions of tables included to help explain the logic

[rioug]

I open a PR to fix the ruby-connector : datafoodconsortium/connector-codegen#10. In the mean time I will monkey patch our code so we can move forward