Convert LHS to exploit v2 data model

[phraenquex]

New scope (2023-05-11 FvD);

XChemAlign and replacement of Fragalysis API need the LHS to be thoroughly reworked, including hierarchical viewing of stuff.
Also addresses epic 5 (how to find stuff on the LHS.)

See mockups further down. Keeping old text because part of spec.

Old text (2022)

Discussed end of January. @duncanpeacock follow-up questions are pasted below.

Before we finalise spec, @reskyner and @phraenquex need to review the workflow of the scientists.

---

Hi Frank and Rachael
I've had a few more thoughts on uploading partial Target sets as I was starting to create the issue (hence formatting). There's a bit more to it than meets the eye. My main worry is the effect on the existing files (hits_ids, sites and alternate names) that are generated from the matedata during the upload (the metadata will also only apply to part of the Target). Changing the delete/recreate processing for these files might prove a little messy so I wondered what these are used for (as files I mean) and whether the data in these files should really be in tables now?

Hope the below is understandable.

Problem:
Currently, when a target set is uploaded, the processing will upsert the molecules in the Target Set file and automatically delete any existing molecules that are not in the file. There is concern about this automatic deletion of molecules from a usability/tracking.

Proposed Solution:

Diamond would like the deletion to be made optional.

1. A "delete molecules not in target file" flag could be added to the upload_tset page which would be unchecked as standard (forcing the user to make a decision.
2. If this is unchecked then the molecules would not be deleted
3. A list of the molecules will be provided in an exception list sent to the email address/upload results page.
4. For the upload results page alternatively a link could be provided that gave you a list of the molecules affected.

Questions/Thoughts

1. Concerned about knock effects by removing the link between zip file and targets as it's a basic relationship in the loading process - Need to check all the follow on processing still works and the files in the database are correct - specific concerns:
   a) metadata.csv provided with the upload is used to further create further files: hits_ids, sites and alternate names. What to do with these? Metadata.csv will also now be a partial upload so the other files will need to be upserted rather than deleted and recreated (to maintain the link with the target dataset). This makes it more complicated.
   b) It might be good to see whether we need these files any more, or whether the data could better be/should be better stored as tables now - if so better to spend the time doing the job properly?
   c) I'm also be bit concerned about the .zip file that were uploaded and will be downloadable at the end of the process. The downloadable file will need to reflect the whole Target and we probably need to store the uploaded file for comparison purposes.
2. Do we also want the validate option to pick these up as exceptions? I would say yes from a users perspective but it's probably a couple of hours extra work.
3. The email is currently sent from a configured gmail account - for Janssen we need to make this configurable for the Janssen email system. For Diamond, it could also optionally be configured to use the STFC mailer.
4. The link to give a list of molecules that are not in the file could fall out of the .zip processing i.e. a downloadable link to not-loaded.zip? It won't be stored in the database so we only know by comparing the uploaded file with what is in the DB.
5. It also occurs to me that we are only comparing the current file with what's in the DB - so effectively only the last update. if we are making database changes, we could also consider a fuller history - or will this be picked up as part of the data provenance project in t

[phraenquex]

What the front-end should do with deprecated molecules in snapshots:

• render all ligand atoms as bad atoms.
• render all protein atoms as transparency = 0.5
• all fonts on LHS to grey (non-black)
• big red no-entry sign (not the orange triangle)
• disable Vector, Interactions, Quality, Electron density. Only P and L

[phraenquex]

What should front-end do if molecules are superceded:

• render the old one like the deprecated one; label should be "PROTEINCODE_old"
• render the new one as normal, but with a green exclamatation mark; tooltip says "supercedes PROTEINCODE_old"

[boriskovar-m2ms]

[phraenquex]

Updated version of the first mockup:

[boriskovar-m2ms]

My notes from 28.3.2023 meeting:

• Add show smiles to compound
• Search by smiles
• Molecules should be renamed to observations
• P00045 is not compound identifier
• Get rid of the values in compound navigator but leave the functionality to display numbers there if the need arises
• Render how many observations a compound has and how many of the are displayed
• Toggles of compound turn on stuff on the first observation (default compound and user should be able to set the default observation for the compound)
• Identities are only for compounds and it could be set only per compound and change is global
• Type of identity in the dialog should have also link with more information
• All modals should be closable by clicking away

[phraenquex]

Leaving this ticket as is, with the full discussion thread of the features; but now labelled as deep-purple release. #1139 will implement the intermediate fix that enables the v2 update to go live.

[phraenquex]

This ticket should (probably?) include the question of how to use and display the numbers, and how they're served from the backend.

Let's discuss when @boriskovar-m2ms is back - might need a separate ticket.

[boriskovar-m2ms]

frontend work tracking moved to #1190