Skip to content

Commit

Permalink
API authentication and security changes (#635)
Browse files Browse the repository at this point in the history 
* style: Reorder functions in the module

* refactor: Moved non-view functions to viewer.utils

* fix: Removed unused save_pdb_zip and minor refactoring

* feat: Removed circular import

* feat: Fix get_open_targets (also get_open_proposals now not _private_)

* feat: Fix get_open_proposals reference

* refactor: ISpyB -> ISPyB

* docs: Updated for use of mixins

* feat: More API security migrations

* feat: More security migrations

* feat: Security migrations for hotspots and hypothesis

* feat: More security fixes

* feat: More security changes

* feat: More security changes (and get_params -> get_img_from_smiles with default w/h)

* fix: Attempt to fix calls to /xcdb/fragspect/ 500 errors

* feat: Another attempot to fix ISPyB

* feat: Use of new user_is_member_of_target()

* feat: Experiment with validator

* feat: Better serializer log

* feat: Even more work on the serializer

* feat: Minor error message tweak

* feat: Add support for TEST_RESTRICTED_TAS_LIST (#614)

Co-authored-by: Alan Christie <alan.christie@matildapeak.com>

* target permission validation mixin pattern implemented for Pose

* feat: Fix restricted logic

* most endpoints secured with VaildateTargetMixin

* fix: Removed unused endpoint

* fix: secure SessionActions serializer

* fix: Removed pset_download

* fix: Design set upload now unsupported (404)

* fix: Snapshots now open again

* fix: CompoundIdentifierTypeView & TagCategoryView now read-only views

* fix: Discourse POST now requires login

* feat: User now needs to be a member of CSET target to download it

* fix: secured TaskStatus endpoint

* feat: Removal of unsed xcdb app

* feat: Add log to use of dicttocsv

* feat: More secure DictToCsv

* feat: More consistent use of _ISPYB_SAFE_QUERY_SET

* feat: Stricter UploadCSet class inheritance

* feat: Fix isort issues

* feat: Fix ListAPIView

* feat: Remove references to xcdb

* fix: secure UploadTaskView and ValidateTaskView

TODO: secure UpdateTaskView (if used)

* Align 1247 with latest staging code (#616)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

---------

Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>

* Align 1247 from staging (#619)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

---------

Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>

* Align 1247 with latest staging (#620)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

---------

Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>

* feat: Rstore CSetUpload post()

* feat: Revert UploadCSet inheritance

* fix: Another attempt to fix UploadCSet

* fix: Another attmept to fix the view

* fix: Anotehr attempt to get UploadCSet

* feat: Fix UploadCSet view

* feat: Fix JobRequest GET (restrict to members of the project)

* feat: Enhanced logging for membership check failures

* docs: Improve docs relating to security

* docs: Minor typo

* fix: Remove TEST_RESTRICTED_TAS_LIST feature

* Align 1247 with staging (#621)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

---------

Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>

* Align 1247 with latest staging (#623)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

---------

Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>

* fix: Attempt to fix PoseView (now Pose)

* fix: Attempt to debug Pose failure

* fix: Another patch to Pose

* fix: Fix log typo

* fix: Attempt to fix permission on create

* fix: Fix for ValidateTargetMixin?

* fix: Better Mixin (renamed and copes with shortest filter string)

* fix: Fix some project mixin views (includes some renaming)

* refactor: View name consistency

* fix: Fix for targetdownload mixin (and extra log)

* fix: Better file handling

* fix: Now serches ExpUpload for first matching record

* fix: Better experiment download (use of only ExpUpload record)

* fix: ExpDownload now inspects Project

* fix: More naming consistency changes

* fix: Attempt to fix  'ManyRelatedManager' is not iterable

* fix: Use of correct download path

* Align 1247 with staging (#624)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

---------

Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>

* Align 1247 with staging (#627)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

---------

Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>

* docs: Tweak messages

* fix: Better file handling

* docs: Doc tweak

* Merge compound fix to 1247 (#628)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

---------

Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>
Co-authored-by: Alan Christie <alan.christie@matildapeak.com>

* refactor: restrict_to_membership now restrict_pubic_to_membership

* fix: ValidateProjectMixin does not insist on public proposal membership for GET

* fix: Apply conflict from staging

* Align 1247 with staging (#631)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: updates to tag generation

Changed how some of the tags are generated as per the comment here: m2ms/fragalysis-frontend#1482 (comment)

* feat: added centroid_res field to CanonSite model

Also, removed fetching centroid_res from CANON_SITES_FILE. Seems that
now it's being added to meta_aligner.yaml, so reading an additional
file is not necessary. I hope...

* feat: added new fields to metadata.csv

Experiment code and centroid res

* feat: added tag aliases to metadata.csv

* Copies Target proposals to new (RHS) Compounds (#629)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

* fix: Add save before copying projects

* fix: Remove unnecessary save()

* ci: Attempt to fix docker-compose problem

* ci: Fix staging and production builds (docker compose)

---------

Co-authored-by: Alan Christie <alan.christie@matildapeak.com>

---------

Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>
Co-authored-by: Alan Christie <alan.christie@matildapeak.com>

* fix: Fix project_id type (aligns with staging)

* Align 1247 with staging (#633)

* fix: updated crystalform site tag generation scheme

* fix: update tag generation scheme

* fix: remove version field from ComputedMolecule

Field was added in error, explicit version tracking is not necessary

* fix: forgot to stage migration file..

* fix: fix querysets in target_loader.py

Came up with pose generation - SiteObservation querysets were called
over the entire table not by the target they should have been
associated with.

* fix: add warning to logs about overwriting ComputedMolecule metadata

* fix: add additional ccp4 files to download (issue 1448)

* fix: updates to tag generation

Changed how some of the tags are generated as per the comment here: m2ms/fragalysis-frontend#1482 (comment)

* feat: added centroid_res field to CanonSite model

Also, removed fetching centroid_res from CANON_SITES_FILE. Seems that
now it's being added to meta_aligner.yaml, so reading an additional
file is not necessary. I hope...

* feat: added new fields to metadata.csv

Experiment code and centroid res

* feat: added tag aliases to metadata.csv

* Copies Target proposals to new (RHS) Compounds (#629)

* fix: Branch for project reference fix

* fix: Projects copied from Target (during RHS cset-upload)

* fix: Add save before copying projects

* fix: Remove unnecessary save()

* ci: Attempt to fix docker-compose problem

* ci: Fix staging and production builds (docker compose)

---------

Co-authored-by: Alan Christie <alan.christie@matildapeak.com>

* fix: Fix typo accessing Target projects (#632)

---------

Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>
Co-authored-by: Alan Christie <alan.christie@matildapeak.com>

---------

Co-authored-by: Alan Christie <alan.christie@matildapeak.com>
Co-authored-by: Kalev Takkis <ktakkis@informaticsmatters.com>
  • Loading branch information
@alanbchristie @kaliif
3 people authored Aug 27, 2024
1 parent 97a56a5 commit 7a5b4a8
Show file tree
Hide file tree
Showing 28 changed files with 1,194 additions and 991 deletions.
70 changes: 51 additions & 19 deletions api/security.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ def ping_configured_connector() -> bool:
return conn is not None


class ISpyBSafeQuerySet(viewsets.ReadOnlyModelViewSet):
class ISPyBSafeQuerySet(viewsets.ReadOnlyModelViewSet):
"""
This ISpyBSafeQuerySet, which inherits from the DRF viewsets.ReadOnlyModelViewSet,
is used for all views that need to yield (filter) view objects based on a
Expand Down Expand Up @@ -216,7 +216,7 @@ def get_queryset(self):
q_filter = self._get_q_filter(proposal_list)
return self.queryset.filter(q_filter).distinct()

def _get_open_proposals(self):
def get_open_proposals(self):
"""
Returns the set of proposals anybody can access.
These consist of any Projects that are marked "open_to_public"
Expand All @@ -226,6 +226,7 @@ def _get_open_proposals(self):
Project.objects.filter(open_to_public=True).values_list("title", flat=True)
)
open_proposals.update(settings.PUBLIC_TAS_LIST)
# End Temporary Test Code (1247)
return open_proposals

def _get_proposals_for_user_from_django(self, user):
Expand Down Expand Up @@ -341,36 +342,67 @@ def _get_proposals_from_connector(self, user, conn):
)
CachedContent.set_content(user.username, prop_id_set)

def user_is_member_of_any_given_proposals(self, user, proposals):
def user_is_member_of_target(
self, user, target, restrict_public_to_membership=True
):
"""
Returns true if the user has access to any proposal the target belongs to.
"""
target_proposals = [p.title for p in target.project_id.all()]
user_proposals = self.get_proposals_for_user(
user, restrict_public_to_membership=restrict_public_to_membership
)
is_member = any(proposal in user_proposals for proposal in target_proposals)
if not is_member:
logger.warning(
"Failed membership check user='%s' target='%s' target_proposals=%s",
user.username,
target.title,
target_proposals,
)
return is_member

def user_is_member_of_any_given_proposals(
self, user, proposals, restrict_public_to_membership=True
):
"""
Returns true if the user has access to any proposal in the given
proposals list.Only one needs to match for permission to be granted.
We 'restrict_to_membership' to only consider proposals the user
proposals list. Only one needs to match for permission to be granted.
We 'restrict_public_to_membership' to only consider proposals the user
has explicit membership.
"""
user_proposals = self.get_proposals_for_user(user, restrict_to_membership=True)
return any(proposal in user_proposals for proposal in proposals)
user_proposals = self.get_proposals_for_user(
user, restrict_public_to_membership=restrict_public_to_membership
)
is_member = any(proposal in user_proposals for proposal in proposals)
if not is_member:
logger.warning(
"Failed membership check user='%s' proposals=%s",
user.username,
proposals,
)
return is_member

def get_proposals_for_user(self, user, restrict_to_membership=False):
def get_proposals_for_user(self, user, restrict_public_to_membership=False):
"""
Returns a list of proposals that the user has access to.
If 'restrict_to_membership' is set only those proposals/visits where the user
If 'restrict_public_to_membership' is set only those proposals/visits where the user
is a member of the visit will be returned. Otherwise the 'public'
proposals/visits will also be returned. Typically 'restrict_to_membership' is
proposals/visits will also be returned. Typically 'restrict_public_to_membership' is
used for uploads/changes - this allows us to implement logic that (say)
only permits explicit members of public proposals to add/load data for that
project (restrict_to_membership=True), but everyone can 'see' public data
(restrict_to_membership=False).
project (restrict_public_to_membership=True), but everyone can 'see' public data
(restrict_public_to_membership=False).
"""
assert user

proposals = set()
ispyb_user = settings.ISPYB_USER
logger.debug(
"ispyb_user=%s restrict_to_membership=%s (DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP=%s)",
"ispyb_user=%s restrict_public_to_membership=%s (DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP=%s)",
ispyb_user,
restrict_to_membership,
restrict_public_to_membership,
settings.DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP,
)
if ispyb_user:
Expand All @@ -384,10 +416,10 @@ def get_proposals_for_user(self, user, restrict_to_membership=False):
# We have all the proposals where the user has authority.
# Add open/public proposals?
if (
not restrict_to_membership
not restrict_public_to_membership
or settings.DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP
):
proposals.update(self._get_open_proposals())
proposals.update(self.get_open_proposals())

# Return the set() as a list()
return list(proposals)
Expand All @@ -411,9 +443,9 @@ def _get_q_filter(self, proposal_list):
return Q(title__in=proposal_list) | Q(open_to_public=True)


class ISpyBSafeStaticFiles:
class ISPyBSafeStaticFiles:
def get_queryset(self):
query = ISpyBSafeQuerySet()
query = ISPyBSafeQuerySet()
query.request = self.request
query.filter_permissions = self.permission_string
query.queryset = self.model.objects.filter()
Expand Down Expand Up @@ -459,7 +491,7 @@ def get_response(self):
raise Http404 from exc


class ISpyBSafeStaticFiles2(ISpyBSafeStaticFiles):
class ISPyBSafeStaticFiles2(ISPyBSafeStaticFiles):
def get_response(self):
logger.info("+ get_response called with: %s", self.input_string)
# it wasn't working because found two objects with test file name
Expand Down
31 changes: 14 additions & 17 deletions api/urls.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,18 @@
from rest_framework.authtoken import views as drf_views
from rest_framework.routers import DefaultRouter

# from xcdb import views as xchem_views
from hotspots import views as hostpot_views
from hypothesis import views as hypo_views
from scoring import views as score_views
from viewer import views as viewer_views
from xcdb import views as xcdb_views

router = DefaultRouter()
# Register the basic data
router.register("compounds", viewer_views.CompoundView, "compounds")
router.register("targets", viewer_views.TargetView, "targets")
router.register("projects", viewer_views.ProjectView)
router.register("session-projects", viewer_views.SessionProjectsView)
router.register("snapshots", viewer_views.SnapshotsView)
router.register("session-projects", viewer_views.SessionProjectView)
router.register("snapshots", viewer_views.SnapshotView)
router.register("action-type", viewer_views.ActionTypeView)
router.register("session-actions", viewer_views.SessionActionsView)
router.register("snapshot-actions", viewer_views.SnapshotActionsView)
Expand All @@ -26,7 +24,7 @@
# Compounds sets
router.register("compound-sets", viewer_views.ComputedSetView)
router.register("compound-molecules", viewer_views.ComputedMoleculesView)
router.register("numerical-scores", viewer_views.NumericalScoresView)
router.register("numerical-scores", viewer_views.NumericalScoreValuesView)
router.register("text-scores", viewer_views.TextScoresView)
router.register("compound-scores", viewer_views.CompoundScoresView, "compound-scores")
router.register(
Expand Down Expand Up @@ -65,16 +63,13 @@
# Get the information
router.register("siteobservationannotation", score_views.SiteObservationAnnotationView)

# fragspect
router.register("fragspect", xcdb_views.FragspectCrystalView)

# discourse posts
router.register(
"discourse_post", viewer_views.DiscoursePostView, basename='discourse_post'
)

# Take a dictionary and return a csv
router.register("dicttocsv", viewer_views.DictToCsv, basename='dicttocsv')
router.register("dicttocsv", viewer_views.DictToCSVView, basename='dicttocsv')

# tags
router.register("tag_category", viewer_views.TagCategoryView, basename='tag_category')
Expand All @@ -92,36 +87,38 @@
# Download a zip file of the requested contents
router.register(
"download_structures",
viewer_views.DownloadStructures,
viewer_views.DownloadStructuresView,
basename='download_structures',
)

# Experiments and Experiment (XChemAlign) upload support
router.register(
"upload_target_experiments",
viewer_views.UploadTargetExperiments,
viewer_views.UploadExperimentUploadView,
basename='upload_target_experiments',
)
router.register(
"download_target_experiments",
viewer_views.DownloadTargetExperiments,
viewer_views.DownloadExperimentUploadView,
basename='download_target_experiments',
)


router.register(
"target_experiment_uploads",
viewer_views.TargetExperimentUploads,
viewer_views.ExperimentUploadView,
basename='target_experiment_uploads',
)
router.register(
"site_observations", viewer_views.SiteObservations, basename='site_observations'
"site_observations", viewer_views.SiteObservationView, basename='site_observations'
)
router.register("canon_sites", viewer_views.CanonSiteView, basename='canon_sites')
router.register(
"canon_site_confs", viewer_views.CanonSiteConfView, basename='canon_site_confs'
)
router.register("canon_sites", viewer_views.CanonSites, basename='canon_sites')
router.register(
"canon_site_confs", viewer_views.CanonSiteConfs, basename='canon_site_confs'
"xtalform_sites", viewer_views.XtalformSiteView, basename='xtalform_sites'
)
router.register("xtalform_sites", viewer_views.XtalformSites, basename='xtalform_sites')
router.register("poses", viewer_views.PoseView, basename='poses')

# Squonk Jobs
Expand Down
12 changes: 4 additions & 8 deletions api/utils.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -305,20 +305,16 @@ def parse_xenons(input_smi):
return bond_ids, bond_colours, e_mol.GetMol()


def get_params(smiles, request):
def get_img_from_smiles(smiles, request):
# try:
smiles = canon_input(smiles)
# except:
# smiles = ""
height = None
mol = None
bond_id_list = []
highlightBondColors = {}
if "height" in request.GET:
height = int(request.GET["height"])
width = None
if "width" in request.GET:
width = int(request.GET["width"])
height = int(request.GET.get("height", "128"))
width = int(request.GET.get("width", "128"))
if "atom_indices" in request.GET:
mol = Chem.MolFromSmiles(smiles)
bond_id_list, highlightBondColors, mol = parse_atom_ids(
Expand Down Expand Up @@ -360,7 +356,7 @@ def get_highlighted_diffs(request):
def mol_view(request):
if "smiles" in request.GET:
smiles = request.GET["smiles"].rstrip(".svg")
return get_params(smiles, request)
return get_img_from_smiles(smiles, request)
else:
return HttpResponse("Please insert SMILES")

Expand Down
Loading

0 comments on commit 7a5b4a8

Please sign in to comment.