API returns duplicates of same assets

[tinok]

Description

The API call for a list of assets for a particular user returns 26 assets 46 times each. The duplicates share the same uid. (User-specific details here.)

Steps to Reproduce

1. Login as user
2. Request /assets/?limit=2000

Expected

Each asset uid is listed only once

Actual behavior

Instead, 26 assets are listed 42 times.

Additional details

User reported that they tried to delete copies but they re-appeared following deletion (through the UI).

[tinok]

Is it possible to check if this issue is happening to other users as well @jnm?

[jnm]

@noliveleger, if you have time to look at this, please jot a note here. Otherwise, I'll try to get to it as quickly as time permits.

On OCHA:

>>> from kpi.models.object_permission import *
>>> get_objects_for_user(User.objects.get(username='integral'), 'kpi.view_asset').count()
62

Yet while logged in as integral, https://kobo.humanitarianresponse.info/assets/?format=json&asset_type=survey reports "count":1172. We must be missing a distinct() on a QuerySet somewhere. [Edit: ⚠️ ?asset_type=survey is not the right way to filter by type! I should have written ?q=asset_type:survey]

[jnm]

I ran a one-off container on OCHA (docker-compose -f docker-compose.frontend.yml -f docker-compose.frontend.override.yml run --rm kpi bash), added a breakpoint above

kpi/kpi/filters.py

Line 104 in 3d04616

return owned_and_explicitly_shared | subscribed

then started a runserver_plus listening on localhost, and made a request to the /assets endpoint using integral's token:

root@kpi:/srv/src/kpi# export DJANGO_ALLOWED_HOSTS='*'
root@kpi:/srv/src/kpi# unset CSRF_COOKIE_TOKEN
root@kpi:/srv/src/kpi# DJANGO_DEBUG=True ./manage.py runserver_plus &
root@kpi:/srv/src/kpi# ipython
<snip>
In [1]: import requests
In [2]: url = 'http://localhost:8000/assets/?format=json&asset_type=survey'
In [3]: h = {'Authorization': 'Token <censored>'}
In [4]: requests.get(url, headers=h).json()['count']

ipdb> l
    100             except FieldError:
    101                 # Neither the model or its parent has a subscription relation
    102                 subscribed = public.none()
    103 
    104         import ipdb; ipdb.set_trace()
--> 105         return owned_and_explicitly_shared | subscribed
    106 
    107 
    108 class RelatedAssetPermissionsFilter(KpiObjectPermissionsFilter):
    109     ''' Uses KpiObjectPermissionsFilter to determine which assets the user
    110     may access, and then filters the provided queryset to include only objects

ipdb> owned_and_explicitly_shared.count()
62
ipdb> subscribed.count()
0
ipdb> (owned_and_explicitly_shared | subscribed).count()
1172
ipdb>