Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bad performance when doing API call on /objects #538

Closed
stevenbal opened this issue Mar 3, 2025 · 1 comment · Fixed by #540
Closed

Bad performance when doing API call on /objects #538

stevenbal opened this issue Mar 3, 2025 · 1 comment · Fixed by #540
Assignees
@stevenbal
Labels
bug Something isn't working objects-api owner: utrecht

Comments

@stevenbal
Copy link
Collaborator

stevenbal commented Mar 3, 2025 •
edited
Loading

Product versie / Product version

2.5.0 / latest

Omschrijf het probleem / Describe the bug

Gemeente Utrecht (taiga issue 335)

CommUtr voert een query uit op de objects-api: geef alle objecten van type Contactmoment waarbij het veld "zaaknummer"={waarde} en sorteer op veld {datum}

Deze relatief eenvoudige API call is erg langzaam. Het duurt eigenlijk altijd minstens 6 seconden maar we zien ook regelmatig 25 seconden

data_attrs = "zaaknummer__exact__{someId}"
ObjectTypeId is provided (contactmoment UUID)
Page number is 1
Page size = 1000
ordering = -record__data__contactmoment__datumContact

Stappen om te reproduceren / Steps to reproduce

No response

Verwacht gedrag / Expected behavior

No response
@stevenbal stevenbal added bug Something isn't working triage labels Mar 3, 2025
@stevenbal stevenbal self-assigned this Mar 3, 2025
@stevenbal stevenbal removed the triage label Mar 3, 2025
@stevenbal stevenbal moved this from Triage to In Progress in Data en API fundament Mar 3, 2025
stevenbal added a commit that referenced this issue Mar 3, 2025
@stevenbal
➕ [#538] Add django-silk for performance profiling
0654a9a
stevenbal added a commit that referenced this issue Mar 3, 2025
@stevenbal
⚡ [#538] Optimize objects list performance
f72f812 
the filter to only include the objectrecords with the highest index per object was causing major performance degradations, especially in combination with filters on data_attrs.
Instead of using `Max(...)` we now use Window to figure out the max index per object
stevenbal added a commit that referenced this issue Mar 3, 2025
@stevenbal
⚡ [#538] Optimize objects list performance
58844ab 
the filter to only include the objectrecords with the highest index per object was causing major performance degradations, especially in combination with filters on data_attrs.
Instead of using `Max(...)` together with GROUP BY, we now use Window to figure out the max index per object which is more efficient for larger datasets
@stevenbal stevenbal mentioned this issue Mar 3, 2025
Merged
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
➕ [#538] Add django-silk for performance profiling
e7bff7b
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
⚡ [#538] Optimize objects list performance
152ed0c 
the filter to only include the objectrecords with the highest index per object was causing major performance degradations, especially in combination with filters on data_attrs.
Instead of using `Max(...)` together with GROUP BY, we now use Window to figure out the max index per object which is more efficient for larger datasets
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
👷 [#538] Add performance test for objects API list
137bec7
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
👷 [#538] Add performance test for objects API list
164f265
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
👷 [#538] Add performance test for objects API list
809561e
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
👷 [#538] Add performance test for objects API list
6737b4c
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
⚡ [#538] Optimize objects list performance
a641bf1 
the filter to only include the objectrecords with the highest index per object was causing major performance degradations, especially in combination with filters on data_attrs.
Instead of using `Max(...)` together with GROUP BY, we now use Window to figure out the max index per object which is more efficient for larger datasets
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
👷 [#538] Add performance test for objects API list
1f5a0b9
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
👷 [#538] Add performance test for objects API list
09ccf70
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
👷 [#538] Add performance test for objects API list
db3a6cd
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
👷 [#538] Add performance test for objects API list
cb45ab2
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
👷 [#538] Add performance test for objects API list
9c8d3db
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
⚡ [#538] Optimize objects list performance
b29a2ca 
the filter to only include the objectrecords with the highest index per object was causing major performance degradations, especially in combination with filters on data_attrs.
    Instead of using `Max(...)` together with GROUP BY, we now use Window to figure out the max index per object which is more efficient for larger datasets
stevenbal added a commit that referenced this issue Mar 4, 2025
@stevenbal
⚡ [#538] Optimize objects list performance
39b01fa 
the filter to only include the objectrecords with the highest index per object was causing major performance degradations, especially in combination with filters on data_attrs.
    Instead of using `Max(...)` together with GROUP BY, we now use Window to figure out the max index per object which is more efficient for larger datasets
@stevenbal stevenbal moved this from In Progress to Implemented in Data en API fundament Mar 4, 2025
@stevenbal
Copy link
Collaborator Author

Found a fix for the issue, might do a second bugfix release at the end of this week for it

stevenbal added a commit that referenced this issue Mar 6, 2025
@stevenbal
Merge pull request #540 from maykinmedia/issue/538-list-performance
bede7c9 
⚡ [#538] Optimize objects list performance
@github-project-automation github-project-automation bot moved this from Implemented to Done in Data en API fundament Mar 6, 2025
@stevenbal stevenbal mentioned this issue Mar 6, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stevenbal stevenbal

Labels
bug Something isn't working objects-api owner: utrecht
Projects
Data en API fundament
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

⚡ [#538] Optimize objects list performance maykinmedia/objects-api
1 participant
@stevenbal