Performance improvements #1536
Performance improvements #1536
Conversation
…ries for them rather than adding the order_by each time
…, so as not to serialise bulky fields that aren't used on listing. Add select_related and prefetch_related to cut down queries."
…, rather that fetching renditions that might not actually be used on the frontend
|
@jacobtoppm do you have any before-and-after metrics, e.g. number of queries run, page response time? |
|
@tomdyson Approximately 650-700 queries down to 60-90ish, and (on my local environment, with silk installed, which can affect things) 7s-9s to 1s-2s. This is specifically for The Weekly, and a few of those queries are then made by the dynamic image serve view, but it should be a substantial improvement. It also eliminates an extra API request that wasn't being used. For the meta API, it takes the previously 2000ms+ view down to 100ms range. The cache is relatively short term, but given NewRelic suggests an rpm of about 13rpm minimum, it's still only running on average 1/25 of requests, so I'm hoping it should be a good improvement. |
|
Thanks, @jacobtoppm, those sound very promising! Could we afford a longer cache for the meta API? 2000ms -> 100ms is a very dramatic improvement. |
|
@tomdyson Because some of what it's caching is updated via the CMS (programs for instance) and it's a relatively high traffic view, Karl suggested a shorter cache (30s) than what I'd added originally and we compromised at 2mins, but I'd definitely be up for extending it if we don't think the update frequency justifies the short cache duration. It's a combination of about pages, programs, content type pages, and subscriptions that get returned, and the latter I don't have a good feeling for how often are updated/how fast you'd want them to respond to an update. @nmorduch, do you have any thoughts on this? |
|
Looking at Jacob's numbers above, the current value of 2 minutes eliminates 96% of the traffic to that view. We could increase it more, but the returns of increasing it diminish rapidly the higher we go. For example, if we now double it to 4 minutes, we would only gain an extra 2% hit rate. |
|
I suppose it depends whether we're going by hit rate or average request time - you're right about the hit rate improvement being slight though. Taking the cached view as somewhere between 100ms and 200ms, doubling the cache time gives us somewhere in the range of 13-20% reduction in average request time. Not huge, but noticeable. |
|
Is this good to merge? |
|
Yes, for sure. The cache length is easily adjustable if we decide it's needed later. |
|
I'm merging this to test on staging! |
For #1533