Feat. Collection - add pagination to limit the amount of fetched data

[ludoboludo]

PR Summary:

Wraps the forloop on the feat. collection section to limit the amount of fetched data, knowing we're only needing as many as our limit is suggesting.

Why are these changes introduced?

Fix concerns about performance.

What approach did you take?

Updated the forloop to be wrapped in a if statement.

Other considerations

Visual impact on existing themes

Shouldn't have any

Testing steps/scenarios

• Added a feat. collection section on any template and make sure it works as expected.
• Test different limits and play with the existing settings for the section

Demo links

• Editor

Checklist

• Added PR summary for release notes
• Requested review from UX (Only for changes that are affecting the experience or perceivable visual details)
• Created a ticket for the help.shopify.com documentation team about updates to theme settings. (Internal-only task)
• Followed theme code principles
• Linted with Theme Check
• Tested on mobile
• Tested on multiple browsers
• Tested for accessibility

[krzksz]

Great change @ludoboludo! Let me add a little bit of context for anyone interested.

One of the most impactful Liquid performance tips is to make sure the code does as few iterations as possible. You don't want to waste any cycles going through the data that you don't actually need.

I used to think that having a limit: in the for loop should be enough to achieve this. You should still use it, but it's only a half of the equation.

I stumbled upon this issue when optimising one of the stores. It was suffering from high server response times on homepage. The layout was quite simple, all featured collection sections already were using the limit: so I had to dig deeper into what's going on.

After consulting our Liquid folks, I learned that all collection.products objects follow the default pagination rules. By default, they will fetch the first 50 products in the given collection. This is not an issue for small or simple catalogues, but can lead to longer queries when you have a lot of variants, prices, etc.

All in all, when iterating over a collection and using limit, you should also use the paginate as well. The former will limit the amount of iteration, the latter makes sure you're not over-fetching the data.

[mxstbr]

Sick!

[Roi-Arthur]

For my own understanding: do we need to use limit on the forloop once we have a paginate in place?

From @krzksz's comment it sounds like paginate will fetch only products_to_show items, that we then iterate on, so we don't risk running too many iterations already?
Or are we saying that the for loop will run through the whole collection but paginate won't output it all?

[Roi-Arthur]

I might be confused, but isn't using the paginate tag already giving us the "limit"?

[ludoboludo]

That's a good point 🤔 In this context it doesn't seem like the limit would be needed if the pagination alone is already limiting the amount of data fetched.
cc: @krzksz

[Roi-Arthur]

It might be a happy accident / unintended result of how these tags work and not best practice; either way just trying to understand these 2 tags better 😁

[ludoboludo]

Updated 👍 Removed the limit from the for loop since like mentioned it's already being paginated.

[Matt-Kaminski]

@krzksz

I tried implementing this change in a theme I’m working on but noticed that the limit on products doesn’t seem to have any effect. After testing it on the Dawn theme, I observed the same issue there as well.

I suspect pagination might not work with section.settings.collection at all. Based on the documentation here, it seems pagination might be limited to certain direct objects and may not apply to objects referenced through section settings.

[krzksz]

Hey @Matt-Kaminski!

I tried implementing this change in a theme I’m working on but noticed that the limit on products doesn’t seem to have any effect.

Can you clarify what you mean by this? Is it the fact that you get more products then the paginate ... by limit should allow?

@ludoboludo I think we need to release a hotfix for this change that restores the limit: in the for loop. I have a Featured carousel on the homepage on my test store (password: pierogi). The product limit is set to 5 but I still get at least 6 products. It seems like there's an internal minimum to how many elements you can paginate by.

[Matt-Kaminski]

Hey @Matt-Kaminski!

I tried implementing this change in a theme I’m working on but noticed that the limit on products doesn’t seem to have any effect.

Can you clarify what you mean by this? Is it the fact that you get more products then the paginate ... by limit should allow?

@ludoboludo I think we need to release a hotfix for this change that restores the limit: in the for loop. I have a Featured carousel on the homepage on my test store (password: pierogi). The product limit is set to 5 but I still get at least 6 products. It seems like there's an internal minimum to how many elements you can paginate by.

As you have noticed, the limit does not work correctly. I don't think there is a minimum number for the limit, but the pagination doesn't seem to work at all. At least, when I tested it, the limit had no effect regardless of the value set.

I think it's possible that pagination only works when applied to global objects or arrays, not those retrieved from section settings. For example, it might work for:

{% paginate collection.products by 10 %}

but not for:

{% paginate section.settings.collection.products by 10 %}

I'm not entirely sure about this. Perhaps combining pagination with a limit, even for objects from the section settings, internally works in a way that optimizes performance. I suspect it might not because of limit not working.

[krzksz]

Got it! I'm sure that paginate does work for objects like section.settings.collection.products and reduce the amount of queried data.

I think the main issue is that is still needs to be combined with the limit: clause.

[Roi-Arthur]

I think it's possible that pagination only works when applied to global objects or arrays, not those retrieved from section settings. For example, it might work for:
{% paginate collection.products by 10 %}
but not for:
{% paginate section.settings.collection.products by 10 %}

I'm not entirely sure about this. Perhaps combining pagination with a limit, even for objects from the section settings, internally works in a way that optimizes performance. I suspect it might not because of limit not working.

Got it! I'm sure that paginate does work for objects like section.settings.collection.products and reduce the amount of queried data.

Just tested out that {%- paginate collections[section.settings.custom_collection.handle].products by section.settings.products_per_page -%} also works as expected, without needing limit: on the for loop

{%- paginate collections[section.settings.custom_collection.handle].products by section.settings.products_per_page -%}
  {% for product in collections[section.settings.custom_collection.handle].products -%}
    {{ product.title }}<br>
  {% endfor %}
{% endpaginate %}

On the other hand, it seems that any manipulation on the collection object also breaks things.
For example the below outputs all items

{% assign reverse_collection = collection.products | reverse %}
{%- paginate reverse_collection by section.settings.products_per_page -%}
  {% for product in reverse_collection -%}
    {{ product.title }}<br>
  {% endfor %}
{% endpaginate %}

Seems like we probably should recommend using the paginate + limit combo everywhere

[ludoboludo]

Hmm weird, when I try the same code it's giving me a liquid error 🤔 Maybe we can pair on it to confirm where it's happening cc: @Roi-Arthur

[Roi-Arthur]

Hmm weird, when I try the same code it's giving me a liquid error 🤔 Maybe we can pair on it to confirm where it's happening cc: @Roi-Arthur

Ah correct… Only worked because I had another liquid block before that

[ludoboludo]

PR here to re add the limit on the forloop

[bakura10]

This is an extremely interesting optimization that I've never heard of. It seems a bit inutiitive though for it to be needed. Shopify can't optimize that with the limit? This should at least be documented somewhere as this feels to be an important perf optimization.

[ludoboludo]

@bakura10 we updated the docs with the info, though we need to update some more to re add limit even when using the pagination