Cache modification performance degradation experienced at a certain threshold size of data, possibly related to optimism lib cache size (?).

[acton-bell]

Summary:

Using cache.modify to change a field on a single item in a large hierarchical data structure is consistently fast (a fixed ~40ms) until the dataset reaches some threshold number of items (some tens of thousands in our case), at which point there appears to be a change of behavior which results in the action taking much longer (from ~40ms in the case of 10K items to ~440ms in the case of 20K items).

Patching the optimism module (@benjamn) used by Apollo Client to use Infinity (or some other large number) for Cache.max makes the operation fast again.

I would be interested to know if:

• there is something we are doing which is inadvertently triggering this behavior,
• if the patch could have unintended consequences (memory usage?),
• if this cache size parameter is something that could be configurable via Apollo Client.

Detail:

We have a hierarchical data structure (with a fixed maximum depth when queried) of this approximate shape:

type TreeItem {
  id: ID!
  isSelected: Boolean
  children: [TreeItem!]
}

We use cache.modify to change the value of isSelected:

client.cache.modify({
  id: 'someId',
  fields: {
    isSelected: () => newValue,
  },
})

We render the associated data in a virtualized tree component, no more than twenty items at once.

With 10,000 items in the tree, we see the modification take ~40ms, with this stack trace:

With 20,000 items in the tree, we see the modification take ~440ms, with this stack trace (profile before this branching is similar to the previous one, I can provide full if helpful):

The patch we make to the optimism module is (I guess equivalent would be to pass the max parameter whenever Apollo Client uses wrap):

function wrap(originalFunction, options) {
    if (options === void 0) { options = Object.create(null); }
    // Old: 
    // var cache = new Cache(options.max || Math.pow(2, 16), function (entry) { return entry.dispose(); });
    // New:
    var cache = new Cache(Infinity, function (entry) { return entry.dispose(); });

With this in place, the performance drop-off is not seen and modification is back to the order of tens of milliseconds (even with +50K items).

We use the default InMemoryCache, have assumeImmutableResults: true, and are on Apollo Client 3.3.6.

We've tried Apollo Client 3.4.0-beta.4 and see the same issue.

If we have broadcast: false in the call to cache.modify, we don't see the performance issue.

Our setup is fairly complex but I can attempt a reproduction if helpful.

Thanks for the library and all the hard work on AC3, have found it a great tool.

Versions

  System:
    OS: Windows 10 10.0.18363
  Binaries:
    Node: 14.15.2 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.5 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
    npm: 6.14.9 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Spartan (44.18362.449.0)

[benjamn]

@acton-bell Thanks for digging all the way down to this level of the implementation. I'm open to making the default cache size limit configurable, though I'd like to do some performance testing to make sure there's not a simpler solution (like postponing the cleanup to happen asynchronously, or something like that).

Your patch to use Infinity definitely should be safe, in the meantime.

[acton-bell]

Great, thanks for the response on this @benjamn.

We'll continue with the patch for now, and happy to test out any alternative approach you identify.

[AnnatarHe]

I have the same issue here. and find the InMemoryCache.performTransaction take 3.45s, my dataset has 120k rows

OS: macOS 11.1
CPU: 3.2 GHz 6-Core Intel Core i7
Memory: 16 GB 2667 MHz DDR4
Chrome: 88.0.4324.96 (Official Build) (x86_64)

[hwillson]

Addressed by #8107 - thanks!