Optimize Datasource-based queries

[kyri-petrou]

This PR contains a series of optimizations aimed at improving the performance of Datasource-backed queries.

Main optimizations

1. Implement Cache.Default using a java.util.ConcurrentHashMap[_, _] in favour of a Ref[Map[_, _]]. This allows us to check if the cache provided is the default one, and if yes, use the "unsafe" methods to mutate the cache, reducing the number of flatmaps for each ZQuery.fromRequest
2. Futher reduce the number of flatmaps for each ZQuery.fromRequest by using the FiberRef#getWith methods for ZQuery.currentCache and ZQuery.cachingEnabled.
3. Rewrite BlockedRequests.run using mutable datastructures and fulfilling promises unsafely
4. Add a new empty constructor to Cache which allows pre-sizing the underlying ConcurrentHashMap
5. Optimize BlockedRequests.flatten by using mutable datastructures

Benchmarking results:

Based on the newly added benchmarks, throughput is improved by:

• Up to 250% when using the default Cache size
• Up to 300% when using a Cache large enough to fit all requests

series/2.x:

[info] Benchmark                                (count)   Mode  Cnt      Score    Error  Units
[info] FromRequestBenchmark.fromRequestDefault      100  thrpt   10  12168.714 ± 97.771  ops/s
[info] FromRequestBenchmark.fromRequestDefault     1000  thrpt   10   1043.727 ± 34.488  ops/s

PR:

[info] Benchmark                                (count)   Mode  Cnt      Score     Error  Units
[info] FromRequestBenchmark.fromRequestDefault      100  thrpt   10  27293.203 ± 254.516  ops/s
[info] FromRequestBenchmark.fromRequestDefault     1000  thrpt   10   2640.471 ±  32.877  ops/s
[info] FromRequestBenchmark.fromRequestSized        100  thrpt   10  29534.166 ± 147.223  ops/s
[info] FromRequestBenchmark.fromRequestSized       1000  thrpt   10   3006.937 ±  14.040  ops/s

[ghostdogpr]

Nice! Maybe try to run the caliban test suite as well to catch anything. Does it improve the caliban benchmarks too?

[kyri-petrou]

Does it improve the caliban benchmarks too?

@ghostdogpr Doubtful, the optimizations are for datasource-backed queries and we only use ZQuery.succeed in the Caliban benchmarks. However, I think we should add a param in Caliban in the execution config to size the Cache. Default value of 16 seems way too low

[kyri-petrou]

Maybe try to run the caliban test suite as well to catch anything

I don't think we have anything in Caliban's test suites that uses datasources. I'll test it against some projects at $WORK (which heavily rely on ZQueries) when we have a snapshot version of zio-query and caliban