Skip to content

Commit

Permalink
Make cache policy more useful
Browse files Browse the repository at this point in the history
Previously, all the logic around updating the cache policy with new
hints was hardcoded inside the cache control plugin. Also,
`overallCachePolicy` undefined was used to represent two distinct
states: "we don't know anything about cache policy yet" and "definitely
uncacheable". There was a `PolicyUpdater` class that was part of the CC
plugin that handled the logic of updating a policy.

After this PR, `overallCachePolicy` is always defined. The former case
is `maxAge === undefined` and the latter case is `maxAge === 0`. This
object also has some helpful methods that don't exist on `CacheHint`:
- `restrict` and `replace` applies a `CacheHint` to it; the former can
  only make the policy more restrictive, whereas the latter overrides
  the current values of whatever fields are defined in its argument
- `cacheablePolicy` either returns null (if we have no information about
  cacheability or if we know it's not cacheable) or returns a
  `CacheHint` whose maxAge is positive and whose scope is defined.
These make `PolicyUpdater` unnecessary.

Inside a resolver, `info.cacheControl.cacheHint` also has these methods!
So while you can still do `info.cacheControl.setCacheHint(hint)` (which
is identical to `info.cacheControl.cacheHint.replace(hint)`), you also
have the ability to do `info.cacheControl.cacheHint.restrict(hint)` if
that's what you'd like to do.

Additionally, the `maxAge` and `scope` fields on
`info.cacheControl.cacheHint` now update if you call `restrict`,
`replace`, or `setCacheHint`.

Also, resolvers now have access to `info.cacheControl.cacheHintFromType`
which it can use to extract cache hints from the schema. (This may be
helpful for a resolver that returns an abstract type, such as
Federation's `Query._entities`).

Also, `extend type @CacheControl(...)` directives are now honored
instead of silently ignored. There's also a cache for directive parsing
instead of examining the AST in detail every single time a field is
resolved.

This change is mostly backwards-compatible, unless you accessed
`requestContext.overallCachePolicy` directly (eg from a plugin). In that
case:
- If you relied on `requestContext.overallCachePolicy` being undefined
  to mean "not cacheable", you should instead see if
  `requestContext.overallCachePolicy.policyIfCacheable()` is null.
- If you assigned a `CacheHint` to `requestContext.overallCachePolicy`,
  you should pass it to `requestContext.overallCachePolicy.replace()`
  instead.
  • Loading branch information
glasser committed Jun 2, 2021
1 parent 88a058d commit ad6cfa3
Show file tree
Hide file tree
Showing 14 changed files with 349 additions and 211 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ The version headers in this history reflect the versions of Apollo Server itself
- `apollo-server-lambda`: The handler returned by `createHandler` can now only be called as an async function returning a `Promise` (it no longer optionally accepts a callback as the third argument). All current Lambda Node runtimes support this invocation mode (so `exports.handler = server.createHandler()` will keep working without any changes), but if you've written your own handler which calls the handler returned by `createHandler` with a callback, you'll need to handle its `Promise` return value instead.
- `apollo-server-lambda`: This package is now implemented as a wrapper around `apollo-server-express`. `createHandler`'s argument now has different options: `expressGetMiddlewareOptions` which includes things like `cors` that is passed through to `apollo-server-express`'s `getMiddleware`, and `expressAppFromMiddleware` which lets you customize HTTP processing. The `context` function now receives an `express: { req, res }` option in addition to `event` and `context`.
- The `tracing` option to `new ApolloServer` has been removed, and the `apollo-server-tracing` package has been deprecated and is no longer being published. This package implemented an inefficient JSON format for execution traces returned on the `tracing` GraphQL response extension; it was only consumed by the deprecated `engineproxy` and Playground. If you really need this format, the old version of `apollo-server-tracing` should still work (`new ApolloServer({plugins: [require('apollo-server-tracing').plugin()]})`).
- The `cacheControl` option to `new ApolloServer` has been removed. The functionality provided by `cacheControl: true` or `cacheControl: {stripFormattedExtensions: false}` (which included a `cacheControl` extension in the GraphQL response, for use by the deprecated `engineproxy`) has been entirely removed. By default, Apollo Server continues to calculate an overall cache policy and to set the `Cache-Control` HTTP header, but this is now implemented directly inside `apollo-server-core` rather than a separate `apollo-cache-control` package (this package has been deprecated and is no longer being published). Tweaking cache control settings like `defaultMaxAge` is now done via the newly exported `ApolloServerPluginCacheControl` plugin rather than as a top-level constructor option. This follows the same pattern as the other built-in plugins like usage reporting. The `CacheHint` and `CacheScope` types are now exported from `apollo-server-types`.
- The `cacheControl` option to `new ApolloServer` has been removed. The functionality provided by `cacheControl: true` or `cacheControl: {stripFormattedExtensions: false}` (which included a `cacheControl` extension in the GraphQL response, for use by the deprecated `engineproxy`) has been entirely removed. By default, Apollo Server continues to calculate an overall cache policy and to set the `Cache-Control` HTTP header, but this is now implemented directly inside `apollo-server-core` rather than a separate `apollo-cache-control` package (this package has been deprecated and is no longer being published). Tweaking cache control settings like `defaultMaxAge` is now done via the newly exported `ApolloServerPluginCacheControl` plugin rather than as a top-level constructor option. This follows the same pattern as the other built-in plugins like usage reporting. The `CacheHint` and `CacheScope` types are now exported from `apollo-server-types`. The `info.cacheControl.cacheHint` object now has additional methods `replace`, `restrict`, and `policyIfCacheable`, and its fields update when those methods or `setCacheHint` are called. These methods also exist on `requestContext.overallCachePolicy`, which is always defined and which should not be overwritten (use `replace` instead). There is also a new function `info.cacheControl.cacheHintFromType` available. `@cacheControl` directives on type extensions are no longer ignored.
- When using a non-serverless framework integration (Express, Fastify, Hapi, Koa, Micro, or Cloudflare), you now *must* `await server.start()` before attaching the server to your framework. (This method was introduced in v2.22 but was optional before Apollo Server 3.) This does not apply to the batteries-included `apollo-server` or to serverless framework integrations.
- Top-level exports have changed. E.g.,

Expand Down
2 changes: 1 addition & 1 deletion docs/source/performance/caching.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ const resolvers = {

The `setCacheHint` method accepts an object with the same fields as [the `@cacheControl` directive](#in-your-schema-static).

The `cacheControl` object also has a `cacheHint` field which returns the hint set in the schema, if any. (Calling `info.cacheControl.setCacheHint` does not update `info.cacheControl.cacheHint`.)
The `cacheControl` object also has a `cacheHint` field which returns the field's current hint. This object also has a few other helpful methods, such as `info.cacheControl.cacheHint.restrict({ maxAge, scope })` which is similar to `setCacheHint` but it will never make `makeAge` larger or change `scope` from `PRIVATE` to `PUBLIC`. There is also a function `info.cacheControl.cacheHintFromType()` which takes an object type from a GraphQL AST and returns a cache hint which can be passed to `setCacheHint` or `restrict`; it may be useful for implementing resolvers that return unions or interfaces.

### Default `maxAge`

Expand Down
2 changes: 2 additions & 0 deletions packages/apollo-server-core/src/ApolloServer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ import {
ApolloServerPluginFrontendGraphQLPlayground,
} from './plugin';
import { InternalPluginId, pluginIsInternal } from './internalPlugin';
import { newCachePolicy } from './cachePolicy';

const NoIntrospection = (context: ValidationContext) => ({
Field(node: FieldDefinitionNode) {
Expand Down Expand Up @@ -912,6 +913,7 @@ export class ApolloServerBase {
},
},
debug: options.debug,
overallCachePolicy: newCachePolicy(),
};

return processGraphQLRequest(options, requestCtx);
Expand Down
109 changes: 109 additions & 0 deletions packages/apollo-server-core/src/__tests__/cachePolicy.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
import { CachePolicy, CacheScope } from 'apollo-server-types';
import { newCachePolicy } from '../cachePolicy';

describe('newCachePolicy', () => {
let cachePolicy: CachePolicy;
beforeEach(() => {
cachePolicy = newCachePolicy();
});

it('starts uncacheable', () => {
expect(cachePolicy.maxAge).toBeUndefined();
expect(cachePolicy.scope).toBeUndefined();
});

it('restricting maxAge positive makes restricted', () => {
cachePolicy.restrict({ maxAge: 10 });
});

it('restricting maxAge 0 makes restricted', () => {
cachePolicy.restrict({ maxAge: 0 });
});

it('restricting scope to private makes restricted', () => {
cachePolicy.restrict({ scope: CacheScope.Private });
});

it('returns lowest max age value', () => {
cachePolicy.restrict({ maxAge: 10 });
cachePolicy.restrict({ maxAge: 20 });

expect(cachePolicy.maxAge).toBe(10);
});

it('returns lowest max age value in other order', () => {
cachePolicy.restrict({ maxAge: 20 });
cachePolicy.restrict({ maxAge: 10 });

expect(cachePolicy.maxAge).toBe(10);
});

it('maxAge 0 if any cache hint has a maxAge of 0', () => {
cachePolicy.restrict({ maxAge: 120 });
cachePolicy.restrict({ maxAge: 0 });
cachePolicy.restrict({ maxAge: 20 });

expect(cachePolicy.maxAge).toBe(0);
});

it('returns undefined if first cache hint has a maxAge of 0', () => {
cachePolicy.restrict({ maxAge: 0 });
cachePolicy.restrict({ maxAge: 20 });

expect(cachePolicy.maxAge).toBe(0);
});

it('only restricting maxAge keeps scope undefined', () => {
cachePolicy.restrict({ maxAge: 10 });

expect(cachePolicy.scope).toBeUndefined();
});

it('returns PRIVATE scope if any cache hint has PRIVATE scope', () => {
cachePolicy.restrict({
maxAge: 10,
scope: CacheScope.Public,
});
cachePolicy.restrict({
maxAge: 10,
scope: CacheScope.Private,
});

expect(cachePolicy).toHaveProperty('scope', CacheScope.Private);
});

it('policyIfCacheable', () => {
expect(cachePolicy.policyIfCacheable()).toBeNull();

cachePolicy.restrict({ scope: CacheScope.Private });
expect(cachePolicy.scope).toBe(CacheScope.Private);
expect(cachePolicy.policyIfCacheable()).toBeNull();

cachePolicy.restrict({ maxAge: 10 });
expect(cachePolicy).toMatchObject({
maxAge: 10,
scope: CacheScope.Private,
});
expect(cachePolicy.policyIfCacheable()).toStrictEqual({
maxAge: 10,
scope: CacheScope.Private,
});

cachePolicy.restrict({ maxAge: 0 });
expect(cachePolicy).toMatchObject({
maxAge: 0,
scope: CacheScope.Private,
});
expect(cachePolicy.policyIfCacheable()).toBeNull();
});

it('replace', () => {
cachePolicy.restrict({ maxAge: 10, scope: CacheScope.Private });
cachePolicy.replace({ maxAge: 20, scope: CacheScope.Public });

expect(cachePolicy).toMatchObject({
maxAge: 20,
scope: CacheScope.Public,
});
});
});
2 changes: 2 additions & 0 deletions packages/apollo-server-core/src/__tests__/runQuery.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import {
} from 'apollo-server-plugin-base';
import { InMemoryLRUCache } from 'apollo-server-caching';
import { generateSchemaHash } from "../utils/schemaHash";
import { newCachePolicy } from '../cachePolicy';

// This is a temporary kludge to ensure we preserve runQuery behavior with the
// GraphQLRequestProcessor refactoring.
Expand Down Expand Up @@ -57,6 +58,7 @@ function runQuery(
context: options.context || {},
debug: options.debug,
cache: {} as any,
overallCachePolicy: newCachePolicy(),
...requestContextExtra,
});
}
Expand Down
33 changes: 33 additions & 0 deletions packages/apollo-server-core/src/cachePolicy.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import { CacheHint, CachePolicy, CacheScope } from 'apollo-server-types';

export function newCachePolicy(): CachePolicy {
return {
maxAge: undefined,
scope: undefined,
restrict(hint: CacheHint) {
if (
hint.maxAge !== undefined &&
(this.maxAge === undefined || hint.maxAge < this.maxAge)
) {
this.maxAge = hint.maxAge;
}
if (hint.scope !== undefined && this.scope !== CacheScope.Private) {
this.scope = hint.scope;
}
},
replace(hint: CacheHint) {
if (hint.maxAge !== undefined) {
this.maxAge = hint.maxAge;
}
if (hint.scope !== undefined) {
this.scope = hint.scope;
}
},
policyIfCacheable() {
if (this.maxAge === undefined || this.maxAge === 0) {
return null;
}
return { maxAge: this.maxAge, scope: this.scope ?? CacheScope.Public };
},
};
}
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,35 @@ describe('@cacheControl directives', () => {
expect(hints).toStrictEqual(new Map([['droid', { maxAge: 60 }]]));
});

it('should set the specified maxAge for a field from a cache hint on the target type extension', async () => {
const schema = buildSchemaWithCacheControlSupport(`
type Query {
droid(id: ID!): Droid
}
type Droid {
id: ID!
name: String!
}
extend type Droid @cacheControl(maxAge: 60)
`);

const hints = await collectCacheControlHints(
schema,
`
query {
droid(id: 2001) {
name
}
}
`,
{ defaultMaxAge: 10 },
);

expect(hints).toStrictEqual(new Map([['droid', { maxAge: 60 }]]));
});

it('should overwrite the default maxAge when maxAge=0 is specified on the type', async () => {
const schema = buildSchemaWithCacheControlSupport(`
type Query {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,10 @@ import { Headers } from 'apollo-server-env';
import {
CacheHint,
CacheScope,
GraphQLRequestContext,
} from 'apollo-server-types';
import {
ApolloServerPluginCacheControl,
ApolloServerPluginCacheControlOptions,
PolicyUpdater,
} from '../';
import {
GraphQLRequestContextWillSendResponse,
Expand Down Expand Up @@ -114,64 +112,4 @@ describe('plugin', () => {
});
});
});

describe('PolicyUpdater', () => {
let hints: PolicyUpdater;
let requestContext: Pick<GraphQLRequestContext, 'overallCachePolicy'>;
beforeEach(() => {
requestContext = {};
hints = new PolicyUpdater(requestContext);
});

it('returns undefined without cache hints', () => {
expect(requestContext.overallCachePolicy).toBeUndefined();
});

it('returns lowest max age value', () => {
hints.addHint({ maxAge: 10 });
hints.addHint({ maxAge: 20 });

expect(requestContext.overallCachePolicy).toHaveProperty('maxAge', 10);
});

it('returns undefined if any cache hint has a maxAge of 0', () => {
hints.addHint({ maxAge: 120 });
hints.addHint({ maxAge: 0 });
hints.addHint({ maxAge: 20 });

expect(requestContext.overallCachePolicy).toBeUndefined();
});

it('returns undefined if first cache hint has a maxAge of 0', () => {
hints.addHint({ maxAge: 0 });
hints.addHint({ maxAge: 20 });

expect(requestContext.overallCachePolicy).toBeUndefined();
});

it('returns PUBLIC scope by default', () => {
hints.addHint({ maxAge: 10 });

expect(requestContext.overallCachePolicy).toHaveProperty(
'scope',
CacheScope.Public,
);
});

it('returns PRIVATE scope if any cache hint has PRIVATE scope', () => {
hints.addHint({
maxAge: 10,
scope: CacheScope.Public,
});
hints.addHint({
maxAge: 10,
scope: CacheScope.Private,
});

expect(requestContext.overallCachePolicy).toHaveProperty(
'scope',
CacheScope.Private,
);
});
});
});
Loading

0 comments on commit ad6cfa3

Please sign in to comment.