Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JSONSelection] Support -> method syntax for inline data transforms #5762

Merged
merged 49 commits into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
7b0c489
Update JSONSelection SVG railroad diagrams.
benjamn Jul 15, 2024
7ce2d5d
Refactor PathSelection enum into struct containing private PathList.
benjamn Jul 15, 2024
4f51926
Decompose SubSelection::parse_naked convenience method.
benjamn Jul 15, 2024
01bb990
Remove never-used Key::Index variant.
benjamn Jul 15, 2024
1fd049b
Forbid space between $ and variable name, as mandated by grammar.
benjamn Jul 15, 2024
f2329ba
Stop forcing top-level JSONSelection to be mapped over arrays.
benjamn Jul 15, 2024
0c82cbe
Bind $ to the root value of the current SubSelection or PathSelection.
benjamn Jul 15, 2024
38d9edd
Immutable &InputPath<JSON> instead of input_path: &mut Vec<JSON>
benjamn Jul 15, 2024
3b77944
Add guiding principle about safe subsetting of JSONSelection syntax.
benjamn Jul 15, 2024
982540b
Support path->method(...) syntax with various built-in methods.
benjamn May 7, 2024
66dfb0d
Replace $typenames.T with $->echo("T").
benjamn Aug 2, 2024
5a39b4c
Support ->get(indexOrKey) method for arrays, strings, and objects.
benjamn Aug 2, 2024
e9e5ba0
Support ->size method for arrays, strings, and objects.
benjamn Aug 2, 2024
61566c7
Support ->has method for array, string, and object indexes/keys.
benjamn Aug 2, 2024
89e6022
Use `SubSelection` in README rather than braces syntax
benjamn Aug 12, 2024
a0c5d92
Remove overcomplicated justification from Guiding Principle #3.
benjamn Aug 13, 2024
85a92e8
Support ->keys, ->values, and ->entries methods for objects.
benjamn Aug 2, 2024
d73bb4d
Make ApplyToError::new take String message instead of &str.
benjamn Aug 14, 2024
50b8e29
Fast path for @ quasi-variable.
benjamn Aug 14, 2024
c57b20d
More comments for PathList variants.
benjamn Aug 14, 2024
2c69781
Report input_path as context for missing variables.
benjamn Aug 14, 2024
c45b068
Rename JS{Literal,Number,...} to Lit{Expr,Number,...}.
benjamn Aug 14, 2024
ab8bcd7
Use serde_json::Number instead of String for LitExpr::Number.
benjamn Aug 14, 2024
491232b
Allow trailing commas in MethodArgs, LitObject, and LitArray.
benjamn Aug 14, 2024
a56b453
Fix bug causing LitObject keys to be prefixed with '.'.
benjamn Aug 14, 2024
bbd0918
Improve immutable InputPath<T>, avoiding recursive drop.
benjamn Aug 15, 2024
489cf55
Implement Key::as_str to avoid cloning new String.
benjamn Aug 16, 2024
d43d32e
Restrict privacy to pub(super) for Path{Selection,List} methods.
benjamn Aug 16, 2024
f63ceee
Add more tests of LitExpr parsing.
benjamn Aug 16, 2024
83009b4
Preallocate serde_json_bytes::Map and Vec<JSON> of known sizes.
benjamn Aug 16, 2024
9d45d58
Implement CollectVarPaths trait for JSONSelection AST structures.
benjamn Aug 16, 2024
b324444
Remove unused Serialize trait implementations.
benjamn Aug 17, 2024
98a851a
Merge branch 'next' into benjamn/JSONSelection-nom_locate.
benjamn Aug 19, 2024
533e485
Use .collect_var_paths() instead of StaticParameter abstraction.
benjamn Aug 19, 2024
b8cf701
Restrict -> methods to small initial vocabulary.
benjamn Aug 19, 2024
242d3a1
Merge branch 'next' into benjamn/JSONSelection-method-syntax.
benjamn Aug 21, 2024
7353f00
Remove single-element [<default>] case from ->match method.
benjamn Aug 21, 2024
3407386
Use HashSet::with_capacity_and_hasher.
benjamn Aug 22, 2024
3c92561
Remove Hash requirement from CollectVarPaths trait.
benjamn Aug 22, 2024
94a0186
Add names to ArrowMethod function parameters.
benjamn Aug 22, 2024
8c18a5f
Use //! module comment syntax for lit_expr.rs.
benjamn Aug 22, 2024
f1daee6
Move non-public methods to future:: namespace in methods.rs.
benjamn Aug 23, 2024
60b8777
Break methods.rs file into methods/{future,public,tests}.rs.
benjamn Aug 26, 2024
0a71e1c
Rename `ApplyTo` to `ApplyToInternal` and keep it `pub(super)`.
benjamn Aug 26, 2024
c2c130c
Use KnownVariable enum for better variable-related rustc feedback.
benjamn Aug 26, 2024
599f064
Rename CollectVarPaths to ExternalVarPaths.
benjamn Aug 26, 2024
483adf8
TODO Parameter::Context
benjamn Aug 26, 2024
05f681d
De-nest extract_params_from_body.
benjamn Aug 26, 2024
faada35
Support $config as KnownVariable (not $context).
benjamn Aug 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,12 @@ expression: connectors.by_service_name
body: None,
},
selection: Path(
Var(
"$",
Empty,
),
PathSelection {
path: Var(
"$",
Empty,
),
},
),
config: None,
entity_resolver: None,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -202,10 +202,12 @@ expression: connectors.by_service_name
body: None,
},
selection: Path(
Var(
"$",
Empty,
),
PathSelection {
path: Var(
"$",
Empty,
),
},
),
config: None,
entity_resolver: Some(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -217,15 +217,17 @@ expression: connectors.by_service_name
body: None,
},
selection: Path(
Var(
"$",
Key(
Field(
"phone",
PathSelection {
path: Var(
"$",
Key(
Field(
"phone",
),
Empty,
),
Empty,
),
),
},
),
config: None,
entity_resolver: Some(
Expand Down
163 changes: 139 additions & 24 deletions apollo-federation/src/sources/connect/json_selection/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,23 @@ improvements, we should adhere to the following principles:
avoided because it limits the developer's ability to subselect fields of the
opaque `JSON` value in GraphQL operations.

3. Backwards compatibility should be maintained as we release new versions of
3. `JSONSelection` syntax may be subsetted arbitrarily, either by generating a
reduced `JSONSelection` that serves the needs of a particular GraphQL
operation, or by skipping unneeded selections during `ApplyTo` execution.
When this subsetting happens, it would be highly undesirable for the behavior
of the remaining selections to change unexpectedly.

For example, it is simpler to insist that `{...}` selections are always
mapped over input arrays, rather than sometimes mapping (when named fields
are present, say) and sometimes applying directly to the array itself (when
only array-safe `$->method` selections are present, hypothetically). Even if
array safety was easy to decide, it would be a mistake to base the mapping
decision on the presence or absence of certain kinds of selections, because
then the mapping behavior could change unexpectedly whenever certain kinds of
selections happen to be omitted, which is an ever-present possibility thanks
to subsetting.
benjamn marked this conversation as resolved.
Show resolved Hide resolved

4. Backwards compatibility should be maintained as we release new versions of
the `JSONSelection` syntax along with new versions of the (forthcoming)
`@link(url: "https://specs.apollo.dev/connect/vX.Y")` specification. Wherever
possible, we should only add new functionality, not remove or change existing
Expand All @@ -81,9 +97,10 @@ NamedFieldSelection ::= Alias? Identifier SubSelection?
NamedQuotedSelection ::= Alias StringLiteral SubSelection?
NamedGroupSelection ::= Alias SubSelection
Alias ::= Identifier ":"
PathSelection ::= (VarPath | KeyPath) SubSelection?
PathSelection ::= (VarPath | KeyPath | AtPath) SubSelection?
VarPath ::= "$" (NO_SPACE Identifier)? PathStep*
KeyPath ::= Key PathStep+
AtPath ::= "@" PathStep*
PathStep ::= "." Key | "->" Identifier MethodArgs?
Key ::= Identifier | StringLiteral
Identifier ::= [a-zA-Z_] NO_SPACE [0-9a-zA-Z_]*
Expand All @@ -99,7 +116,7 @@ JSArray ::= "[" (JSLiteral ("," JSLiteral)*)? "]"
StarSelection ::= Alias? "*" SubSelection?
NO_SPACE ::= !SpacesOrComments
SpacesOrComments ::= (Spaces | Comment)+
Spaces ::= (" " | "\t" | "\r" | "\n")+
Spaces ::= ("" | "\t" | "\r" | "\n")+
benjamn marked this conversation as resolved.
Show resolved Hide resolved
Comment ::= "#" [^\n]*
```

Expand Down Expand Up @@ -472,8 +489,8 @@ type User @key(fields: "id") {
```

In addition to variables like `$this` and `$args`, a special `$` variable is
always bound to the current value being processed, which allows you to transform
input data that looks like this
always bound to the value received by the closest enclosing `{...}` selection
set, which allows you to transform input data that looks like this
benjamn marked this conversation as resolved.
Show resolved Hide resolved

```json
{
Expand Down Expand Up @@ -560,6 +577,52 @@ character (so `.data { id name }`, or even `.data.id` or `.data.name`) to mean
the same thing as `$.`, but this is no longer recommended, since `.data` is easy
to mistype and misread, compared to `$.data`.

### `AtPath ::=`

![AtPath](./grammar/AtPath.svg)

Similar to the special `$` variable, the `@` character always represents the
current value being processed, which is often equal to `$`, but may differ from
the `$` variable when `@` is used within the arguments of `->` methods.

For example, when you want to compute the logical conjunction of several
properties of the current object, you can keep using `$` with different property
selections:

```graphql
all: $.first->and($.second)->and($.third)
```

If the `$` variable were rebound to the input value received by the `->and`
method, this style of method chaining would not work, because the `$.second`
expression would attempt to select a `second` property from the value of
`$.first`. Instead, the `$` remains bound to the same value received by the
closest enclosing `{...}` selection set, or the root value when used at the top
level of a `JSONSelection`.
benjamn marked this conversation as resolved.
Show resolved Hide resolved

The `@` character becomes useful when you need to refer to the input value
received by a `->` method, as when using the `->echo` method to wrap a given
input value:

```graphql
wrapped: field->echo({ fieldValue: @ })
children: parent->echo([@.child1, @.child2, @.child3])
```

The `->map` method has the special ability to apply its argument to each element
of its input array, so `@` will take on the value of each of those elements,
rather than referring to the array itself:

```graphql
doubled: numbers->map({ value: @->mul(2) })
types: values->map(@->typeof)
```

This special behavior of `@` within `->map` is available to any method
implementation, since method arguments are not evaluated before calling the
method, but are passed in as expressions that the method may choose to evaluate
(or even repeatedly reevaluate) however it chooses.

### `PathStep ::=`

![PathStep](./grammar/PathStep.svg)
Expand All @@ -574,25 +637,77 @@ registered in the `JSONSelection` parser in order to be recognized.
For the time being, only a fixed set of known methods are supported, though this
list may grow and/or become user-configurable in the future:

> Full disclosure: even this list is still aspirational, but suggestive of the
> kinds of methods that are likely to be supported in the next version of the
> `JSONSelection` parser.

```graphql
list->first { id name }
list->last.name
list->slice($args.start, $args.end)
list->reverse
some.value->times(2)
some.value->plus($addend)
some.value->minus(100)
some.value->div($divisor)
isDog: kind->eq("dog")
isNotCat: kind->neq("cat")
__typename: kind->match({ "dog": "Dog", "cat": "Cat" })
decoded: utf8Bytes->decode("utf-8")
utf8Bytes: string->encode("utf-8")
encoded: bytes->encode("base64")
# The ->echo method returns its first input argument as-is, ignoring
# the input data. Useful for embedding literal values, as in
# $->echo("give me this string"), or wrapping the input value.
__typename: $->echo("Book")
wrapped: field->echo({ fieldValue: @ })

# Returns the type of the data as a string, e.g. "object", "array",
# "string", "number", "boolean", or "null". Note that `typeof null` is
# "object" in JavaScript but "null" for our purposes.
typeOfValue: value->typeof

# When invoked against an array, ->map evaluates its first argument
# against each element of the array, binding the element values to `@`,
# and returns an array of the results. When invoked against a non-array,
# ->map evaluates its first argument against that value and returns the
# result without wrapping it in an array.
doubled: numbers->map(@->mul(2))
types: values->map(@->typeof)

# Returns true if the data is deeply equal to the first argument, false
# otherwise. Equality is solely value-based (all JSON), no references.
isObject: value->typeof->eq("object")

# Takes any number of pairs [candidate, value], and returns value for
# the first candidate that equals the input data. If none of the
# pairs match, a runtime error is reported, but a single-element
# [<default>] array as the final argument guarantees a default value.
__typename: kind->match(
["dog", "Canine"],
["cat", "Feline"],
["Exotic"]
)
dylan-apollo marked this conversation as resolved.
Show resolved Hide resolved

# Like ->match, but expects the first element of each pair to evaluate
# to a boolean, returning the second element of the first pair whose
# first element is true. This makes providing a final catch-all case
# easy, since the last pair can be [true, <default>].
__typename: kind->matchIf(
[@->eq("dog"), "Canine"],
[@->eq("cat"), "Feline"],
[true, "Exotic"]
dylan-apollo marked this conversation as resolved.
Show resolved Hide resolved
)

# Arithmetic methods, supporting both integers and floating point values,
# similar to JavaScript.
sum: $.a->add($.b)->add($.c)
difference: $.a->sub($.b)->sub($.c)
product: $.a->mul($.b, $.c)
quotient: $.a->div($.b)
remainder: $.a->mod($.b)
benjamn marked this conversation as resolved.
Show resolved Hide resolved

# Array/string methods
benjamn marked this conversation as resolved.
Show resolved Hide resolved
first: list->first
last: list->last
index3: list->get(3)
secondToLast: list->get(-2)
# The ->get method also works for object keys.
aValue: $->echo({ a: 123 })->get("a")
slice: list->slice(0, 5)
benjamn marked this conversation as resolved.
Show resolved Hide resolved
arraySize: array->size
stringLength: string->size
numberOfProperties: object->size

# Logical methods
negation: $.condition->not
bangBang: $.condition->not->not
disjunction: $.a->or($.b)->or($.c)
conjunction: $.a->and($.b, $.c)
aImpliesB: $.a->not->or($.b)
excludedMiddle: $.toBe->or($.toBe->not)->eq(true)
dylan-apollo marked this conversation as resolved.
Show resolved Hide resolved
```

### `MethodArgs ::=`
Expand All @@ -601,7 +716,7 @@ encoded: bytes->encode("base64")

When a `PathStep` invokes an `->operator` method, the method invocation may
optionally take a sequence of comma-separated `JSLiteral` arguments in
parentheses, as in `list->slice(0, 5)` or `kilometers: miles->times(1.60934)`.
parentheses, as in `list->slice(0, 5)` or `kilometers: miles->mul(1.60934)`.

Methods do not have to take arguments, as in `list->first` or `list->last`,
which is why `MethodArgs` is optional in `PathStep`.
Expand Down
Loading