Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JSONSelection] Support -> method syntax for inline data transforms #5762

Merged
merged 49 commits into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
7b0c489
Update JSONSelection SVG railroad diagrams.
benjamn Jul 15, 2024
7ce2d5d
Refactor PathSelection enum into struct containing private PathList.
benjamn Jul 15, 2024
4f51926
Decompose SubSelection::parse_naked convenience method.
benjamn Jul 15, 2024
01bb990
Remove never-used Key::Index variant.
benjamn Jul 15, 2024
1fd049b
Forbid space between $ and variable name, as mandated by grammar.
benjamn Jul 15, 2024
f2329ba
Stop forcing top-level JSONSelection to be mapped over arrays.
benjamn Jul 15, 2024
0c82cbe
Bind $ to the root value of the current SubSelection or PathSelection.
benjamn Jul 15, 2024
38d9edd
Immutable &InputPath<JSON> instead of input_path: &mut Vec<JSON>
benjamn Jul 15, 2024
3b77944
Add guiding principle about safe subsetting of JSONSelection syntax.
benjamn Jul 15, 2024
982540b
Support path->method(...) syntax with various built-in methods.
benjamn May 7, 2024
66dfb0d
Replace $typenames.T with $->echo("T").
benjamn Aug 2, 2024
5a39b4c
Support ->get(indexOrKey) method for arrays, strings, and objects.
benjamn Aug 2, 2024
e9e5ba0
Support ->size method for arrays, strings, and objects.
benjamn Aug 2, 2024
61566c7
Support ->has method for array, string, and object indexes/keys.
benjamn Aug 2, 2024
89e6022
Use `SubSelection` in README rather than braces syntax
benjamn Aug 12, 2024
a0c5d92
Remove overcomplicated justification from Guiding Principle #3.
benjamn Aug 13, 2024
85a92e8
Support ->keys, ->values, and ->entries methods for objects.
benjamn Aug 2, 2024
d73bb4d
Make ApplyToError::new take String message instead of &str.
benjamn Aug 14, 2024
50b8e29
Fast path for @ quasi-variable.
benjamn Aug 14, 2024
c57b20d
More comments for PathList variants.
benjamn Aug 14, 2024
2c69781
Report input_path as context for missing variables.
benjamn Aug 14, 2024
c45b068
Rename JS{Literal,Number,...} to Lit{Expr,Number,...}.
benjamn Aug 14, 2024
ab8bcd7
Use serde_json::Number instead of String for LitExpr::Number.
benjamn Aug 14, 2024
491232b
Allow trailing commas in MethodArgs, LitObject, and LitArray.
benjamn Aug 14, 2024
a56b453
Fix bug causing LitObject keys to be prefixed with '.'.
benjamn Aug 14, 2024
bbd0918
Improve immutable InputPath<T>, avoiding recursive drop.
benjamn Aug 15, 2024
489cf55
Implement Key::as_str to avoid cloning new String.
benjamn Aug 16, 2024
d43d32e
Restrict privacy to pub(super) for Path{Selection,List} methods.
benjamn Aug 16, 2024
f63ceee
Add more tests of LitExpr parsing.
benjamn Aug 16, 2024
83009b4
Preallocate serde_json_bytes::Map and Vec<JSON> of known sizes.
benjamn Aug 16, 2024
9d45d58
Implement CollectVarPaths trait for JSONSelection AST structures.
benjamn Aug 16, 2024
b324444
Remove unused Serialize trait implementations.
benjamn Aug 17, 2024
98a851a
Merge branch 'next' into benjamn/JSONSelection-nom_locate.
benjamn Aug 19, 2024
533e485
Use .collect_var_paths() instead of StaticParameter abstraction.
benjamn Aug 19, 2024
b8cf701
Restrict -> methods to small initial vocabulary.
benjamn Aug 19, 2024
242d3a1
Merge branch 'next' into benjamn/JSONSelection-method-syntax.
benjamn Aug 21, 2024
7353f00
Remove single-element [<default>] case from ->match method.
benjamn Aug 21, 2024
3407386
Use HashSet::with_capacity_and_hasher.
benjamn Aug 22, 2024
3c92561
Remove Hash requirement from CollectVarPaths trait.
benjamn Aug 22, 2024
94a0186
Add names to ArrowMethod function parameters.
benjamn Aug 22, 2024
8c18a5f
Use //! module comment syntax for lit_expr.rs.
benjamn Aug 22, 2024
f1daee6
Move non-public methods to future:: namespace in methods.rs.
benjamn Aug 23, 2024
60b8777
Break methods.rs file into methods/{future,public,tests}.rs.
benjamn Aug 26, 2024
0a71e1c
Rename `ApplyTo` to `ApplyToInternal` and keep it `pub(super)`.
benjamn Aug 26, 2024
c2c130c
Use KnownVariable enum for better variable-related rustc feedback.
benjamn Aug 26, 2024
599f064
Rename CollectVarPaths to ExternalVarPaths.
benjamn Aug 26, 2024
483adf8
TODO Parameter::Context
benjamn Aug 26, 2024
05f681d
De-nest extract_params_from_body.
benjamn Aug 26, 2024
faada35
Support $config as KnownVariable (not $context).
benjamn Aug 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 67 additions & 31 deletions apollo-federation/src/sources/connect/expand/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -206,8 +206,7 @@ mod helpers {
use crate::schema::position::TypeDefinitionPosition;
use crate::schema::FederationSchema;
use crate::schema::ValidFederationSchema;
use crate::sources::connect::json_selection::ExtractParameters;
use crate::sources::connect::json_selection::StaticParameter;
use crate::sources::connect::json_selection::KnownVariable;
use crate::sources::connect::url_template::Parameter;
use crate::sources::connect::ConnectSpecDefinition;
use crate::sources::connect::Connector;
Expand Down Expand Up @@ -514,6 +513,10 @@ mod helpers {
}
}

Parameter::Config { item: _, paths: _ } => {
// TODO Implement $config handling
}

// All sibling fields marked by $this in a transport must be carried over to the output type
// regardless of its use in the output selection.
Parameter::Sibling { field, paths } => {
Expand Down Expand Up @@ -799,36 +802,69 @@ mod helpers {
fn extract_params_from_body(
connector: &Connector,
) -> Result<HashSet<Parameter>, FederationError> {
let body_parameters = connector
.transport
.body
.as_ref()
.and_then(JSONSelection::extract_parameters)
.unwrap_or_default();

body_parameters
.iter()
.map(|StaticParameter { name, paths }| {
let mut parts = paths.iter();
let field = parts.next().ok_or(FederationError::internal(
"expected parameter in JSONSelection to contain a field",
))?;

match *name {
"$args" => Ok(Parameter::Argument {
argument: field,
paths: parts.copied().collect(),
}),
"$this" => Ok(Parameter::Sibling {
field,
paths: parts.copied().collect(),
}),
other => Err(FederationError::internal(format!(
"got unsupported parameter: {other}"
))),
let Some(body) = &connector.transport.body else {
return Ok(HashSet::default());
};

use crate::sources::connect::json_selection::ExternalVarPaths;
let var_paths = body.external_var_paths();

let mut results = HashSet::with_capacity_and_hasher(var_paths.len(), Default::default());

for var_path in var_paths {
match var_path.var_name_and_nested_keys() {
Some((KnownVariable::Args, keys)) => {
let mut keys_iter = keys.into_iter();
let first_key = keys_iter.next().ok_or(FederationError::internal(
"expected at least one key in $args",
))?;
results.insert(Parameter::Argument {
argument: first_key,
paths: keys_iter.collect(),
});
}
Some((KnownVariable::This, keys)) => {
let mut keys_iter = keys.into_iter();
let first_key = keys_iter.next().ok_or(FederationError::internal(
"expected at least one key in $this",
))?;
results.insert(Parameter::Sibling {
field: first_key,
paths: keys_iter.collect(),
});
}
})
.collect::<Result<HashSet<_>, _>>()
Some((KnownVariable::Config, keys)) => {
let mut keys_iter = keys.into_iter();
let first_key = keys_iter.next().ok_or(FederationError::internal(
"expected at least one key in $config",
))?;
results.insert(Parameter::Config {
item: first_key,
paths: keys_iter.collect(),
});
}
// To get the safety benefits of the KnownVariable enum, we need
// to enumerate all the cases explicitly, without wildcard
// matches. However, body.external_var_paths() only returns free
// (externally-provided) variables like $this, $args, and
// $config. The $ and @ variables, by contrast, are always bound
// to something within the input data.
Some((kv @ KnownVariable::Dollar, _)) | Some((kv @ KnownVariable::AtSign, _)) => {
return Err(FederationError::internal(format!(
"got unexpected non-external variable: {:?}",
kv,
)));
}
None => {
return Err(FederationError::internal(format!(
"could not extract variable from path: {:?}",
var_path,
)));
}
};
}

Ok(results)
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,12 @@ expression: connectors.by_service_name
body: None,
},
selection: Path(
Var(
"$",
Empty,
),
PathSelection {
path: Var(
$,
Empty,
),
},
),
config: None,
max_requests: None,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,15 +64,17 @@ expression: connectors.by_service_name
Alias {
name: "emailDomain",
},
Var(
"$args",
Key(
Field(
"email",
PathSelection {
path: Var(
$args,
Key(
Field(
"email",
),
Empty,
),
Empty,
),
),
},
),
],
star: None,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -209,15 +209,17 @@ expression: connectors.by_service_name
Alias {
name: "with_b",
},
Var(
"$this",
Key(
Field(
"b",
PathSelection {
path: Var(
$this,
Key(
Field(
"b",
),
Empty,
),
Empty,
),
),
},
),
],
star: None,
Expand All @@ -226,10 +228,12 @@ expression: connectors.by_service_name
),
},
selection: Path(
Var(
"$",
Empty,
),
PathSelection {
path: Var(
$,
Empty,
),
},
),
config: None,
max_requests: None,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -219,15 +219,17 @@ expression: connectors.by_service_name
body: None,
},
selection: Path(
Var(
"$",
Key(
Field(
"phone",
PathSelection {
path: Var(
$,
Key(
Field(
"phone",
),
Empty,
),
Empty,
),
),
},
),
config: None,
max_requests: None,
Expand Down
Loading