YQL-18053: Add block implementation for AsStruct callable #3485
Conversation
|
⚪
|
|
⚪
|
| @@ -5468,7 +5468,7 @@ bool CollectBlockRewrites(const TMultiExprType* multiInputType, bool keepInputCo | |||
|
|
|||
| TExprNode::TListType funcArgs; | |||
| std::string_view arrowFunctionName; | |||
| if (node->IsList() || node->IsCallable({"And", "Or", "Xor", "Not", "Coalesce", "If", "Just", "Nth", "ToPg", "FromPg", "PgResolvedCall", "PgResolvedOp"})) | |||
| if (node->IsList() || node->IsCallable({"And", "Or", "Xor", "Not", "Coalesce", "If", "Just", "Member", "Nth", "ToPg", "FromPg", "PgResolvedCall", "PgResolvedOp"})) | |||
There was a problem hiding this comment.
The revision was irrelevant. Force-pushed.
965f431 to
66c3d24
Compare
|
⚪
|
|
⚪
|
66c3d24 to
80ceb23
Compare
|
⚪
|
|
⚪
|
80ceb23 to
568b6c8
Compare
|
⚪
|
|
⚪
|
568b6c8 to
3fa0116
Compare
|
⚪
|
|
⚪
|
3fa0116 to
f8dc908
Compare
|
⚪
|
|
⚪
|
<AsStruct> arguments (i.e. the members of the resulting structure object) are literal tuples, that don't propagate their child rewrites. As a result, if there is a block rewrite occurred in the expression graph below any of these lists, it takes no effect above this "list barrier". Hence, all <AsStruct> children should be traversed separately to either wrap the complete expressions, supported by the block engine, with <AsScalar> callable, or apply the necessary rewrites. If none of the conditions above are met, the block rewrite for <AsStruct> node is aborted, since one of the children being proceseed is rewritten neither to block, nor scalar, ergo, violating <BlockAsStruct> type requirements.
Since Tuple type support in block execution is implemented via Arrow Struct type, all the machinery is renamed by s/AsTuple/AsContainer/ rule and will be used for Struct type support either.
f8dc908 to
3383779
Compare
|
⚪
|
|
⚪
|
3383779 to
e40c78f
Compare
|
⚪
|
|
⚪
|
…b-platform#3485)" This reverts commit f1bf06c since there are several broken tests, that were not fixed in ydb-platform#3485.
This check is already implemented in <TBlockFuncNode::DoCalculate>, so if the type annotation is invalid or a given Arrow Datum doesn't match the annotated type, the debug build will hit the assertion. This patch makes the Datum type check in <TWideFromBlocksWrapper::TState::Get> as strict as in <TBlockFuncNode::DoCalculate>. Relates to ydb-platform#3485
This check is already implemented in <TBlockFuncNode::DoCalculate>, so if the type annotation is invalid or a given Arrow Datum doesn't match the annotated type, the debug build will hit the assertion. This patch makes the Datum type check in <TWideFromBlocksWrapper::TState::Get> as strict as in <TBlockFuncNode::DoCalculate>. Relates to ydb-platform#3485
This patchset introduces block implementation for
AsStructcallable. The semantics of this callable differs fromAsTuplecallable only in the resulting type of the object being processed (structure vs tuple). Hence, the first patch of the series turnsNthcallable block implementation into genericAsContainerhelper, being used by bothAsTupleandAsStructruntime node emitter.Apart from the type annotation and runtime node emitter, there is a patch in the series that makes
CollectBlockRewriteshandle the rewrites related to theAsStructarguments.AsStructarguments (i.e. the members of the resulting structure object) are literal tuples, that don't propagate their child rewrites. As a result, if there is a block rewrite occurred in the expression graph below any of these lists, it takes no effect above this "list barrier". Hence, allAsStructchildren should be traversed separately to either wrap the complete expressions, supported by the block engine, withAsScalarcallable, or apply the necessary rewrites. If none of the conditions above are met, the block rewrite forAsStructnode is aborted, since one of the children being proceseed is rewritten neither to block, nor scalar, ergo, violatingBlockAsStructtype requirements.Changelog category