Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor tree_map and replace apply_to_primitive_constituents #1570

Open
wants to merge 76 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
8add029
refactor[next]: itir embedded: cleaner closure run
havogt Apr 4, 2024
853d3e1
cleanup
havogt Apr 4, 2024
f661cd3
fix test
havogt Apr 4, 2024
09e568d
without temporaries
havogt Apr 8, 2024
12b8696
temporaries
havogt Apr 8, 2024
540a2d8
cleanup
havogt Apr 9, 2024
23ddef1
move to SetAt
havogt Apr 10, 2024
e64b986
Merge branch 'refactor_itir_embedded' into itir_program_embedded2
havogt Apr 10, 2024
c99f44d
embedded
havogt Apr 10, 2024
1a6f885
roundtrip+double_roundtrip with shortcuts
havogt Apr 11, 2024
39d6d7c
changes
havogt Apr 11, 2024
ab44009
fencil2program only for gtfn
havogt Apr 11, 2024
12f1663
fix import
havogt Apr 11, 2024
aa80949
Merge remote-tracking branch 'upstream/main' into itir_program
havogt Apr 11, 2024
5037493
fix builtins list
havogt Apr 11, 2024
751581e
add comment
havogt Apr 11, 2024
3d2f33e
fix type checker
havogt Apr 11, 2024
53bad75
Merge branch 'itir_program' into itir_program_embedded2
havogt Apr 11, 2024
4cbce7e
Apply suggestions from code review
havogt Apr 12, 2024
c955645
format
havogt Apr 12, 2024
6effe10
pretty printing/parsing
havogt Apr 12, 2024
66de3ec
Apply suggestions from code review
havogt Apr 15, 2024
e63da77
address more review comments
havogt Apr 15, 2024
45fba85
move tmp to pretty_printer
havogt Apr 15, 2024
1a70218
pparse for temporaries
havogt Apr 15, 2024
c39c603
rename gtfn.FencilDefinition -> Program
havogt Apr 15, 2024
705cfcf
remove TODO
havogt Apr 15, 2024
c5e78c4
Apply suggestions from code review
havogt Apr 15, 2024
a336bf5
rename as_field_operator -> as_fieldop
havogt Apr 15, 2024
af16f40
missed a file
havogt Apr 15, 2024
2d6bfbf
Merge branch 'itir_program' into itir_program_embedded2
havogt Apr 15, 2024
df1146b
Merge remote-tracking branch 'upstream/main' into itir_program_embedded2
havogt Apr 15, 2024
bc2c2d3
add fencil2program to roundtrip
havogt Apr 16, 2024
c7ccd6a
Merge remote-tracking branch 'upstream/main' into itir_program_embedded2
havogt Apr 16, 2024
f45b460
pre-allocate result buffer
havogt Apr 17, 2024
5d4fc3d
fix tracer context
havogt Apr 17, 2024
1d93192
first (almost) complete embedded version
havogt Apr 17, 2024
5882c28
add dim kind to print/parse
havogt Apr 17, 2024
b7cbf16
fix tests
havogt Apr 17, 2024
e97ca25
cleanup test_program
havogt Apr 17, 2024
8c2bd8f
re-enable lift mode in roundtrip
havogt Apr 18, 2024
21b230b
replace lift_mode fixture by backend in program_processor
havogt Apr 18, 2024
0946783
fix doctests
havogt Apr 18, 2024
f93da09
fix tests
havogt Apr 18, 2024
97663bd
undo quickstart changes
havogt Apr 18, 2024
3297f7b
undo delete cpp_backend_tests
havogt Apr 18, 2024
f37b372
fix quickstart guide again
havogt Apr 18, 2024
e242ab6
remove runtime lift
havogt Apr 19, 2024
a07d8ea
Merge remote-tracking branch 'upstream/main' into itir_program_embedded2
havogt Apr 19, 2024
3c2b9a5
Merge remote-tracking branch 'upstream/main' into test_lift_mode_to_p…
havogt Apr 19, 2024
a9f1043
Update docs/user/next/QuickstartGuide.md
havogt Apr 19, 2024
e7195a5
cleanup out field construction
havogt Apr 19, 2024
3f67746
Update src/gt4py/next/program_processors/runners/double_roundtrip.py
havogt Apr 22, 2024
369eae7
read config.DEBUG at execution
havogt Apr 22, 2024
35f2132
remove LiftMode.SIMPLE_HEURISTIC
havogt Apr 22, 2024
4a4f9b1
fix formatting
havogt Apr 23, 2024
2825588
Merge branch 'test_lift_mode_to_processor' into itir_program_embedded2
havogt Apr 23, 2024
65abde7
Merge remote-tracking branch 'upstream/main' into itir_program_embedded2
havogt Apr 23, 2024
7373eb6
Merge remote-tracking branch 'upstream/main' into itir_program_embedded2
havogt May 6, 2024
bfcc118
move ordering of unstructured domain to gtfn
havogt May 15, 2024
54f44cc
fix problem in column dtype if contains None
havogt May 16, 2024
b8b26e6
address more review comments
havogt May 16, 2024
b7b489e
Merge remote-tracking branch 'upstream/main' into itir_program_embedded2
havogt May 16, 2024
9ee02e4
fix tuples in columns
havogt May 16, 2024
d104633
fix preserve axis kind in global tmps
havogt May 17, 2024
866c3d6
Merge remote-tracking branch 'upstream/main' into itir_program_embedded2
havogt May 17, 2024
66e2464
fix follow up issue
havogt May 17, 2024
56e0086
Start using tree_map instead of apply_to_primitive_constituents
SF-N Jun 10, 2024
8ab97c4
Add functionality to call also tree_map(lambda x: x + 1, ((1, 2), 3))…
SF-N Jul 3, 2024
7fbd083
Merge main
SF-N Dec 31, 2024
827a4d3
Run pre-commit
SF-N Dec 31, 2024
0a5ac37
Minor
SF-N Dec 31, 2024
70283e6
Replace more apply_to_primitive_constituents by tree_map
SF-N Dec 31, 2024
14619d2
Minor fix
SF-N Dec 31, 2024
fac65de
Revert replacing when tuple_constructor is present
SF-N Dec 31, 2024
de34ef6
Try to use result_collection_constructor
SF-N Jan 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
pretty printing/parsing
  • Loading branch information
havogt committed Apr 12, 2024
commit 6effe10be05af055936cbe6c92b1e26dd56ee381
32 changes: 32 additions & 0 deletions src/gt4py/next/iterator/pretty_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
start: fencil_definition
| function_definition
| stencil_closure
| set_at
| program
| prec0

SYM: CNAME
Expand Down Expand Up @@ -64,6 +66,7 @@
| "·" prec7 -> deref
| "¬" prec7 -> bool_not
| "↑" prec7 -> lift
| "⇑" prec7 -> apply_stencil

?prec8: prec9
| prec8 "[" prec0 "]" -> tuple_get
Expand All @@ -81,7 +84,9 @@
named_range: AXIS_NAME ":" "[" prec0 "," prec0 ")"
function_definition: ID_NAME "=" "λ(" ( SYM "," )* SYM? ")" "→" prec0 ";"
stencil_closure: prec0 "←" "(" prec0 ")" "(" ( SYM_REF ", " )* SYM_REF ")" "@" prec0 ";"
set_at: prec0 "@" prec0 "←" prec1 ";"
fencil_definition: ID_NAME "(" ( SYM "," )* SYM ")" "{" ( function_definition )* ( stencil_closure )+ "}"
program: ID_NAME "(" ( SYM "," )* SYM ")" "{" ( function_definition )* ( set_at )+ "}"

%import common (CNAME, SIGNED_FLOAT, SIGNED_INT, WS)
%ignore WS
Expand Down Expand Up @@ -167,6 +172,9 @@ def deref(self, arg: ir.Expr) -> ir.FunCall:
def lift(self, arg: ir.Expr) -> ir.FunCall:
return ir.FunCall(fun=ir.SymRef(id="lift"), args=[arg])

def apply_stencil(self, arg: ir.Expr) -> ir.FunCall:
return ir.FunCall(fun=ir.SymRef(id="apply_stencil"), args=[arg])

def astype(self, arg: ir.Expr) -> ir.FunCall:
return ir.FunCall(fun=ir.SymRef(id="cast_"), args=[arg])

Expand Down Expand Up @@ -202,6 +210,10 @@ def stencil_closure(self, *args: ir.Expr) -> ir.StencilClosure:
output, stencil, *inputs, domain = args
return ir.StencilClosure(domain=domain, stencil=stencil, output=output, inputs=inputs)

def set_at(self, *args: ir.Expr) -> ir.SetAt:
target, domain, expr = args
return ir.SetAt(expr=expr, domain=domain, target=target)

def fencil_definition(self, fid: str, *args: ir.Node) -> ir.FencilDefinition:
params = []
function_definitions = []
Expand All @@ -218,6 +230,26 @@ def fencil_definition(self, fid: str, *args: ir.Node) -> ir.FencilDefinition:
id=fid, function_definitions=function_definitions, params=params, closures=closures
)

def program(self, fid: str, *args: ir.Node) -> ir.Program:
params = []
function_definitions = []
body = []
for arg in args:
if isinstance(arg, ir.Sym):
params.append(arg)
elif isinstance(arg, ir.FunctionDefinition):
function_definitions.append(arg)
else:
assert isinstance(arg, ir.SetAt)
body.append(arg)
return ir.Program(
id=fid,
function_definitions=function_definitions,
params=params,
body=body,
declarations=[], # TODO
)

def start(self, arg: ir.Node) -> ir.Node:
return arg

Expand Down
36 changes: 35 additions & 1 deletion src/gt4py/next/iterator/pretty_printer.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
}

# replacements for builtin unary operations
UNARY_OPS: Final = {"deref": "·", "lift": "↑", "not_": "¬"}
UNARY_OPS: Final = {"deref": "·", "lift": "↑", "not_": "¬", "apply_stencil": "⇑"}

# operator precedence
PRECEDENCE: Final = {
Expand All @@ -63,6 +63,7 @@
"deref": 7,
"not_": 7,
"lift": 7,
"apply_stencil": 7,
"tuple_get": 8,
"__call__": 8,
}
Expand Down Expand Up @@ -272,6 +273,21 @@ def visit_StencilClosure(self, node: ir.StencilClosure, *, prec: int) -> list[st
)
return self._optimum(h, v)

def visit_SetAt(self, node: ir.SetAt, *, prec: int) -> list[str]:
expr = self.visit(node.expr, prec=0)
domain = self.visit(node.domain, prec=0)
target = self.visit(node.target, prec=0)

head = self._hmerge(target, [" @ "], domain)
foot = self._hmerge([" ← "], expr, [";"])

h = self._hmerge(head, foot)
v = self._vmerge(
head,
self._indent(self._indent(foot)),
)
return self._optimum(h, v)

def visit_FencilDefinition(self, node: ir.FencilDefinition, *, prec: int) -> list[str]:
assert prec == 0
function_definitions = self.visit(node.function_definitions, prec=0)
Expand All @@ -291,6 +307,24 @@ def visit_FencilDefinition(self, node: ir.FencilDefinition, *, prec: int) -> lis
params, self._indent(function_definitions), self._indent(closures), ["}"]
)

def visit_Program(self, node: ir.Program, *, prec: int) -> list[str]:
assert prec == 0
function_definitions = self.visit(node.function_definitions, prec=0)
body = self.visit(node.body, prec=0)
# TODO declarations
params = self.visit(node.params, prec=0)

hparams = self._hmerge([node.id + "("], *self._hinterleave(params, ", "), [") {"])
vparams = self._vmerge(
[node.id + "("], *self._hinterleave(params, ",", indent=True), [") {"]
)
params = self._optimum(hparams, vparams)

function_definitions = self._vmerge(*function_definitions)
body = self._vmerge(*body)

return self._vmerge(params, self._indent(function_definitions), self._indent(body), ["}"])

@classmethod
def apply(cls, node: ir.Node, indent: int, width: int) -> str:
return "\n".join(cls(indent=indent, width=width).visit(node, prec=0))
Expand Down
39 changes: 39 additions & 0 deletions tests/next_tests/unit_tests/iterator_tests/test_pretty_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,13 @@ def test_lift():
assert actual == expected


def test_apply_stencil():
testee = "⇑x"
expected = ir.FunCall(fun=ir.SymRef(id="apply_stencil"), args=[ir.SymRef(id="x")])
actual = pparse(testee)
assert actual == expected


def test_bool_arithmetic():
testee = "¬(¬a ∨ b ∧ (c ∨ d))"
expected = ir.FunCall(
Expand Down Expand Up @@ -195,6 +202,17 @@ def test_stencil_closure():
assert actual == expected


def test_set_at():
testee = "y @ cartesian_domain() ← x;"
expected = ir.SetAt(
expr=ir.SymRef(id="x"),
domain=ir.FunCall(fun=ir.SymRef(id="cartesian_domain"), args=[]),
target=ir.SymRef(id="y"),
)
actual = pparse(testee)
assert actual == expected


def test_fencil_definition():
testee = "f(d, x, y) {\n g = λ(x) → x;\n y ← (deref)(x) @ cartesian_domain();\n}"
expected = ir.FencilDefinition(
Expand All @@ -214,3 +232,24 @@ def test_fencil_definition():
)
actual = pparse(testee)
assert actual == expected


def test_program():
testee = "f(d, x, y) {\n g = λ(x) → x;\n y @ cartesian_domain() ← x;\n}"
expected = ir.Program(
id="f",
function_definitions=[
ir.FunctionDefinition(id="g", params=[ir.Sym(id="x")], expr=ir.SymRef(id="x"))
],
params=[ir.Sym(id="d"), ir.Sym(id="x"), ir.Sym(id="y")],
body=[
ir.SetAt(
expr=ir.SymRef(id="x"),
domain=ir.FunCall(fun=ir.SymRef(id="cartesian_domain"), args=[]),
target=ir.SymRef(id="y"),
)
],
declarations=[], # TODO
)
actual = pparse(testee)
assert actual == expected
39 changes: 39 additions & 0 deletions tests/next_tests/unit_tests/iterator_tests/test_pretty_printer.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,13 @@ def test_lift():
assert actual == expected


def test_apply_stencil():
testee = ir.FunCall(fun=ir.SymRef(id="apply_stencil"), args=[ir.SymRef(id="x")])
expected = "⇑x" # TODO consider ⇈
actual = pformat(testee)
assert actual == expected


def test_bool_arithmetic():
testee = ir.FunCall(
fun=ir.SymRef(id="not_"),
Expand Down Expand Up @@ -299,6 +306,17 @@ def test_stencil_closure():
assert actual == expected


def test_set_at():
testee = ir.SetAt(
expr=ir.SymRef(id="x"),
domain=ir.FunCall(fun=ir.SymRef(id="cartesian_domain"), args=[]),
target=ir.SymRef(id="y"),
)
expected = "y @ cartesian_domain() ← x;"
actual = pformat(testee)
assert actual == expected


def test_fencil_definition():
testee = ir.FencilDefinition(
id="f",
Expand All @@ -318,3 +336,24 @@ def test_fencil_definition():
actual = pformat(testee)
expected = "f(d, x, y) {\n g = λ(x) → x;\n y ← (deref)(x) @ cartesian_domain();\n}"
assert actual == expected


def test_program():
testee = ir.Program(
id="f",
function_definitions=[
ir.FunctionDefinition(id="g", params=[ir.Sym(id="x")], expr=ir.SymRef(id="x"))
],
params=[ir.Sym(id="d"), ir.Sym(id="x"), ir.Sym(id="y")],
declarations=[], # TODO
body=[
ir.SetAt(
expr=ir.SymRef(id="x"),
domain=ir.FunCall(fun=ir.SymRef(id="cartesian_domain"), args=[]),
target=ir.SymRef(id="y"),
)
],
)
actual = pformat(testee)
expected = "f(d, x, y) {\n g = λ(x) → x;\n y @ cartesian_domain() ← x;\n}"
assert actual == expected
Loading