Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor[next] error handling #1275

Merged
merged 56 commits into from
Jul 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
be9849d
stuff
petiaccja Jun 5, 2023
743042e
refactor past and foast parsers
petiaccja Jun 5, 2023
c38116f
foast type deduction uses new exceptions
petiaccja Jun 6, 2023
d59808e
past type deduction uses new exceptions
petiaccja Jun 6, 2023
601737a
remove unused old exception classes
petiaccja Jun 6, 2023
05b5ef2
rename fields in source definition to better reflect meaning
petiaccja Jun 6, 2023
99785f3
experiment exception printing
petiaccja Jun 6, 2023
4c60811
exception hook and cleanups
petiaccja Jun 13, 2023
1451e79
fix infinite recursion
petiaccja Jun 14, 2023
ba17c63
remove old exceptions
petiaccja Jun 14, 2023
384b62f
improve printing & exception hierarchy
petiaccja Jun 15, 2023
3ea99d5
tune
petiaccja Jun 21, 2023
8aef5f0
fix & clean SourceLocation
petiaccja Jun 21, 2023
f59732c
Merge remote-tracking branch 'GridTools/main' into exceptions
petiaccja Jun 21, 2023
f498e00
revert source location str
petiaccja Jun 21, 2023
e1c1f04
fix qa
petiaccja Jun 21, 2023
360a6f0
fix doctests
petiaccja Jun 21, 2023
d081374
print extra info for uncaught exceptions for gt4py developers
petiaccja Jun 30, 2023
79dca31
use only single source location and not stack
petiaccja Jun 30, 2023
1bbeec7
add more tests for error handling, refactor
petiaccja Jul 5, 2023
9f95b76
keep parameters of exception for further access
petiaccja Jul 5, 2023
5d01311
Update src/gt4py/next/errors/excepthook.py
petiaccja Jul 12, 2023
03c01c7
Update src/gt4py/next/errors/excepthook.py
petiaccja Jul 12, 2023
61696cc
Update src/gt4py/next/errors/exceptions.py
petiaccja Jul 12, 2023
5c09705
rename functions
petiaccja Jul 12, 2023
3b5f1f4
avoid stringifying annotations
petiaccja Jul 12, 2023
a17ec99
Update src/gt4py/next/errors/exceptions.py
petiaccja Jul 12, 2023
e010bb8
type annotations
petiaccja Jul 12, 2023
73b8a91
type annotations
petiaccja Jul 12, 2023
4951df7
rename vars to be consistent with class name
petiaccja Jul 12, 2023
519f242
use linecache instead of loading file from disc
petiaccja Jul 12, 2023
f6bfdc4
remove blanket exception handling
petiaccja Jul 12, 2023
c7d7eb7
improve docstrings
petiaccja Jul 12, 2023
89fc3a0
remove blanket imports
petiaccja Jul 12, 2023
935d37b
tests for the exception hook
petiaccja Jul 13, 2023
d8d603f
use typeerror
petiaccja Jul 13, 2023
fb24895
return None for invalid env var
petiaccja Jul 13, 2023
084d218
change names of tests to be more descriptive
petiaccja Jul 13, 2023
c53fdc1
remove developer mode autodetection
petiaccja Jul 13, 2023
8371cab
delete some unused exception classes
petiaccja Jul 13, 2023
8e8e008
rename developer mode to verbose exceptions
petiaccja Jul 13, 2023
5e08a50
Update src/gt4py/next/ffront/func_to_foast.py
petiaccja Jul 13, 2023
1bcbdf7
delete test files
petiaccja Jul 13, 2023
cc162b3
use fixtures
petiaccja Jul 13, 2023
e04c7fd
rename exception classes
petiaccja Jul 13, 2023
8fe6d06
remove test removed in another PR
petiaccja Jul 13, 2023
7a14225
add doscstrings to modules
petiaccja Jul 18, 2023
a7e7566
fix doc string formatting
petiaccja Jul 19, 2023
2cac27e
import module for type checking
petiaccja Jul 19, 2023
98b0bad
remove direct imports
petiaccja Jul 19, 2023
de7f9e9
split string into multiple lines proper
petiaccja Jul 19, 2023
9327893
fix concatenated strings
petiaccja Jul 19, 2023
1bf640c
fix escape sequence, reformat
petiaccja Jul 20, 2023
eafdde2
fix verbose exceptions env var, tests for it
petiaccja Jul 20, 2023
7638763
improve docstrings
petiaccja Jul 20, 2023
af78ab9
Merge remote-tracking branch 'GridTools/main' into exceptions
petiaccja Jul 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/gt4py/cartesian/frontend/node_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,4 +147,4 @@ def recurse(node: Node) -> Generator[Node, None, None]:
def location_to_source_location(loc: Optional[Location]) -> Optional[eve.SourceLocation]:
if loc is None or loc.line <= 0 or loc.column <= 0:
return None
return eve.SourceLocation(line=loc.line, column=loc.column, source=loc.scope)
return eve.SourceLocation(line=loc.line, column=loc.column, filename=loc.scope)
2 changes: 1 addition & 1 deletion src/gt4py/cartesian/gtc/dace/expansion/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ def get_dace_debuginfo(node: common.LocNode):
node.loc.column,
node.loc.line,
node.loc.column,
node.loc.source,
node.loc.filename,
)
else:
return dace.dtypes.DebugInfo(0)
Expand Down
47 changes: 15 additions & 32 deletions src/gt4py/eve/concepts.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@

from __future__ import annotations

import ast
import copy
import re

Expand Down Expand Up @@ -58,58 +57,42 @@ class SymbolRef(ConstrainedStr, regex=_SYMBOL_NAME_RE):

@datamodels.datamodel(slots=True, frozen=True)
class SourceLocation:
"""Source code location (line, column, source)."""
"""File-line-column information for source code."""

filename: Optional[str]
line: int = datamodels.field(validator=_validators.ge(1))
column: int = datamodels.field(validator=_validators.ge(1))
source: str
end_line: Optional[int] = datamodels.field(validator=_validators.optional(_validators.ge(1)))
end_column: Optional[int] = datamodels.field(validator=_validators.optional(_validators.ge(1)))

@classmethod
def from_AST(cls, ast_node: ast.AST, source: Optional[str] = None) -> SourceLocation:
if (
not isinstance(ast_node, ast.AST)
or getattr(ast_node, "lineno", None) is None
or getattr(ast_node, "col_offset", None) is None
):
raise ValueError(
f"Passed AST node '{ast_node}' does not contain a valid source location."
)
if source is None:
source = f"<ast.{type(ast_node).__name__} at 0x{id(ast_node):x}>"
return cls(
ast_node.lineno,
ast_node.col_offset + 1,
source,
end_line=ast_node.end_lineno,
end_column=ast_node.end_col_offset + 1 if ast_node.end_col_offset is not None else None,
)
egparedes marked this conversation as resolved.
Show resolved Hide resolved

def __init__(
self,
filename: Optional[str],
line: int,
column: int,
source: str,
*,
end_line: Optional[int] = None,
end_column: Optional[int] = None,
) -> None:
assert end_column is None or end_line is not None
self.__auto_init__( # type: ignore[attr-defined] # __auto_init__ added dynamically
line=line, column=column, source=source, end_line=end_line, end_column=end_column
filename=filename, line=line, column=column, end_line=end_line, end_column=end_column
)

def __str__(self) -> str:
src = self.source or ""
filename_str = self.filename or "-"

end_line_str = self.end_line if self.end_line is not None else "-"
end_column_str = self.end_column if self.end_column is not None else "-"

end_part = ""
if self.end_line is not None:
end_part += f" to {self.end_line}"
end_str: Optional[str] = None
if self.end_column is not None:
end_part += f":{self.end_column}"
end_str = f"{end_line_str}:{end_column_str}"
elif self.end_line is not None:
end_str = f"{end_line_str}"

return f"<{src}:{self.line}:{self.column}{end_part}>"
if end_str is not None:
return f"<{filename_str}:{self.line}:{self.column} to {end_str}>"
return f"<{filename_str}:{self.line}:{self.column}>"


@datamodels.datamodel(slots=True, frozen=True)
Expand Down
38 changes: 0 additions & 38 deletions src/gt4py/next/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -108,41 +108,3 @@ class NeighborTable(Connectivity, Protocol):
class GridType(StrEnum):
CARTESIAN = "cartesian"
UNSTRUCTURED = "unstructured"


class GTError:
"""Base class for GridTools exceptions.

Notes:
This base class has to be always inherited together with a standard
exception, and thus it should not be used as direct superclass
for custom exceptions. Inherit directly from :class:`GTTypeError`,
:class:`GTTypeError`, ...

"""

...


class GTRuntimeError(GTError, RuntimeError):
"""Base class for GridTools run-time errors."""

...


class GTSyntaxError(GTError, SyntaxError):
"""Base class for GridTools syntax errors."""

...


class GTTypeError(GTError, TypeError):
"""Base class for GridTools type errors."""

...


class GTValueError(GTError, ValueError):
"""Base class for GridTools value errors."""

...
39 changes: 39 additions & 0 deletions src/gt4py/next/errors/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# GT4Py - GridTools Framework
#
# Copyright (c) 2014-2023, ETH Zurich
# All rights reserved.
#
# This file is part of the GT4Py project and the GridTools framework.
# GT4Py is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or any later
# version. See the LICENSE.txt file at the top-level directory of this
# distribution for a copy of the license or check <https://www.gnu.org/licenses/>.
#
# SPDX-License-Identifier: GPL-3.0-or-later

"""Contains the exception classes and other utilities for error handling."""

from . import ( # noqa: module needs to be loaded for pretty printing of uncaught exceptions.
egparedes marked this conversation as resolved.
Show resolved Hide resolved
excepthook,
)
from .excepthook import set_verbose_exceptions
from .exceptions import (
DSLError,
InvalidParameterAnnotationError,
MissingAttributeError,
MissingParameterAnnotationError,
UndefinedSymbolError,
UnsupportedPythonFeatureError,
)


__all__ = [
"DSLError",
"InvalidParameterAnnotationError",
"MissingAttributeError",
"MissingParameterAnnotationError",
"UndefinedSymbolError",
"UnsupportedPythonFeatureError",
"set_verbose_exceptions",
]
87 changes: 87 additions & 0 deletions src/gt4py/next/errors/excepthook.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# GT4Py - GridTools Framework
#
# Copyright (c) 2014-2023, ETH Zurich
# All rights reserved.
#
# This file is part of the GT4Py project and the GridTools framework.
# GT4Py is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or any later
# version. See the LICENSE.txt file at the top-level directory of this
# distribution for a copy of the license or check <https://www.gnu.org/licenses/>.
#
# SPDX-License-Identifier: GPL-3.0-or-later

"""
Loading this module registers an excepthook that formats :class:`DSLError`.

The excepthook is necessary because the default hook prints :class:`DSLError`s
in an inconvenient way. The previously set excepthook is used to print all
other errors.
"""

import os
import sys
import warnings
from typing import Callable

from . import exceptions, formatting


def _get_verbose_exceptions_envvar() -> bool:
"""Detect if the user enabled verbose exceptions in the environment variables."""
env_var_name = "GT4PY_VERBOSE_EXCEPTIONS"
if env_var_name in os.environ:
false_values = ["0", "false", "off"]
true_values = ["1", "true", "on"]
value = os.environ[env_var_name].lower()
if value in false_values:
return False
petiaccja marked this conversation as resolved.
Show resolved Hide resolved
elif value in true_values:
return True
else:
values = ", ".join([*false_values, *true_values])
msg = f"the 'GT4PY_VERBOSE_EXCEPTIONS' environment variable must be one of {values} (case insensitive)"
warnings.warn(msg)
return False


_verbose_exceptions: bool = _get_verbose_exceptions_envvar()


def set_verbose_exceptions(enabled: bool = False) -> None:
"""Programmatically set whether to use verbose printing for uncaught errors."""
global _verbose_exceptions
_verbose_exceptions = enabled


def _format_uncaught_error(err: exceptions.DSLError, verbose_exceptions: bool) -> list[str]:
if verbose_exceptions:
return formatting.format_compilation_error(
type(err),
err.message,
err.location,
err.__traceback__,
err.__cause__,
)
else:
return formatting.format_compilation_error(type(err), err.message, err.location)


def compilation_error_hook(fallback: Callable, type_: type, value: BaseException, tb) -> None:
"""
Format `CompilationError`s in a neat way.
petiaccja marked this conversation as resolved.
Show resolved Hide resolved

All other Python exceptions are formatted by the `fallback` hook. When
verbose exceptions are enabled, the stack trace and cause of the error is
also printed.
"""
if isinstance(value, exceptions.DSLError):
exc_strs = _format_uncaught_error(value, _verbose_exceptions)
print("".join(exc_strs), file=sys.stderr)
else:
fallback(type_, value, tb)


_fallback = sys.excepthook
sys.excepthook = lambda ty, val, tb: compilation_error_hook(_fallback, ty, val, tb)
104 changes: 104 additions & 0 deletions src/gt4py/next/errors/exceptions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# GT4Py - GridTools Framework
#
# Copyright (c) 2014-2023, ETH Zurich
# All rights reserved.
#
# This file is part of the GT4Py project and the GridTools framework.
# GT4Py is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or any later
# version. See the LICENSE.txt file at the top-level directory of this
# distribution for a copy of the license or check <https://www.gnu.org/licenses/>.
#
# SPDX-License-Identifier: GPL-3.0-or-later

"""
The list of exception classes used in the library.

Exception classes that represent errors within an IR go here as a subclass of
:class:`DSLError`. Exception classes that represent other errors, like
the builtin ValueError, go here as well, although you should use Python's
builtin error classes if you can. Exception classes that are specific to a
certain submodule and have no use for the entire application may be better off
in that submodule as opposed to being in this file.
"""

from __future__ import annotations

import textwrap
petiaccja marked this conversation as resolved.
Show resolved Hide resolved
from typing import Any, Optional

from gt4py.eve import SourceLocation

from . import formatting


class DSLError(Exception):
location: Optional[SourceLocation]

def __init__(self, location: Optional[SourceLocation], message: str) -> None:
self.location = location
super().__init__(message)

@property
def message(self) -> str:
return self.args[0]

def with_location(self, location: Optional[SourceLocation]) -> DSLError:
self.location = location
return self

def __str__(self) -> str:
if self.location:
loc_str = formatting.format_location(self.location, show_caret=True)
return f"{self.message}\n{textwrap.indent(loc_str, ' ')}"
return self.message


class UnsupportedPythonFeatureError(DSLError):
feature: str

def __init__(self, location: Optional[SourceLocation], feature: str) -> None:
super().__init__(location, f"unsupported Python syntax: '{feature}'")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
super().__init__(location, f"unsupported Python syntax: '{feature}'")
super().__init__(location, f"Unsupported Python syntax: `{feature}`")

We are currently using a different format for errors (in type_deduction, type_info and many other places). First word is capatilized, backticks instead of quotes and sentences are ended with a dot. This is inconsistent with the error printing in Python, but this is a separate issue for another discussion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll leave things as they are now as the actual error messages are out of the scope of this PR.

self.feature = feature


class UndefinedSymbolError(DSLError):
sym_name: str

def __init__(self, location: Optional[SourceLocation], name: str) -> None:
super().__init__(location, f"name '{name}' is not defined")
self.sym_name = name


class MissingAttributeError(DSLError):
attr_name: str

def __init__(self, location: Optional[SourceLocation], attr_name: str) -> None:
super().__init__(location, f"object does not have attribute '{attr_name}'")
self.attr_name = attr_name


class TypeError_(DSLError):
def __init__(self, location: Optional[SourceLocation], message: str) -> None:
super().__init__(location, message)


class MissingParameterAnnotationError(TypeError_):
param_name: str

def __init__(self, location: Optional[SourceLocation], param_name: str) -> None:
super().__init__(location, f"parameter '{param_name}' is missing type annotations")
self.param_name = param_name


class InvalidParameterAnnotationError(TypeError_):
param_name: str
annotated_type: Any

def __init__(self, location: Optional[SourceLocation], param_name: str, type_: Any) -> None:
super().__init__(
location, f"parameter '{param_name}' has invalid type annotation '{type_}'"
)
self.param_name = param_name
self.annotated_type = type_
Loading