-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add node type codes to more events + more hook log data #4378
Conversation
We don't need to prefix the 'started_at' and 'finished_at' keys with 'dbt_internal_' anymore since it's now not in the config dict. Could we change that back? |
core/dbt/events/types.py
Outdated
@@ -1928,8 +1928,8 @@ class PrintHookStartLine(InfoLevel, Cli, File, NodeInfo): | |||
index: int | |||
total: int | |||
truncate: bool | |||
report_node_data: Any # TODO use ParsedHookNode here | |||
code: str = "Z032" | |||
report_node_data: ParsedHookNode |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I couldn't figure out how to get these mypy
checks passing either!
core/dbt/task/run.py:361: error: Argument "report_node_data" to "PrintHookStartLine" has incompatible type "dbt.contracts.graph.parsed.ParsedHookNode"; expected "dbt.events.stubs.ParsedHookNode"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Take a close look at that error. There's two ParsedHookNode
s: one is a type stub and one is a real class. We need to make sure we're not using the type stub, and delete it if we can.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I am using the stub type. But I"m also referencing ParsedHookNode
on other events that don't trigger the error. Then I'm using the stub type for ParsedModelNode
on 16 events that don't trigger this. Why does it only trigger sometimes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
probably because mypy doesn't run on all the files so some of the call sites don't get caught.
I think we can make a better choice than this. What would you like to do here? |
@nathaniel-may I'm honestly thinking now it's on purpose. It's actually consistent with how it gets handled on other log lines. The second line is the one I'm speaking of specifically.
|
Maybe there's a better way to model this behavior so it's more obviously intentional. I'm ok leaving that for later unless you're particularly motivated now. |
ed98b37
to
2f0fa95
Compare
2f0fa95
to
e2a2753
Compare
791564a
to
abcb909
Compare
ParsedHookNode, | ||
ParsedModelNode, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only conflict I fixed myself with the rebase was removing _ReferenceKey
from here since @nathaniel-may added it above in his PR
* add node type codes to more events + more hook log * minor fixes * renames started/finished keys * made process more clear * fixed errors * Put back report_node_data in fresshness.py Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
* add node type codes to more events + more hook log * minor fixes * renames started/finished keys * made process more clear * fixed errors * Put back report_node_data in fresshness.py Co-authored-by: Gerda Shank <gerda@dbtlabs.com> Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com> Co-authored-by: Gerda Shank <gerda@dbtlabs.com>
* add node type codes to more events + more hook log * minor fixes * renames started/finished keys * made process more clear * fixed errors * Put back report_node_data in fresshness.py Co-authored-by: Gerda Shank <gerda@dbtlabs.com> automatic commit by git-black, original commits: b3039fd
on-run-start/end hooks were not getting
node_status
,node_started_at
,node_finished_at
populated innode_status
of logs. Fixed.Made it a little more clear when we use the
_message
method inAdapterResponse
for logging purposes.Fixed up a few logging codes to point to nodes.