Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dagster event logging not configurable through dagster api execute_run #4992

Open
cdchan opened this issue Oct 1, 2021 · 0 comments
Open

Comments

@cdchan
Copy link
Contributor

cdchan commented Oct 1, 2021

Summary

As discussed with @OwenKephart:

The dagster api execute_run command captures all Dagster events generated for the run, and then prints them out after the run is complete. This logging to stdout cannot be configured via the run config or through any logging configuration.

This can be a problem when:

  1. stdout is used as logging. For example, we forward stdout to Datadog as logs.
  2. dagster api execute_run command is how various RunLaunchers (like the DockerRunLauncher) launch runs
  3. The combination of the two generates a lot of log spam that cannot be controlled or configured. Log spam makes it harder to pick out important or relevant events.

Reproduction

Any run using dagster api execute_run will generate this stdout output

Dagit UI/UX Issue Screenshots

Additional Info about Your Environment


Message from the maintainers:

Impacted by this bug? Give it a 👍. We factor engagement into prioritization.

@cdchan cdchan added the type: bug Something isn't working label Oct 1, 2021
@yuhan yuhan added area: logging Related to Logging type: feature-request and removed type: bug Something isn't working labels Oct 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants