You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we aren't consistently windowing the production of metrics to StatsD from Apache Beam. This means that during batch ingestion, an extremely high volume of metrics are sent to StatsD. This can cause StatsD to become unresponsive or crash.
To make matters worse, these metrics are less useful for batch ingestion than they are for streaming ingestion, so its unexpected that batch users would need these metrics, especially at a low granularity.
The proposed solution is to window the metrics and only push them to StatsD at fixed windows.
The text was updated successfully, but these errors were encountered:
Currently we aren't consistently windowing the production of metrics to StatsD from Apache Beam. This means that during batch ingestion, an extremely high volume of metrics are sent to StatsD. This can cause StatsD to become unresponsive or crash.
To make matters worse, these metrics are less useful for batch ingestion than they are for streaming ingestion, so its unexpected that batch users would need these metrics, especially at a low granularity.
The proposed solution is to window the metrics and only push them to StatsD at fixed windows.
The text was updated successfully, but these errors were encountered: