Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PandasReporter using daily sensor as input misses value on spring DST transition #1121

Closed
Flix6x opened this issue Jul 8, 2024 · 0 comments · Fixed by #1122
Closed

PandasReporter using daily sensor as input misses value on spring DST transition #1121

Flix6x opened this issue Jul 8, 2024 · 0 comments · Fixed by #1122
Assignees
Labels
bug Something isn't working Data Reporting

Comments

@Flix6x
Copy link
Contributor

Flix6x commented Jul 8, 2024

Two underlying issues:

  1. search_data_sources and search_beliefs use slightly different filters for event timing
  2. Passing a one-day resolution to search_beliefs leads to clipping off the 23-hour day of the spring DST transition.

Issue 1

  • search_beliefs uses event_ends_after and event_starts_before internally, resulting in events that partly fall in the searched time window to be included. After resampling, the results are clipped to the desired time interval.
  • search_data_sources uses event_starts_after and event_ends_before, resulting in event that partly fall in the searched time window to be excluded. On the day of the spring DST transition, the 23-hour event is being interpreted as a 24-hour event that extends 1 one hour past midnight.

Related to #484.

Issue 2

Given some sensor (here, with ID 40755) with a "P1D" event resolution in the "Europe/Amsterdam" timezone, and an event registered on that sensor with event start at "2023-03-26 00:00:00+01:00" (day of spring DST):

from flexmeasures import Sensor
import pandas as pd
sensor = Sensor.query.get(40755)

# one row, as expected
sensor.search_beliefs(event_starts_after="2023-03-26 00:00:00+01:00", event_ends_before="2023-03-27 00:00:00+02:00")

# empty frame, unexpected
sensor.search_beliefs(event_starts_after="2023-03-26 00:00:00+01:00", event_ends_before="2023-03-27 00:00:00+02:00", resolution=pd.Timedelta("P1D"))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Data Reporting
Projects
None yet
1 participant