Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Date range generation for persons_urls #11468

Closed
wants to merge 32 commits into from
Closed

fix: Date range generation for persons_urls #11468

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
64b0b18
Fix date range generation for persons
benjackwhite Aug 24, 2022
787ac4e
Linting fixes
benjackwhite Aug 24, 2022
07b7954
Removed date modifications to instead respect the given values
benjackwhite Aug 25, 2022
d2c0ac8
Save change
benjackwhite Aug 25, 2022
651c7d2
Fix for test scenarios
benjackwhite Aug 25, 2022
cb004f4
Update snapshots
benjackwhite Aug 25, 2022
ce25611
Fix
benjackwhite Aug 25, 2022
c487c93
Update snapshots
benjackwhite Aug 25, 2022
7d7610c
Update snapshots
posthog-bot Aug 25, 2022
bf58b87
Fix linting and tests
benjackwhite Aug 30, 2022
1dac07a
Update snapshots
benjackwhite Aug 30, 2022
577d447
Fix sorting
benjackwhite Aug 30, 2022
f9a14c8
Merge branch 'master' into fix/persons-timezone-offset
benjackwhite Sep 5, 2022
87b169e
Fix
benjackwhite Sep 5, 2022
40d094d
Fixed tests
benjackwhite Sep 5, 2022
d8bc505
Fix tests
benjackwhite Sep 5, 2022
7b02857
Fix snapshots
benjackwhite Sep 5, 2022
c97cda1
Fix tests
benjackwhite Sep 5, 2022
a6d6575
Fix tests
benjackwhite Sep 5, 2022
a03e8a3
Merge branch 'master' into fix/persons-timezone-offset
benjackwhite Sep 5, 2022
3f51b1c
Update snapshots
benjackwhite Sep 5, 2022
15938a3
Update snapshots
posthog-bot Sep 5, 2022
4c87300
Update snapshots
posthog-bot Sep 5, 2022
c6f584e
Merge branch 'master' into fix/persons-timezone-offset
benjackwhite Sep 6, 2022
5f85b19
Fix
benjackwhite Sep 6, 2022
7deab3c
Fix tests
benjackwhite Sep 6, 2022
addcd36
Fixed tests
benjackwhite Sep 6, 2022
e0536f3
Fix times
benjackwhite Sep 6, 2022
16dd995
Fix
benjackwhite Sep 6, 2022
3dc726f
Merge branch 'master' into fix/persons-timezone-offset
benjackwhite Sep 6, 2022
bceea66
Removed old util
benjackwhite Sep 6, 2022
89dcb23
Merge branch 'master' into fix/persons-timezone-offset
benjackwhite Sep 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions ee/clickhouse/queries/test/__snapshots__/test_breakdown_props.ambr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toDateTime('2020-01-01 00:00:00')
AND timestamp <= toDateTime('2020-01-12 23:59:59')
AND timestamp <= toDateTime('2020-01-12 00:00:00')
AND ((isNull(replaceRegexpAll(JSONExtractRaw(group_properties_0, 'out'), '^"|"$', ''))
OR NOT JSONHas(group_properties_0, 'out')))
GROUP BY value
Expand All @@ -43,7 +43,7 @@
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toDateTime('2020-01-01 00:00:00')
AND timestamp <= toDateTime('2020-01-12 23:59:59')
benjackwhite marked this conversation as resolved.
Show resolved Hide resolved
AND timestamp <= toDateTime('2020-01-12 00:00:00')
AND ((isNull(replaceRegexpAll(JSONExtractRaw(group_properties_0, 'out'), '^"|"$', ''))
OR NOT JSONHas(group_properties_0, 'out')))
GROUP BY value
Expand Down Expand Up @@ -166,12 +166,12 @@
WHERE $session_id != ''
AND team_id = 2
AND timestamp >= toDateTime('2020-01-02 00:00:00') - INTERVAL 24 HOUR
AND timestamp <= toDateTime('2020-01-12 23:59:59') + INTERVAL 24 HOUR
AND timestamp <= toDateTime('2020-01-12 00:00:00') + INTERVAL 24 HOUR
GROUP BY $session_id) AS sessions ON sessions.$session_id = e.$session_id
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toDateTime('2020-01-02 00:00:00')
AND timestamp <= toDateTime('2020-01-12 23:59:59')
AND timestamp <= toDateTime('2020-01-12 00:00:00')
GROUP BY value
ORDER BY count DESC, value DESC
LIMIT 25
Expand Down Expand Up @@ -207,12 +207,12 @@
WHERE $session_id != ''
AND team_id = 2
AND timestamp >= toDateTime('2020-01-02 00:00:00') - INTERVAL 24 HOUR
AND timestamp <= toDateTime('2020-01-12 23:59:59') + INTERVAL 24 HOUR
AND timestamp <= toDateTime('2020-01-12 00:00:00') + INTERVAL 24 HOUR
GROUP BY $session_id) AS sessions ON sessions.$session_id = e.$session_id
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toDateTime('2020-01-02 00:00:00')
AND timestamp <= toDateTime('2020-01-12 23:59:59')
AND timestamp <= toDateTime('2020-01-12 00:00:00')
GROUP BY value
ORDER BY count DESC, value DESC
LIMIT 25
Expand Down Expand Up @@ -248,12 +248,12 @@
WHERE $session_id != ''
AND team_id = 2
AND timestamp >= toDateTime('2020-01-02 00:00:00') - INTERVAL 24 HOUR
AND timestamp <= toDateTime('2020-01-12 23:59:59') + INTERVAL 24 HOUR
AND timestamp <= toDateTime('2020-01-12 00:00:00') + INTERVAL 24 HOUR
GROUP BY $session_id) AS sessions ON sessions.$session_id = e.$session_id
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toDateTime('2020-01-02 00:00:00')
AND timestamp <= toDateTime('2020-01-12 23:59:59')
AND timestamp <= toDateTime('2020-01-12 00:00:00')
GROUP BY value
ORDER BY count DESC, value DESC
LIMIT 25
Expand Down
4 changes: 2 additions & 2 deletions ee/clickhouse/queries/test/__snapshots__/test_event_query.ambr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toTimezone(toDateTime(toStartOfDay(toDateTime('2020-01-01 00:00:00')), 'UTC'), 'UTC')
AND timestamp <= toDateTime('2020-01-12 23:59:59')
AND timestamp <= toDateTime('2020-01-12 00:00:00')
AND ((has(['finance'], replaceRegexpAll(JSONExtractRaw(group_properties_0, 'industry'), '^"|"$', '')))
AND (has(['value'], replaceRegexpAll(JSONExtractRaw(group_properties_1, 'another'), '^"|"$', ''))))
'
Expand Down Expand Up @@ -298,7 +298,7 @@
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toTimezone(toDateTime(toStartOfDay(toDateTime('2020-01-01 00:00:00')), 'UTC'), 'UTC')
AND timestamp <= toDateTime('2020-01-12 23:59:59')
AND timestamp <= toDateTime('2020-01-12 00:00:00')
AND ((has(['finance'], replaceRegexpAll(JSONExtractRaw(group_properties_0, 'industry'), '^"|"$', ''))))
'
---
Expand Down
32 changes: 16 additions & 16 deletions ee/clickhouse/queries/test/__snapshots__/test_lifecycle.ambr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,8 +251,8 @@
'
WITH 'day' AS selected_period,
periods AS
(SELECT dateSub(day, number, dateTrunc(selected_period, toDateTime('2020-01-18 23:59:59', 'UTC'))) AS start_of_period
FROM numbers(dateDiff('day', dateTrunc('day', toDateTime('2020-01-11 00:00:00')), dateTrunc('day', toDateTime('2020-01-18 23:59:59') + INTERVAL 1 day))))
(SELECT dateSub(day, number, dateTrunc(selected_period, toDateTime('2020-01-18 00:00:00', 'UTC'))) AS start_of_period
FROM numbers(dateDiff('day', dateTrunc('day', toDateTime('2020-01-11 00:00:00')), dateTrunc('day', toDateTime('2020-01-18 00:00:00') + INTERVAL 1 day))))
SELECT groupArray(start_of_period) as date,
groupArray(counts) as data,
status
Expand Down Expand Up @@ -318,9 +318,9 @@
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toDateTime(dateTrunc('day', toDateTime('2020-01-11 00:00:00'))) - INTERVAL 1 day
AND timestamp < toDateTime(dateTrunc('day', toDateTime('2020-01-18 23:59:59'))) + INTERVAL 1 day )))
AND timestamp < toDateTime(dateTrunc('day', toDateTime('2020-01-18 00:00:00'))) + INTERVAL 1 day )))
WHERE period_status_pairs.2 != '' SETTINGS allow_experimental_window_functions = 1 )
WHERE start_of_period <= dateTrunc('day', toDateTime('2020-01-18 23:59:59', 'UTC'))
WHERE start_of_period <= dateTrunc('day', toDateTime('2020-01-18 00:00:00', 'UTC'))
AND start_of_period >= dateTrunc('day', toDateTime('2020-01-11 00:00:00', 'UTC'))
GROUP BY start_of_period,
status)
Expand All @@ -334,8 +334,8 @@
'
WITH 'day' AS selected_period,
periods AS
(SELECT dateSub(day, number, dateTrunc(selected_period, toDateTime('2020-01-19 23:59:59', 'UTC'))) AS start_of_period
FROM numbers(dateDiff('day', dateTrunc('day', toDateTime('2020-01-12 00:00:00')), dateTrunc('day', toDateTime('2020-01-19 23:59:59') + INTERVAL 1 day))))
(SELECT dateSub(day, number, dateTrunc(selected_period, toDateTime('2020-01-19 00:00:00', 'UTC'))) AS start_of_period
FROM numbers(dateDiff('day', dateTrunc('day', toDateTime('2020-01-12 00:00:00')), dateTrunc('day', toDateTime('2020-01-19 00:00:00') + INTERVAL 1 day))))
SELECT groupArray(start_of_period) as date,
groupArray(counts) as data,
status
Expand Down Expand Up @@ -408,10 +408,10 @@
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toDateTime(dateTrunc('day', toDateTime('2020-01-12 00:00:00'))) - INTERVAL 1 day
AND timestamp < toDateTime(dateTrunc('day', toDateTime('2020-01-19 23:59:59'))) + INTERVAL 1 day
AND timestamp < toDateTime(dateTrunc('day', toDateTime('2020-01-19 00:00:00'))) + INTERVAL 1 day
AND (has(['value'], replaceRegexpAll(JSONExtractRaw(group_properties_0, 'key'), '^"|"$', ''))) )))
WHERE period_status_pairs.2 != '' SETTINGS allow_experimental_window_functions = 1 )
WHERE start_of_period <= dateTrunc('day', toDateTime('2020-01-19 23:59:59', 'UTC'))
WHERE start_of_period <= dateTrunc('day', toDateTime('2020-01-19 00:00:00', 'UTC'))
AND start_of_period >= dateTrunc('day', toDateTime('2020-01-12 00:00:00', 'UTC'))
GROUP BY start_of_period,
status)
Expand All @@ -425,8 +425,8 @@
'
WITH 'day' AS selected_period,
periods AS
(SELECT dateSub(day, number, dateTrunc(selected_period, toDateTime('2020-01-19 23:59:59', 'UTC'))) AS start_of_period
FROM numbers(dateDiff('day', dateTrunc('day', toDateTime('2020-01-12 00:00:00')), dateTrunc('day', toDateTime('2020-01-19 23:59:59') + INTERVAL 1 day))))
(SELECT dateSub(day, number, dateTrunc(selected_period, toDateTime('2020-01-19 00:00:00', 'UTC'))) AS start_of_period
FROM numbers(dateDiff('day', dateTrunc('day', toDateTime('2020-01-12 00:00:00')), dateTrunc('day', toDateTime('2020-01-19 00:00:00') + INTERVAL 1 day))))
SELECT groupArray(start_of_period) as date,
groupArray(counts) as data,
status
Expand Down Expand Up @@ -492,9 +492,9 @@
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toDateTime(dateTrunc('day', toDateTime('2020-01-12 00:00:00'))) - INTERVAL 1 day
AND timestamp < toDateTime(dateTrunc('day', toDateTime('2020-01-19 23:59:59'))) + INTERVAL 1 day )))
AND timestamp < toDateTime(dateTrunc('day', toDateTime('2020-01-19 00:00:00'))) + INTERVAL 1 day )))
WHERE period_status_pairs.2 != '' SETTINGS allow_experimental_window_functions = 1 )
WHERE start_of_period <= dateTrunc('day', toDateTime('2020-01-19 23:59:59', 'UTC'))
WHERE start_of_period <= dateTrunc('day', toDateTime('2020-01-19 00:00:00', 'UTC'))
AND start_of_period >= dateTrunc('day', toDateTime('2020-01-12 00:00:00', 'UTC'))
GROUP BY start_of_period,
status)
Expand All @@ -508,8 +508,8 @@
'
WITH 'day' AS selected_period,
periods AS
(SELECT dateSub(day, number, dateTrunc(selected_period, toDateTime('2020-01-19 23:59:59', 'US/Pacific'))) AS start_of_period
FROM numbers(dateDiff('day', dateTrunc('day', toDateTime('2020-01-12 00:00:00')), dateTrunc('day', toDateTime('2020-01-19 23:59:59') + INTERVAL 1 day))))
(SELECT dateSub(day, number, dateTrunc(selected_period, toDateTime('2020-01-19 00:00:00', 'US/Pacific'))) AS start_of_period
FROM numbers(dateDiff('day', dateTrunc('day', toDateTime('2020-01-12 00:00:00')), dateTrunc('day', toDateTime('2020-01-19 00:00:00') + INTERVAL 1 day))))
SELECT groupArray(start_of_period) as date,
groupArray(counts) as data,
status
Expand Down Expand Up @@ -575,9 +575,9 @@
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toDateTime(dateTrunc('day', toDateTime('2020-01-12 00:00:00'))) - INTERVAL 1 day
AND timestamp < toDateTime(dateTrunc('day', toDateTime('2020-01-19 23:59:59'))) + INTERVAL 1 day )))
AND timestamp < toDateTime(dateTrunc('day', toDateTime('2020-01-19 00:00:00'))) + INTERVAL 1 day )))
WHERE period_status_pairs.2 != '' SETTINGS allow_experimental_window_functions = 1 )
WHERE start_of_period <= dateTrunc('day', toDateTime('2020-01-19 23:59:59', 'US/Pacific'))
WHERE start_of_period <= dateTrunc('day', toDateTime('2020-01-19 00:00:00', 'US/Pacific'))
AND start_of_period >= dateTrunc('day', toDateTime('2020-01-12 00:00:00', 'US/Pacific'))
GROUP BY start_of_period,
status)
Expand Down
14 changes: 7 additions & 7 deletions ee/clickhouse/queries/test/__snapshots__/test_trends.ambr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@
WHERE team_id = 2
AND event = 'sign up'
AND timestamp >= toDateTime('2020-01-02 00:00:00')
AND timestamp <= toDateTime('2020-01-02 23:59:59')
AND timestamp <= toDateTime('2020-01-03 23:59:59')
AND (has(['technology'], replaceRegexpAll(JSONExtractRaw(group_properties_0, 'industry'), '^"|"$', ''))) )
GROUP BY actor_id
LIMIT 200
Expand Down Expand Up @@ -273,7 +273,7 @@
WHERE team_id = 2
AND event = 'sign up'
AND timestamp >= toDateTime('2020-01-01 00:00:00')
AND timestamp <= toDateTime('2020-01-12 23:59:59')
AND timestamp <= toDateTime('2020-01-12 00:00:00')
GROUP BY value
ORDER BY count DESC, value DESC
LIMIT 25
Expand All @@ -297,7 +297,7 @@
ticks.day_start as day_start,
breakdown_value
FROM
(SELECT toStartOfDay(toDateTime('2020-01-12 23:59:59', 'UTC') - number * 86400) as day_start
(SELECT toStartOfDay(toDateTime('2020-01-12 00:00:00', 'UTC') - number * 86400) as day_start
FROM numbers(12)
UNION ALL SELECT toStartOfDay(toDateTime('2020-01-01 00:00:00', 'UTC')) as day_start) as ticks
CROSS JOIN
Expand Down Expand Up @@ -335,7 +335,7 @@
WHERE e.team_id = 2
AND event = 'sign up'
AND timestamp >= toTimezone(toDateTime(toStartOfDay(toDateTime('2020-01-01 00:00:00')), 'UTC'), 'UTC')
AND timestamp <= toDateTime('2020-01-12 23:59:59')
AND timestamp <= toDateTime('2020-01-12 00:00:00')
AND replaceRegexpAll(JSONExtractRaw(group_properties_0, 'industry'), '^"|"$', '') in (['finance'])
GROUP BY day_start,
breakdown_value))
Expand Down Expand Up @@ -456,8 +456,8 @@
day_start
from
(SELECT toUInt16(0) AS total,
toStartOfDay(toDateTime('2020-01-12 23:59:59') - toIntervalDay(number), 'UTC') AS day_start
FROM numbers(dateDiff('day', toStartOfDay(toDateTime('2020-01-01 00:00:00'), 'UTC'), toDateTime('2020-01-12 23:59:59'), 'UTC'))
toStartOfDay(toDateTime('2020-01-12 00:00:00') - toIntervalDay(number), 'UTC') AS day_start
FROM numbers(dateDiff('day', toStartOfDay(toDateTime('2020-01-01 00:00:00'), 'UTC'), toDateTime('2020-01-12 00:00:00'), 'UTC'))
UNION ALL SELECT toUInt16(0) AS total,
toStartOfDay(toDateTime('2020-01-01 00:00:00'), 'UTC')
UNION ALL SELECT count(*) as data,
Expand Down Expand Up @@ -490,7 +490,7 @@
WHERE team_id = 2
AND event = '$pageview'
AND timestamp >= toTimezone(toDateTime(toStartOfDay(toDateTime('2020-01-01 00:00:00')), 'UTC'), 'UTC')
AND timestamp <= toDateTime('2020-01-12 23:59:59')
AND timestamp <= toDateTime('2020-01-12 00:00:00')
AND ((has(['finance'], replaceRegexpAll(JSONExtractRaw(group_properties_0, 'industry'), '^"|"$', '')))) )
GROUP BY date)
group by day_start
Expand Down
Loading