Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Realtime to Offline segment conversion task fails to progress with data gaps #6988

Closed
laxmanchekka opened this issue May 26, 2021 · 6 comments · Fixed by #7814
Closed

Realtime to Offline segment conversion task fails to progress with data gaps #6988

laxmanchekka opened this issue May 26, 2021 · 6 comments · Fixed by #7814

Comments

@laxmanchekka
Copy link
Contributor

Realtime to Offline segment conversion task fails to progress when there are larger data gaps in REALTIME table.
Noticed when the data gap is larger than the bucketTimePeriod.
@laxmanchekka
Copy link
Contributor Author

@Jackie-Jiang @npawar

@kriti-sc
Copy link
Contributor

kriti-sc commented Aug 5, 2021

Picking this up

@Eywek
Copy link

Eywek commented Sep 28, 2021

Hello,
Any news on this?

@deepakcse2k5
Copy link

Hi kriti-sc,

Is there any update on this ?

@kriti-sc
Copy link
Contributor

kriti-sc commented Oct 8, 2021

No, still working on this. @Eywek @deepakcse2k5

@deepakcse2k5
Copy link

To resolve this data gap , there is one workaround to update watermark manually from where the data is available in realtime segment.

@kriti-sc kriti-sc mentioned this issue Oct 20, 2021
Closed
kriti-sc added a commit to kriti-sc/incubator-pinot that referenced this issue Oct 22, 2021
@kriti-sc
Merge branch 'master' of git://github.com/apache/incubator-pinot into a…
c49a547 
…pache#6988-update-watermark-if-no-data-in-window
@Jackie-Jiang Jackie-Jiang mentioned this issue Nov 22, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@Eywek @deepakcse2k5 @kriti-sc @laxmanchekka