Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the bug of SectorRetryFinalize dead loop after SubmitCommitAggregate while FinalizeEarly is true #6651

Closed
wants to merge 1 commit into from

Conversation

lionsoul2014
Copy link

@lionsoul2014 lionsoul2014 commented Jul 1, 2021

with FinalizeEarly set to true:

Before SubmitCommitAggregate and everything is find and sectors were finalized:

commit-finalize-02

Also the data were all downloaded to the final storage server and all the sectors were on state SubmitCommitAggregate:

commit-aggregate-02

After SubmitCommitAggregate by force publish and the sectors state keep firing the following state changes:

SubmitCommitAggregate(on.SectorCommitAggregateSent)->CommitWait(on.SectorProving)->FinalizeSector

since the sectors were deleted before the SubmitCommitAggregate finalize call and the call to Finalize now will fire SectorFinalizeFailed CUZ of sector not found, it then fire FinalizeFailed and enter the following SectorRetryFinalize dead loop.

FinalizeSector: planOne(
    on(SectorFinalized{}, Proving),
    on(SectorFinalizeFailed{}, FinalizeFailed),
),
FinalizeFailed: planOne(
    on(SectorRetryFinalize{}, FinalizeSector),
),

BUG: it keeps retrying finalize sectors and the sectors state will always be FinalizedFailed.

About this PR:
1, Add handleCommitFinalizeSector to handler the commit finalize and it keeps retry until the finalize finished.
2, Ignore the finalize failed during handleFinalizeSector if cfg.FinalizeEarly is true since the commit finalize will handle this situation, the second call of finalize will return update the sector state to proving (For FinalizeEarly is set to true and ignore finalize retry).
3, everything as usual if cfg.FinalizeEarly is set to false.

now second calling of finalize failed:
finalize-failed

And all sectors state were updated to Proving:

state-proving

…gregate while FinalizeEarly were set to true
@lionsoul2014 lionsoul2014 changed the title Fix the bug of SectorRetryFinalize dead loop for after SubmitCommitAggregate while FinalizeEarly set to true Fix the bug of SectorRetryFinalize dead loop after SubmitCommitAggregate while FinalizeEarly is true Jul 1, 2021
@magik6k
Copy link
Contributor

magik6k commented Jul 1, 2021

#6653 should fix the underlying Finalize issue, it would be really great if you were able to see if it fixes the issue for you

@lionsoul2014 lionsoul2014 reopened this Jul 2, 2021
@lionsoul2014
Copy link
Author

#6653 should fix the underlying Finalize issue, it would be really great if you were able to see if it fixes the issue for you

i am going to test it in a local 2k network.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants