Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: Fix FinalizeSector with sectors in stoage paths #6653

Merged
merged 1 commit into from
Jul 2, 2021

Conversation

magik6k
Copy link
Contributor

@magik6k magik6k commented Jul 1, 2021

This fixes the case where sectors could be moved in one call to Finalize to a long-term storage, and then fail in second call to Finalize because we expected to find those files still in sealing storage

Related to #6651 and #6649

Copy link
Contributor

@ZenGround0 ZenGround0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confused about some of the basics but what I understand looks good

itests/kit/node_miner.go Outdated Show resolved Hide resolved
@@ -14,7 +14,7 @@ func TestDealsWithSealingAndRPC(t *testing.T) {

kit.QuietMiningLogs()

var blockTime = 1 * time.Second
var blockTime = 50 * time.Millisecond
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double checking: this is just to speed up the test and has no behavior change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct

selector := newExistingSelector(m.index, sector.ID, storiface.FTCache|storiface.FTSealed, false)

err := m.sched.Schedule(ctx, sector, sealtasks.TTFinalize, selector,
m.schedFetch(sector, storiface.FTCache|storiface.FTSealed|unsealed, storiface.PathSealing, storiface.AcquireMove),
m.schedFetch(sector, storiface.FTCache|storiface.FTSealed|unsealed, pathType, storiface.AcquireMove),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am brand new to this code so I have some very basic questions.

If we have already finalized this sector id then all of the sealedStores above will have store.CanSeal equal to false ?

Why do we need the worker to finalize again if the pathType is PathStorage? Do sectors in long term storage need to be finalized again during subsequent finalize calls? Similarly why do we need to do the MoveStorage below in this case? Aren't all the sectors already in long term storage?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have already finalized this sector id then all of the sealedStores above will have store.CanSeal equal to false ?

In some setups storage paths can be used both for sealing and storage

Why do we need the worker to finalize again if the pathType is PathStorage?

It's possible that the sector wasn't properly finalized for a number of other reasons (sealing path switched to storage, sectors moved manually, etc.). The finalize operation is quite cheap (removing/trimming some files), so there is little downside to calling it twice for the added robustness.

Similarly why do we need to do the MoveStorage below in this case? Aren't all the sectors already in long term storage?

MoveStorage already has that check internally

@magik6k magik6k force-pushed the fix/finalize-in-storage branch from ff9c7a0 to 8a94ab6 Compare July 2, 2021 17:54
@magik6k magik6k merged commit e2f48b2 into master Jul 2, 2021
@magik6k magik6k deleted the fix/finalize-in-storage branch July 2, 2021 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants