Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sector-storage supports snap deals #7525

Closed
BlocksOnAChain opened this issue Oct 15, 2021 · 6 comments
Closed

sector-storage supports snap deals #7525

BlocksOnAChain opened this issue Oct 15, 2021 · 6 comments
Assignees
Milestone

Comments

@BlocksOnAChain
Copy link
Contributor

BlocksOnAChain commented Oct 15, 2021

This issue will be used to track our design + implementation efforts to create a more flexible way to sector storage for SnapDeals

Direct DRI to deliver will be:
@magik6k will do the design + @ZenGround0 will be involved

  • @ZenGround0 will do the implementation after we approve the design
@BlocksOnAChain
Copy link
Contributor Author

@BlocksOnAChain we should share redesigns with the Venus team, but we still believe this is just a change on the Lotus side.

@jennijuju
Copy link
Member

In case the UpdateReplica message fails, lotus should be able to keep the to-be-upgrade CC sectors and reuse it for new deals (we probably should keep the copy of the data for the original CC sectors until the upgrade is confirmed to be successful.

@jennijuju
Copy link
Member

It would be also nice to have some sort of configuration variable for storage provider to set, if the current collateral is Y FIL higher than the CC sector collateral, upgrade a CC sector for the deals instead of sealing a new deal sector

@ZenGround0
Copy link
Contributor

FYI Currently I'm adding a new file type for updated replicas and their caches. This won't generalize too nicely with multiple snap deals updates but it should be enough for the next update we expect to release. Will link a draft PR soon.

@BlocksOnAChain BlocksOnAChain changed the title Redesign/design more flexible way sector storage for SnapDeals Redesign/design more flexible ways to sector storage for SnapDeals Nov 11, 2021
@ZenGround0 ZenGround0 changed the title Redesign/design more flexible ways to sector storage for SnapDeals sector-storage supports snap deals Nov 11, 2021
@ZenGround0
Copy link
Contributor

ZenGround0 commented Nov 11, 2021

Description

The lotus-miner binary's proving logic is organized into roughly two layers, the FSM which handles chain and deal subsystem interaction and implements configurable operation logic, and the sector storage layer which manages sector data and associated files, calling into the proving ffi as needed.

This issue is to add snap deals replica updating support to the sector-storage part of this stack. It requires integrating the FFI, adding filetypes, storage manager, scheduler and worker support for doing these ffi operations and storing relevant files and presenting an interface for the FSM

Done criteria

  1. There is a test that runs the sector-storage manager snap deals replica update function on a real sealed sector and successfully validates the proof of update. To start it should run on 2k sectors. This test will then be handed off to people with good hardware and tested on all sector sizes.
  2. There is an interface for the FSM to call into sector storage. This will be a thin wrapper around the manager call covered by the above test.

Dependencies

This PR needs the following things done before it can be completed

Ongoing work

#7615

@jennijuju
Copy link
Member

closed by #7615

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants