[DataCap Application] Ghost Byte Inc - Human PanGenomics Project [3/3]

[Trevor-K-Smith]

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

DP Info

• Organization Name: Ghost Byte Inc
• Website / Social Media: https://ghostbyte.io/
• Total amount of DataCap being requested (between 500 TiB and 5 PiB): 5 PiB
• Weekly allocation of DataCap requested (usually between 1-100TiB): 512 TiB
• On-chain address for first allocation: f13sg6o4kkzdkrda6xumjrplntswhkapaywq67ita

Client Info

• Organization Name: Human PanGenomics Project
• Website / Social Media: https://registry.opendata.aws/hpgp-data/

Data Info

• Dataset Size: 1.32 PiB
• Replications: 10x
• Total Datacap Request: 13.2 PiB

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

Ghost Byte Inc is a storage provider seeking to onboard data to meet the high demand of FIL+ for itself and its partners. Ghost Byte has a history of actively participating in NA weekly calls, helping community members on the slack channel, testing beta software with feedback, and overall ongoing support in the community of filecoin. Ghost Byte works with industry partners to assist the growth in web3 adoption.
Ref: https://www.youtube.com/watch?v=6PejYUlN0AM

What is the primary source of funding for this project?

Ghost Byte Inc

What other projects/ecosystem stakeholders is this project associated with?

Ghost Byte Inc

Use-case details

Describe the data being stored onto Filecoin

Each parent in the trio was sequenced with Illumina short reads, each child was sequenced with Illumina short reads, 10X Genomics, Nanopore, PacBio CLR and HiFi, Bionano and Hi-C.

For nanopore datasets, each folder contains the fast5, fastq (basecalled with Guppy 2.3.5 flip flop with the high accuracy model), and a sequencing summary file.

For PacBio CLR data, each folder contains a subread bam file which can be converted to fasta/q using either bam2fastq or samtools fasta. The HiFi folders contain ccs.bam files which have already been converted from subreads into high-fidelity reads. As before, they can be converted to fasta/q using bam2fastq or samtools fasta.

For Bionano data, each folder contains both the assembled optical map (cmap) and the individual molecules (bnx.gz)

For the remaining short-read data, each folder contains one or more subfolders with fastq.gz files.

Below is short blub on details of the data.

Where was the data in this dataset sourced from?

This data is being replicated from AWS Opendata to Filecoin. The dataset being replicated is Human PanGenomics Project. This dataset contains 188,823 Total Objects and is 1.32PiB Total Size. The data will be replicated a total of 10 times for a total datacap request of 13.2 PiB.
Ref: https://registry.opendata.aws/hpgp-data/

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this.

https://www.dropbox.com/sh/ksod9gdgq1m0cp2/AADkprdDb4zdiO44sEvUDe8Ya?dl=0

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Public -https://registry.opendata.aws/hpgp-data/

What is the expected retrieval frequency for this data?

1-3 Year

For how long do you plan to keep this dataset stored on Filecoin?

540 Days, subject to renewal when the time comes.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

Global partners. No restrictions, will spread out as much as possible.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Data will be send over boostd to participating storage providers. Otherwise, offline deals can be done for those with special requirements.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

Storage Providers will be found in the active community slack, partners met at events, and online data sources. 1 per actor, 2 per organization, spread as evenly across the globe a possible. Total 10 replications. SP's will be confirmed ahead of replication of cars that they intend to allow the cars to be accessible and retrievable. Car files for this replication will not be enforced to keep unsealed sectors as this is disaster recovery data, and retrievals will be low. 

How will you be distributing deals across storage providers?

Singularity will be used to serve up the deals and track the progress of each CAR file being replicated. 1 per actor, 2 per organization, spread as evenly across the globe a possible. Total 10 replications.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes, we have the resources to get started right away. We do not need help at this time. Thank you!

[large-datacap-requests]

Thanks for your request!
Everything looks good. 👌

A Governance Team member will review the information provided and contact you back pretty soon.

[simonkim0515]

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

100TiB

Client address

f1bqhekdcmuqgnajsdkvmhqefaj7jexnob565rzya

[large-datacap-requests]

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f13sg6o4kkzdkrda6xumjrplntswhkapaywq67ita

DataCap allocation requested

50TiB

Id

f4caaab2-ed08-4cf9-b6f9-683717df5f0b

[psh0691]

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebnnkgnn4nn7sjlflxlrmneqbkby5o2lpkfe64bmzslzkbd6vzfhg

Address

f1bqhekdcmuqgnajsdkvmhqefaj7jexnob565rzya

Datacap Allocated

50.00TiB

Signer Address

f1qdko4jg25vo35qmyvcrw4ak4fmuu3f5rif2kc7i

Id

f4caaab2-ed08-4cf9-b6f9-683717df5f0b

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebnnkgnn4nn7sjlflxlrmneqbkby5o2lpkfe64bmzslzkbd6vzfhg

[cryptowhizzard]

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecpvgfhdefggah7qngunomstgflnalomgm367uajj32znx7iozzpm

Address

f1bqhekdcmuqgnajsdkvmhqefaj7jexnob565rzya

Datacap Allocated

50.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

f4caaab2-ed08-4cf9-b6f9-683717df5f0b

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecpvgfhdefggah7qngunomstgflnalomgm367uajj32znx7iozzpm

[large-datacap-requests]

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!

[Trevor-K-Smith]

@simonkim0515 @raghavrmadya

Can i kindly get this application reviewed? Thanks

[herrehesse]

Dear Filecoin+ Github applicant,

We have noticed that the dataset is already (partly) on chain. While we appreciate your enthusiasm to contribute to the Filecoin network, we want to remind you that this behaviour may not be beneficial to the network. Can you explain to me what happend here?

Thank you for your understanding and cooperation.

[Trevor-K-Smith]

@herrehesse

Looking at the dates on all the other applications, they are all after mine.

I performed research and selected a dataset to be on boarded to the network. At the time of issue creation, the dataset was not on the filecoin network. The applications posted above all have creation dates well over a month after mine was created. They did not do the research to find a dataset not already in process to be onboarded to the network.

My applications are clearly labeled. They should have seen them.

Why the other applications are being approved when i had prior approval for this entire dataset?

[large-datacap-requests]

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!

[large-datacap-requests]

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!

[xinaxu]

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebwssyfgkcbbsisah3gy76js52ekaie6vl424a2e2ibbjieuu3bvu

Address

f13sg6o4kkzdkrda6xumjrplntswhkapaywq67ita

Datacap Allocated

50.00TiB

Signer Address

f1k3ysofkrrmqcot6fkx4wnezpczlltpirmrpsgui

Id

f4caaab2-ed08-4cf9-b6f9-683717df5f0b

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebwssyfgkcbbsisah3gy76js52ekaie6vl424a2e2ibbjieuu3bvu

[data-programs]

This user’s identity has been verified through filplus.storage