Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Community Diligence Review of GreaterHeat Allocator #14

Closed
filecoin-watchdog opened this issue May 10, 2024 · 6 comments
Closed

Community Diligence Review of GreaterHeat Allocator #14

filecoin-watchdog opened this issue May 10, 2024 · 6 comments
Assignees
Labels
Diligence Audit in Process Governance team is reviewing the DataCap distributions and verifying the deals were within standards

Comments

@filecoin-watchdog
Copy link
Collaborator

filecoin-watchdog commented May 10, 2024

Review of GreaterHeat Allocations from @AlanGreaterheat
Allocator Application: filecoin-project/notary-governance#1041

First example:
DataCap was given to:
AlanGreaterheat/Greaterheat-Allocator#7

Public Open Dataset - key compliance requirement: Retrievability

1st point)
Allocation schedule per allocator:
First Round: 512 TiB
Second Round: 1 PiB
Third Round: 2 PiB
Fourth Round: 2 PiB
Overall Cap for Each Client: (2 PiB)

Actual allocation: 500Tib, 1PiB - this follows the guidelines

2nd point)
No sign of KYC or KYB of client or dataset as mentioned in allocator application.

3rd point)
Client said these were the SPs,
f03044609 Kenny Korea
f02953066 bluesky USA
f02837684 Xunlei HK

SPs to be prepared soon:
f02956073 Alice Singapore
f03028412 ZC Guangdong

Actual data storage report:
https://check.allocator.tech/report/AlanGreaterheat/Greaterheat-Allocator/issues/7/1715347592512.md

Provider | Location | Total Deals Sealed | Percentage | Unique Data | Duplicate Deals
f03028412 | Guangzhou, Guangdong, CNHuawei Cloud Service data center | 44.78 TiB | 8.68% | 44.78 TiB | 0.00%
f02953066 | Los Angeles, California, USIPTELECOM ASIA | 148.47 TiB | 28.77% | 148.47 TiB | 0.00%
f02837684 | Hong Kong, Hong Kong, HKIPTELECOM Global | 149.94 TiB | 29.06% | 149.94 TiB | 0.00%
f02956073 | Singapore, Singapore, SGIPTELECOM Global | 49.97 TiB | 9.68% | 49.97 TiB | 0.00%
f03044609 | Paripark, Seoul, KRThe Constant Company, LLC | 122.81 TiB | 23.80% | 122.81 TiB | 0.00%

5 of 5 SPs listed match those taking deals per report. Additional diligence needed to confirm entity and actual storage locations

4th point)
Second allocation awarded to this client.
However, per Spark dashboard, all SPs are either not available or have 0% retrievability.

The Allocator showed no sign of diligence after 1st allocation and gave the 2nd allocation of 1PiB to the client.

@Kevin-FF-USA Kevin-FF-USA self-assigned this May 13, 2024
@Kevin-FF-USA Kevin-FF-USA added the Diligence Audit in Process Governance team is reviewing the DataCap distributions and verifying the deals were within standards label May 13, 2024
@Kevin-FF-USA
Copy link
Collaborator

Hi @AlanGreaterheat

On the next Fil+ Allocator meeting we will be going over each refill application. Wanted to ensure you were tracking the review discussion taking place in #14.

If your schedule allows, recommend coming to the May 28th meeting to answer/discuss the issues raised in the recent distributions. This will allow you to faster address - or, the issue in Allocator Governance for ongoing written discussion.

Warmly,
-Kevin
https://calendar.google.com/calendar/embed?src=c_k1gkfoom17g0j8c6bam6uf43j0%40group.calendar.google.com&ctz=America%2FLos_Angeles

image

@galen-mcandrew
Copy link
Collaborator

Based on an additional compliance review, it appears this allocator is attempting to work with public open dataset clients.

However, the data associated with this pathway is not currently able to be retrieved at scale, and testing for retrieval is currently noncompliant.

As a reminder, the allocator team is responsible for verifying, supporting, and intervening with their clients. If a client is NOT providing accurate deal-making info (such as incomplete or inaccurate SP details) or making deals with noncompliant unretrievable SPs, then the allocator needs to intervene and require client updates before more DataCap should be awarded.

Before we will submit a request for more DataCap to this allocator, please verify that you will instruct, support, and require your clients to work with retrievable storage providers.

@AlanGreaterheat can you verify that you will enforce retrievability requirements, such as through Spark? Please reply here with acknowledgement and any additional details for our review.

@AlanGreaterheat
Copy link

Of course, we will follow the official requirements for Spark retrieval rate compliance requirements (after the launch of the Spark tool), and at the same time, we also require clients to contact partner SPs to communicate to improve the Spark retrieval success rate.
image

Happily, the Spark retrieval rate is steadily improving in recent stages.

last week
image

now
image

@AlanGreaterheat
Copy link

AlanGreaterheat commented Jun 12, 2024

According to today's latest governance meeting, we made a live speech, and based on Kevin-FF's @Kevin-FF-USA suggestion, before the governance team further refreshes Datacap, as Greaterheat deeply rooted in the Filecoin ecosystem, we would like to clarify again that we hope to obtain a Datacap quota of 10PiB @galen-mcandrew .

Regarding the second point of the report: No sign of KYC or KYB of client or dataset as mentioned in allocator application.

In fact, we have conducted multiple rounds of investigations, and we also pay attention to the past reputation of GitHub accounts.
image
image

Regarding the third point of the report: As for the geographical location, the accuracy is 100%, and we even conducted random email location confirmations.
image

Regarding the fourth point of the report: Regarding the Spark retrieval rate, before officially launching the Spark retrieval rate, we have already allocated according to the original checker: manualTrigger display report, and the report shows no issues.
image

After participating in the governance meetings of the past two weeks, we requested clients to contact SPs and research the Spark retrieval principle. It can be seen that the Spark retrieval rate has improved, some even reaching 68.71%, which is about to meet the governance team's requirement of 85%.
image

At the same time, during the feedback process with the Spark team, the SP technical personnel acknowledged that Spark still needs to be upgraded to work in coordination with the allocator path.
image

In conclusion, the improvement of the fil-plus ecosystem requires efforts from all parties. In the subsequent 10PiB DC allocation process, Greaterheat will focus on data retrieval, closely monitor SP disclosure, and make the fil-plus ecosystem more prosperous.

@AlanGreaterheat
Copy link

@galen-mcandrew @Kevin-FF-USA

I would like to follow up on this. Appreciate to let me know anything else I need to provide to expedite this request for DC refill.It is encouraging to note that the success rate of SP spark searches is increasing.

image

@AlanGreaterheat
Copy link

@galen-mcandrew @Kevin-FF-USA Another update based on the latest governance meeting, Spark retrieval rates for nodes are improving, looking forward to good news, thanks!

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Diligence Audit in Process Governance team is reviewing the DataCap distributions and verifying the deals were within standards
Projects
None yet
Development

No branches or pull requests

4 participants