Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Describe size limitations #171

Open
realloc opened this issue Aug 26, 2021 · 5 comments
Open

Describe size limitations #171

realloc opened this issue Aug 26, 2021 · 5 comments
Labels
documentation Improvements or additions to documentation enhancement Improving existing functionality I3 Minimal impact S4 Routine U4 Nothing urgent

Comments

@realloc
Copy link

realloc commented Aug 26, 2021

There is no clear statement on the maximal sizes of object header, attribute, eACL records, etc. It may be important for different language implementations.

@AnnaShaleva
Copy link
Member

AnnaShaleva commented May 29, 2023

The problem with missing object header size limitation affects the Neo Oracle service. One of Oracle NeoFS request types is getting header, and large header may be a problem for the Oracle node. Even if it won't fit in MaxOracleResponseSize limit, the Oracle node will try to load the whole header into memory before the response creation which may cause OOM. Ref. neo-project/neo-modules#803.

@roman-khimov
Copy link
Member

Describing an existing limitation and adding a limit where it never existed are somewhat different. In some cases from above it's the first, but in some (like object header size) it's the second.

@AnnaShaleva
Copy link
Member

AnnaShaleva commented May 30, 2023

It should be noted (thanks to @notimetoname digging) that although there's no explicit limit from the neo-fs side, the header itself is restricted by a single gRPC package (at least ObjectHead operation as far as object replications during ObjectPut). From the server-side gRPC client, this constraint is set to be max int32 value by default, and this default isn't changed anywhere. From the client side this value even smaller and set to be 4MB by default, this default isn't changed anywhere inside the neo-fs as far. Thus, if using neofs-api-go client, the user can't get header with size more than 4MB.

Thus, within the scope of this issue it would also be nice to provide the ability to edit client/server gRPC package size constraints as an option.

For now, consider neo-go Oracle response limit problem, we can be sure that the node is safe even if it doesn't restrict the received header size and no OOM could happen beause we have this implisit gRPC package limit. However, the C# node still may face with this issue. @roman-khimov, what do you think?

@roman-khimov
Copy link
Member

roman-khimov commented May 30, 2023

There has to be some sane NeoFS protocol limit for it, simple as that. 4M is too much, we've got 64M limit for the payload for the record. We need to take a closer look at this, but 4K seems to be a nice limit to me, at least I'd expect to see some value of that order for it.

@cthulhu-rider
Copy link
Contributor

we can start with some small value 1-4K, so we'll be able to increase the limit if there is no other way

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement Improving existing functionality I3 Minimal impact S4 Routine U4 Nothing urgent
Projects
None yet
Development

No branches or pull requests

4 participants