Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Extended search with attributes returned #306

Closed
roman-khimov opened this issue Sep 3, 2024 · 1 comment
Closed

Extended search with attributes returned #306

roman-khimov opened this issue Sep 3, 2024 · 1 comment
Labels
feature Completely new functionality I2 Regular impact S2 Regular significance U4 Nothing urgent
Milestone
v2.17.0

Comments

@roman-khimov
Copy link
Member

Is your feature request related to a problem? Please describe.

I'm always frustrated when we can't have something like ORDER BY in our searches. It'd simplify nspcc-dev/neo-go#3496 greatly, but:

  • individual node can't sort objects, it doesn't know the criteria
  • even if it's to know the criteria we can't merge results from different nodes

Describe the solution you'd like

We can extend search result with attributes if they're requested. It can be different from "ORDER BY", something like "WITH FIELD", but the result would be []{oid, []attributeValue} (ordered by requested attributes) instead of []oid. Then the request originator can do anything it wants with these fields including ordering or choosing the most appropriate version of an object (REST).

Describe alternatives you've considered

Dunno. Currently it's just an idea.
@roman-khimov roman-khimov added U4 Nothing urgent S2 Regular significance I2 Regular impact feature Completely new functionality labels Sep 3, 2024
@roman-khimov
Copy link
Member Author

By "FIELD" I mean attribute key, of course.

@roman-khimov roman-khimov mentioned this issue Sep 19, 2024
Open
@roman-khimov roman-khimov mentioned this issue Dec 11, 2024
Open
roman-khimov added a commit that referenced this issue Dec 18, 2024
@roman-khimov
object: specify SearchV2, fix #306
6e65d39 
1. Additional fields allow to avoid HEADs in most cases since applications
   tend to use 1-2 fields to choose between results/check for flags and pick
   the object they want to GET. Or they just want to have some fields to
   display object list, but they don't need a full HEAD.
2. Sorting is critical here, absent sorting we can't limit reply in number of
   elements returned and we can't have cursors to continue search.
3. Limits on number of returned elements allow to use single message instead
   of a stream. 4M of the default message fits 256 full-sized 16K headers and
   we expect just a few attributes in fact, so 1000 elements shouldn't be a
   problem.
4. We can't go backwards with the order easily, so this is omitted for now.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
@roman-khimov roman-khimov mentioned this issue Dec 18, 2024
Closed
roman-khimov added a commit that referenced this issue Dec 23, 2024
@roman-khimov
object: specify SearchV2, fix #306
b35bf05 
1. Additional fields allow to avoid HEADs in most cases since applications
   tend to use 1-2 fields to choose between results/check for flags and pick
   the object they want to GET. Or they just want to have some fields to
   display object list, but they don't need a full HEAD.
2. Sorting is critical here, absent sorting we can't limit reply in number of
   elements returned and we can't have cursors to continue search.
3. Limits on number of returned elements allow to use single message instead
   of a stream. 4M of the default message fits 256 full-sized 16K headers and
   we expect just a few attributes in fact, so 1000 elements shouldn't be a
   problem.
4. We can't go backwards with the order easily, so this is omitted for now.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
@roman-khimov roman-khimov added this to the v2.17.0 milestone Dec 24, 2024
cthulhu-rider added a commit that referenced this issue Feb 13, 2025
@cthulhu-rider
object: Increase number of attributes to 8 in SearchV2
7ba2b13 
A bit more to cover current NeoFS S3 gateway needs.

Refs #306.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
cthulhu-rider added a commit that referenced this issue Feb 13, 2025
@cthulhu-rider
object: Increase number of requested attributes for SearchV2
9c8ea62 
A bit more to cover current NeoFS S3 gateway needs.

Refs #306.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
cthulhu-rider added a commit that referenced this issue Feb 18, 2025
@cthulhu-rider
object: Require 1st filter and attribute equality in SearchV2
0264841 
Without this, it makes almost impossible to make proper inter-node
merge-sort and continuation. Since not all matchers allow understanding
the primary attribute's value, compliance with the presented requirement
is in demand.

Refs #306.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
cthulhu-rider added a commit that referenced this issue Feb 18, 2025
@cthulhu-rider
object: Require 1st filter and attribute equality in SearchV2
b5fdf47 
Without this, it makes almost impossible to make proper inter-node
merge-sort and continuation. Since not all matchers allow understanding
the primary attribute's value, compliance with the presented requirement
is in demand.

Refs #306.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
cthulhu-rider added a commit that referenced this issue Feb 19, 2025
@cthulhu-rider
object: Free from sorting by secondary attributes in SearchV2
ebdb8cc 
There are currently no known use cases where it is necessary and/or
could be more efficient. For example, NeoFS S3 Gateway requests
timestamps, but it is interested in their descending order (fresh data
first). Therefore, for now, secondary sorting is left to the apps.

Refs #306.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
cthulhu-rider added a commit that referenced this issue Feb 19, 2025
@cthulhu-rider
object: Mention numeric limits for SearchV2
a06fb77 
The selected range should cover all practical use cases for years to
come.

Refs #306.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
cthulhu-rider added a commit that referenced this issue Feb 24, 2025
@cthulhu-rider
object: Free from sorting by secondary attributes in SearchV2
8988cad 
There are currently no known use cases where it is necessary and/or
could be more efficient. For example, NeoFS S3 Gateway requests
timestamps, but it is interested in their descending order (fresh data
first). Therefore, for now, secondary sorting is left to the apps.

Refs #306.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
cthulhu-rider added a commit that referenced this issue Feb 24, 2025
@cthulhu-rider
object: Mention numeric limits for SearchV2
febb3bf 
The selected range should cover all practical use cases for years to
come.

Refs #306.

Signed-off-by: Leonard Lyubich <leonard@morphbits.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature Completely new functionality I2 Regular impact S2 Regular significance U4 Nothing urgent
Projects
None yet
Milestone
v2.17.0
Development

No branches or pull requests
1 participant
@roman-khimov