Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor structure of stats module #736

Merged
merged 3 commits into from
Jan 23, 2023

Conversation

jmazanec15
Copy link
Member

@jmazanec15 jmazanec15 commented Jan 20, 2023

Description

Removes KNNStatsConfig structure. This structure only had a single static variable. Moved the logic directly into the KNNStats class into a private non-static method. This will allow us to pass dependencies into stats during construction as opposed to have to initialize them statically.

Modifies KNNStatsRequest to not take validStats as an argument. Valid stats are the same every time so I dont see reason for making this configurable. Kept the stream logic to prevent break in bwc.

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Removes valid stats arg from KNNStatsRequest. Valid stats is a global
constant so there is not a good reason to make it configurable. It is
still written in and out of the stream. This is to prevent breakage in
any kind of backwards compatibility.

Signed-off-by: John Mazanec <jmazane@amazon.com>
Removes KNNStatsConfig class and moves logic directly into the KNNStats
class. KNNStatsConfig just hosted a single map of the stats, so it felt
unneccessary to have the whole class.

Signed-off-by: John Mazanec <jmazane@amazon.com>
@jmazanec15 jmazanec15 added Refactoring Improve the design, structure, and implementation while preserving its functionality backport 2.x labels Jan 20, 2023
@jmazanec15 jmazanec15 requested a review from a team January 20, 2023 18:39
public class StatsIT extends AbstractRollingUpgradeTestCase {
private KNNStats knnStats = new KNNStats(KNN_STATS);
private KNNStats knnStats = new KNNStats();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you want to init this KNNStats the better way to do is using a function having with annotation @before on top of it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure will update.

@@ -75,4 +76,73 @@ private Map<String, KNNStat<?>> getClusterOrNodeStats(Boolean getClusterStats) {
}
return statsMap;
}

private Map<String, KNNStat<?>> buildStatsMap() {
return ImmutableMap.<String, KNNStat<?>>builder()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's break this function into smaller chunks of private functions which tells what stats we are adding. So that it becomes easier to debug and read.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, will update.

@@ -23,8 +24,7 @@ public class KNNStatsRequest extends BaseNodesRequest<KNNStatsRequest> {
* Key indicating all stats should be retrieved
*/
public static final String ALL_STATS_KEY = "_all";

private Set<String> validStats;
private Set<String> validStats = StatNames.getNames();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

init this variable via constructor rather than here, this will help us mock things later on if required.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update.

@navneet1v
Copy link
Collaborator

There is NPE happening in one of the Tests. please fix that too.

@jmazanec15 jmazanec15 requested a review from navneet1v January 23, 2023 00:09
@codecov-commenter
Copy link

codecov-commenter commented Jan 23, 2023

Codecov Report

Merging #736 (c7844ed) into main (8e2ad45) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main     #736      +/-   ##
============================================
- Coverage     84.85%   84.85%   -0.01%     
- Complexity     1074     1075       +1     
============================================
  Files           152      151       -1     
  Lines          4364     4370       +6     
  Branches        390      390              
============================================
+ Hits           3703     3708       +5     
+ Misses          483      482       -1     
- Partials        178      180       +2     
Impacted Files Coverage Δ
...va/org/opensearch/knn/index/KNNCircuitBreaker.java 60.00% <100.00%> (-20.00%) ⬇️
...main/java/org/opensearch/knn/plugin/KNNPlugin.java 100.00% <100.00%> (ø)
...pensearch/knn/plugin/rest/RestKNNStatsHandler.java 90.00% <100.00%> (-0.91%) ⬇️
...java/org/opensearch/knn/plugin/stats/KNNStats.java 100.00% <100.00%> (+20.00%) ⬆️
...ensearch/knn/plugin/transport/KNNStatsRequest.java 60.00% <100.00%> (+12.17%) ⬆️
...ain/java/org/opensearch/knn/index/KNNSettings.java 84.37% <0.00%> (-2.35%) ⬇️
...ava/org/opensearch/knn/plugin/stats/StatNames.java 100.00% <0.00%> (+10.25%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Signed-off-by: John Mazanec <jmazane@amazon.com>
@jmazanec15 jmazanec15 merged commit 185b54d into opensearch-project:main Jan 23, 2023
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-736-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 185b54daee1f9009c7a67dfa24adb153cfdcd160
# Push it to GitHub
git push --set-upstream origin backport/backport-736-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-736-to-2.x.

jmazanec15 added a commit to jmazanec15/k-NN-1 that referenced this pull request Jan 23, 2023
Removes valid stats arg from KNNStatsRequest. Valid stats is a global
constant so there is not a good reason to make it configurable. It is
still written in and out of the stream. This is to prevent breakage in
any kind of backwards compatibility.

Removes KNNStatsConfig class and moves logic directly into the KNNStats
class. KNNStatsConfig just hosted a single map of the stats, so it felt
unneccessary to have the whole class.

Signed-off-by: John Mazanec <jmazane@amazon.com>
(cherry picked from commit 185b54d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Refactoring Improve the design, structure, and implementation while preserving its functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants