Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented computation of segment replication stats at shard level #17055

Merged
merged 34 commits into from
Feb 27, 2025
Merged
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
44a1134
Implemented computation of segment replication stats at shard level
vinaykpud Jan 19, 2025
5d138e3
Updated style checks in the test
vinaykpud Jan 19, 2025
18664d2
Updated changelog
vinaykpud Jan 19, 2025
43d0798
Merge branch 'main' into node-stats
vinaykpud Jan 19, 2025
3468164
fixed style issues
vinaykpud Jan 19, 2025
4e693a5
Fix the failing integration test
vinaykpud Jan 21, 2025
5a3d1ef
Fix stylecheck
vinaykpud Jan 21, 2025
a94240f
Fixed the comments for the initial revision
vinaykpud Jan 21, 2025
dd0406d
Updated to use System.nanoTime() for lag calculation
vinaykpud Jan 21, 2025
1104c1f
Fixed the integration test for node stats
vinaykpud Jan 22, 2025
59e2617
Merge branch 'main' into node-stats
vinaykpud Jan 22, 2025
90b96a8
Modified the version in the ReplicationCheckpoint for backward compat…
vinaykpud Jan 23, 2025
7f465a0
Added precomputation logic for the stats calculation
vinaykpud Jan 27, 2025
23cedac
Merge branch 'main' into node-stats
vinaykpud Jan 27, 2025
28f1cfc
Removed unwanted lines
vinaykpud Jan 27, 2025
f80791f
Clean up the maps when index closed
vinaykpud Jan 27, 2025
29ffb01
Added a null check for the indexshard checkpoint
vinaykpud Jan 28, 2025
4fe2f87
fix style checks
vinaykpud Jan 28, 2025
5d1180f
Merge branch 'main' into node-stats
vinaykpud Jan 28, 2025
c838033
Updated version and added bwc for RemoteSegmentMetadata
vinaykpud Feb 4, 2025
1fdd5d2
Upated the javadoc comments
vinaykpud Feb 4, 2025
73efd49
Merge branch 'main' into node-stats
vinaykpud Feb 4, 2025
85dd342
Address comments PR
vinaykpud Feb 19, 2025
7252842
Merge branch 'main' into node-stats
vinaykpud Feb 19, 2025
c993370
Removed the latestReceivedCheckpoint map from SegmentReplicationTarge…
vinaykpud Feb 20, 2025
6879f4c
Added granular locks for the concurrency of stats methods
vinaykpud Feb 24, 2025
6ee1808
Style check fixes
vinaykpud Feb 24, 2025
4e2b335
Merge branch 'main' into node-stats
vinaykpud Feb 24, 2025
e928e80
Changes to maintain atomicity
vinaykpud Feb 25, 2025
04ba008
Merge branch 'main' into node-stats
vinaykpud Feb 25, 2025
3d030d5
spotlessApply
vinaykpud Feb 25, 2025
fcc57bf
removed querying the remotestore when replication is in progress
vinaykpud Feb 26, 2025
25fd006
Merge branch 'main' into node-stats
vinaykpud Feb 26, 2025
d8585f7
spotlessApply
vinaykpud Feb 26, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix style checks
Signed-off-by: Vinay Krishna Pudyodu <vinkrish.neo@gmail.com>
  • Loading branch information
vinaykpud committed Jan 28, 2025
commit 4fe2f873b9c0f9923ed327ffe1ba8cd6c19afdf3
Original file line number Diff line number Diff line change
@@ -137,7 +137,7 @@ public void updateReplicationCheckpointStats(final ReplicationCheckpoint latestC
long segmentInfosVersion = latestCheckPoint.getSegmentInfosVersion();
if (!existingCheckpointStats.containsKey(segmentInfosVersion)) {
ReplicationCheckpoint replicationCheckpoint = indexShard.getLatestReplicationCheckpoint();
if(replicationCheckpoint != null) {
if (replicationCheckpoint != null) {
long bytesBehind = calculateBytesBehind(latestCheckPoint, replicationCheckpoint);
if (bytesBehind > 0) {
existingCheckpointStats.put(