GRPC Streaming add channel buffer per subscription and metrics #1687
Conversation
WalkthroughThe recent changes encompass updates to workflow triggers, metrics, and gRPC streaming management. Workflow files now include triggers for branches matching Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Workflow
participant Metrics
participant gRPC Manager
Client->>Workflow: Push to jonfung/*
Workflow-->>Client: Trigger build and push
Client->>gRPC Manager: Subscribe to updates
gRPC Manager->>Metrics: Increment GrpcAddToSubscriptionChannelCount
gRPC Manager->>gRPC Manager: Buffer updates in updatesChannel
Metrics->>gRPC Manager: Track GrpcSubscriptionChannelLength
gRPC Manager->>Client: Send buffered updates
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
| @@ -6,6 +6,8 @@ on: # yamllint disable-line rule:truthy | |||
| - main | |||
| - 'release/protocol/v[0-9]+.[0-9]+.x' # e.g. release/protocol/v0.1.x | |||
| - 'release/protocol/v[0-9]+.x' # e.g. release/protocol/v1.x | |||
| - 'jonfung/**' # e.g. jonfung/hi | |||
There was a problem hiding this comment.
Actionable comments posted: 1
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (4)
- .github/workflows/protocol-build-and-push-snapshot.yml (1 hunks)
- .github/workflows/protocol-build-and-push.yml (1 hunks)
- protocol/lib/metrics/metric_keys.go (1 hunks)
- protocol/streaming/grpc/grpc_streaming_manager.go (11 hunks)
Additional comments not posted (7)
protocol/lib/metrics/metric_keys.go (3)
72-72: The new metricGrpcAddUpdateToBufferCountis well-named and follows the existing naming conventions. Good job on maintaining consistency.
73-73: The metricGrpcAddToSubscriptionChannelCountis appropriately named and aligns with the module's naming standards.
78-78: The metricGrpcSubscriptionChannelLengthis a useful addition for monitoring the length of the subscription channel. This should help in identifying bottlenecks or potential overflow issues..github/workflows/protocol-build-and-push.yml (1)
9-9: Adding the branch patternjonfung/**expands the conditions under which the workflow is triggered. Ensure that this is intended and that it won't lead to unintended builds or resource wastage.Verification successful
The search for the branch pattern
jonfung/**did not yield any relevant results in the workflow files or elsewhere in the repository, except for a comment in a test file. This suggests that the pattern is not used elsewhere and is likely intended to be isolated to the specified workflow file.
Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the usage of the new branch pattern in other workflow files. # Test: Search for the branch pattern usage. Expect: Consistent usage across workflow files. rg --type yaml $'jonfung/**'Length of output: 29
Script:
#!/bin/bash # Description: Verify the usage of the new branch pattern in all files. # Test: Search for the branch pattern usage across all files. Expect: Consistent usage or isolated to the specified workflow file. rg 'jonfung/**'Length of output: 133
.github/workflows/protocol-build-and-push-snapshot.yml (1)
9-10: The addition of thejonfung/**branch pattern here mirrors the change in the other workflow file. This consistency is good, but as previously mentioned, ensure it's intended to avoid unnecessary builds.protocol/streaming/grpc/grpc_streaming_manager.go (2)
Line range hint
47-59: The addition ofsubscriptionIdandupdatesChanneltoOrderbookSubscriptionis crucial for managing individual subscriptions effectively. Ensure proper synchronization mechanisms are in place to handle concurrent access to these channels.
168-177: The error handling for failed message sends is robust, logging the error and removing the subscription. This should help maintain the stability of the streaming service under failure conditions.
| @@ -17,6 +17,11 @@ import ( | |||
|
|
|||
| var _ types.GrpcStreamingManager = (*GrpcStreamingManagerImpl)(nil) | |||
|
|
|||
| const ( | |||
| // TODO CHANGE THIS CONFIGURABLE | |||
| MAX_BUFF_LEN = 1000 | |||
There was a problem hiding this comment.
Tip
Codebase Verification
The search results indicate that MAX_BUFF_LEN is currently hard-coded and there is a TODO comment suggesting that it should be made configurable. However, there is no existing configuration or documentation justifying the value of 1000.
protocol/streaming/grpc/grpc_streaming_manager.go: const (// TODO CHANGE THIS CONFIGURABLEMAX_BUFF_LEN = 1000
This confirms that the value is not yet configurable and lacks justification.
Analysis chain
Setting MAX_BUFF_LEN to 1000 is a significant change. Ensure this value is configurable or justified by load testing, as hard-coded limits can lead to scalability issues.
Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify if `MAX_BUFF_LEN` is configurable or justified.
# Test: Search for configuration options related to `MAX_BUFF_LEN`. Expect: Configurable options or documentation justifying the value.
rg --type go $'MAX_BUFF_LEN'
Length of output: 217
Script:
#!/bin/bash
# Description: Verify if `MAX_BUFF_LEN` is configurable or justified.
# Test 1: Search for configuration files or environment variables related to `MAX_BUFF_LEN`.
rg 'MAX_BUFF_LEN' --type toml --type yaml --type json
rg 'MAX_BUFF_LEN' --type sh
# Test 2: Search for comments or documentation justifying the value of `MAX_BUFF_LEN`.
rg 'MAX_BUFF_LEN' --type go --context 5
Length of output: 1808
This reverts commit 76b5613.
There was a problem hiding this comment.
Actionable comments posted: 1
Outside diff range and nitpick comments (6)
protocol/app/app.go (6)
Line range hint
381-381: The functioninitDatadogProfileris undefined. Ensure that it is implemented or imported correctly.
Line range hint
383-383: The functionGetEncodingConfigis undefined. Ensure that it is implemented or imported correctly.
Line range hint
498-498: The variablemaccPermsis undefined. Ensure that it is declared or imported correctly.
Line range hint
507-507: The methodMsgServiceRouteris undefined for the type*App. Ensure that it is declared correctly in theAppstruct.Also applies to: 607-607, 652-652
Line range hint
515-515: The functionBlockedAddressesis undefined. Ensure that it is implemented or imported correctly.
Line range hint
774-774: The methodLoggerorGRPCQueryRouteris undefined for the type*App. Ensure that these methods are declared correctly in theAppstruct.Also applies to: 1379-1379, 1385-1385
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (4)
- protocol/app/app.go (1 hunks)
- protocol/app/flags/flags.go (7 hunks)
- protocol/app/flags/flags_test.go (5 hunks)
- protocol/streaming/grpc/grpc_streaming_manager.go (10 hunks)
Files skipped from review as they are similar to previous changes (1)
- protocol/streaming/grpc/grpc_streaming_manager.go
Additional context used
golangci-lint
protocol/app/app.go
381-381: undefined: initDatadogProfiler (typecheck)
383-383: undefined: GetEncodingConfig (typecheck)
498-498: undefined: maccPerms (typecheck)
507-507: app.MsgServiceRouter undefined (type *App has no field or method MsgServiceRouter) (typecheck)
515-515: undefined: BlockedAddresses (typecheck)
607-607: app.MsgServiceRouter undefined (type *App has no field or method MsgServiceRouter) (typecheck)
652-652: app.MsgServiceRouter undefined (type *App has no field or method MsgServiceRouter) (typecheck)
774-774: app.Logger undefined (type *App has no field or method Logger) (typecheck)
1379-1379: app.GRPCQueryRouter undefined (type *App has no field or method GRPCQueryRouter) (typecheck)
1385-1385: app.GRPCQueryRouter undefined (type *App has no field or method GRPCQueryRouter) (typecheck)
Additional comments not posted (9)
protocol/app/flags/flags.go (6)
24-27: The addition of new flags for gRPC streaming configuration is clear and well-structured.
44-47: The new constants for gRPC streaming flags are consistent with existing naming conventions.
60-63: The default values for the new gRPC streaming flags are appropriately set.
105-112: The new flags are correctly added to the command with clear descriptions and default values.
133-141: The validation logic for the new gRPC streaming flags is robust, ensuring that all values are positive.
219-227: The logic to retrieve and cast the new gRPC streaming flag values from options is correctly implemented.protocol/app/flags/flags_test.go (3)
38-42: The tests correctly verify the presence of the new gRPC streaming flags in the command.
Line range hint
75-127: The validation tests for the new gRPC streaming flags are comprehensive, covering a range of scenarios to ensure robustness.
Line range hint
147-244: The tests for retrieving gRPC streaming flag values from options are thorough and effectively validate the functionality.
| appFlags.GrpcStreamingMaxBatchSize, | ||
| appFlags.GrpcStreamingMaxChannelBufferSize, |
There was a problem hiding this comment.
The GrpcStreamingManager constructor is called with incorrect parameters.
- return streaming.NewGrpcStreamingManager(
- logger,
- appFlags.GrpcStreamingFlushIntervalMs,
- appFlags.GrpcStreamingMaxBatchSize,
- appFlags.GrpcStreamingMaxChannelBufferSize,
- )
+ return streaming.NewGrpcStreamingManager(
+ logger,
+ time.Duration(appFlags.GrpcStreamingFlushIntervalMs) * time.Millisecond, // Ensure time duration is correctly formatted
+ appFlags.GrpcStreamingMaxBatchSize,
+ appFlags.GrpcStreamingMaxChannelBufferSize,
+ )Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| appFlags.GrpcStreamingMaxBatchSize, | |
| appFlags.GrpcStreamingMaxChannelBufferSize, | |
| return streaming.NewGrpcStreamingManager( | |
| logger, | |
| time.Duration(appFlags.GrpcStreamingFlushIntervalMs) * time.Millisecond, // Ensure time duration is correctly formatted | |
| appFlags.GrpcStreamingMaxBatchSize, | |
| appFlags.GrpcStreamingMaxChannelBufferSize, | |
| ) |
Adds a channel buffer per subscription so that the full node won't stall on laggy connections.
Also adds fine-grained metrics.
Summary by CodeRabbit
New Features
jonfung/**.Updates
Refactor
GrpcStreamingMaxBufferSizetoGrpcStreamingMaxBatchSizeand introducedGrpcStreamingMaxChannelBufferSizein configuration and validation.Tests