Nats module improvements #22445
Nats module improvements #22445
Conversation
💚 Flaky test reportTests succeeded. Expand to view the summary
Test stats 🧪
|
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
💚 Flaky test reportTests succeeded. Expand to view the summary
Test stats 🧪
|
Signed-off-by: chrismark <chrismarkou92@gmail.com>
Signed-off-by: chrismark <chrismarkou92@gmail.com>
Signed-off-by: chrismark <chrismarkou92@gmail.com>
Signed-off-by: chrismark <chrismarkou92@gmail.com>
Signed-off-by: chrismark <chrismarkou92@gmail.com>
Signed-off-by: chrismark <chrismarkou92@gmail.com>
|
Pinging @elastic/integrations-platforms (Team:Platforms) |
Signed-off-by: chrismark <chrismarkou92@gmail.com>
Signed-off-by: chrismark <chrismarkou92@gmail.com>
There was a problem hiding this comment.
I think that a metricset that collects events per connection shouldn't be enabled by default.
With this change, we are modifying the behaviour of a default metricset from collecting a summary of metrics for all the connections, to collect one event per connection. I guess that a production nats server can have hundreds or thousands of connections at a given time, would this generate such number of events per fetch period?
I think that the summary, and the metrics per connection should be in different metricsets.
I see that we also collect the number of connections in the stats metricset (assuming that nats.stats.total_connections is the same thing).
Maybe a plan could be to:
- Leave the summarized metrics (the total count would be the only one at least by now) in the
statsmetricset. - Deprecate the
connectionsmetricset in 7.x/7.11, and remove it in master/8.0, the only metric it collects is also collected by thestatsmetricset. - Add a new, non-default
connectionmetricset, that collects the metrics per connection. Other reason to move fromconnectionstoconnectionis that singular namespaces is usually preferred in metric names (e.g:system.process...,postgresql.statement...).
For the routes we could follow a similar plan, to deprecate routes and create a new non-default route metricset, but we would first need to add the count of routes to the stats metricset if possible.
Signed-off-by: chrismark <chrismarkou92@gmail.com>
Signed-off-by: chrismark <chrismarkou92@gmail.com>
|
Added 2 new extra metricsets, disabled by default, to collect metrics per connection/route. After this we can deprecate |
Signed-off-by: chrismark <chrismarkou92@gmail.com>
Signed-off-by: chrismark <chrismarkou92@gmail.com>
Signed-off-by: chrismark <chrismarkou92@gmail.com>
| @@ -13,7 +13,3 @@ | |||
| type: keyword | |||
| description: > | |||
| The server ID | |||
| - name: server.time | |||
There was a problem hiding this comment.
We may want to keep this in existing modules to avoid breaking changes, though I don't think this field is going to be missed.
| if err != nil { | ||
| return errors.Wrap(err, "failure applying module schema") | ||
| } | ||
| timestamp, err := util.GetNatsTimestamp(moduleFields) | ||
| moduleFields.Delete("server.time") |
There was a problem hiding this comment.
We may want to keep this value to avoid a smallish breaking change.
Signed-off-by: chrismark <chrismarkou92@gmail.com>
(cherry picked from commit 70ff0a7)
What this PR adds
This PR adds more metrics at
connectionsandroutesmetricsets. Prior to this no metrics have been collected for each of the connections and for each of the routes. With this PR we collect metrics like in/out bytes etc for each the connections and for each of the routes. In order to have routes in a NATS server a more than 1 NATS servers should be presented creating a NATS cluster. For this docker-compose file of the module adds an extra NATS instance so as populate routes and their respective metrics.Why it is important
It related to elastic/integrations#359. Next steps would be to enhance/improve dashboards.
How to test this manually
docker-composefile of nats moduleconnectionsandroutesmetricsets and point them to the monitoring port of thenatscontainer (exported port of 8222 internal)