Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bugfix: allow perf_schema.memory summary current_bytes to be negative #517

Merged
merged 1 commit into from
Nov 22, 2020
Merged

bugfix: allow perf_schema.memory summary current_bytes to be negative #517

merged 1 commit into from
Nov 22, 2020

Conversation

dafydd-t
Copy link
Contributor

@dafydd-t dafydd-t commented Nov 19, 2020 •
edited
Loading

Related to #75 and PR #515

It is possible to end up with a negative value for CURRENT_NUMBER_OF_BYTES_USED - this can happen if you enabled instrumentation for performance_schema memory at some point after startup time.

Currently, if this is the case, the collector emits the following error when scraped, and stops scraping rows after the first row with a negative value for this column:

caller=exporter.go:174 msg="Error from scraper" scraper=perf_schema.memory_events err="sql: Scan error on column index 3, name \"CURRENT_NUMBER_OF_BYTES_USED\": converting driver.Value type []uint8 (\"-3214848\") to a uint64: invalid syntax"

This change makes the currentBytes variable signed, so it can accept negative values and should allow the collector to scrape all the rows of the query results.

the memory_events_used_bytes is already a gauge, so it can be negative.

@dafydd-t
allow perf_schema.memory summary current_bytes to be negative
0d28578 
Signed-off-by: Dafydd <dafydd.thomas@grafana.com>
@dafydd-t dafydd-t force-pushed the fix-perf-memory-type branch from ac3bbd4 to 0d28578 Compare November 19, 2020 18:09
@dafydd-t
Copy link
Contributor Author

@SuperQ I'm pinging you as you were really helpful with the initial PR for this feature

@SuperQ
Copy link
Member

SuperQ commented Nov 22, 2020

Yup, it looks like the columns is a bigint(20).

CREATE TABLE `memory_summary_global_by_event_name` (
  `EVENT_NAME` varchar(128) NOT NULL,
  `COUNT_ALLOC` bigint(20) unsigned NOT NULL,
  `COUNT_FREE` bigint(20) unsigned NOT NULL,
  `SUM_NUMBER_OF_BYTES_ALLOC` bigint(20) unsigned NOT NULL,
  `SUM_NUMBER_OF_BYTES_FREE` bigint(20) unsigned NOT NULL,
  `LOW_COUNT_USED` bigint(20) NOT NULL,
  `CURRENT_COUNT_USED` bigint(20) NOT NULL,
  `HIGH_COUNT_USED` bigint(20) NOT NULL,
  `LOW_NUMBER_OF_BYTES_USED` bigint(20) NOT NULL,
  `CURRENT_NUMBER_OF_BYTES_USED` bigint(20) NOT NULL,
  `HIGH_NUMBER_OF_BYTES_USED` bigint(20) NOT NULL
) ENGINE=PERFORMANCE_SCHEMA DEFAULT CHARSET=utf8

SuperQ
SuperQ approved these changes Nov 22, 2020
View reviewed changes
Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@SuperQ SuperQ merged commit 0a444d1 into prometheus:master Nov 22, 2020
@dafydd-t dafydd-t deleted the fix-perf-memory-type branch November 23, 2020 08:20
SuperQ added a commit that referenced this pull request Apr 23, 2021
@SuperQ
Release 0.13.0-rc.0
b281883 
Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

* [CHANGE] Switch to go-kit for logs. #433
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517
* [ENHANCEMENT] Support heartbeats in UTC #471
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539

Signed-off-by: Ben Kochie <superq@gmail.com>
@SuperQ SuperQ mentioned this pull request Apr 23, 2021
Merged
SuperQ added a commit that referenced this pull request Apr 25, 2021
@SuperQ
Release 0.13.0-rc.0
2517b84 
BREAKING CHANGES:

Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

Changes:

* [CHANGE] Switch to go-kit for logs. #433
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539
* [ENHANCEMENT] Support heartbeats in UTC #471
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517

Signed-off-by: Ben Kochie <superq@gmail.com>
SuperQ added a commit that referenced this pull request Apr 25, 2021
@SuperQ
Release 0.13.0-rc.0 (#546)
0dde132 
BREAKING CHANGES:

Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

Changes:

* [CHANGE] Switch to go-kit for logs. #433
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539
* [ENHANCEMENT] Support heartbeats in UTC #471
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517

Signed-off-by: Ben Kochie <superq@gmail.com>
SuperQ added a commit that referenced this pull request May 18, 2021
@SuperQ
Release 0.13.0
693f62f 
BREAKING CHANGES:

Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

Changes:

* [CHANGE] Switch to go-kit for logs. #433
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539
* [FEATURE] Add TLS and basic authentication #522
* [ENHANCEMENT] Support heartbeats in UTC #471
* [ENHANCEMENT] Improve parsing of boolean strings #548
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517

Signed-off-by: Ben Kochie <superq@gmail.com>
@SuperQ SuperQ mentioned this pull request May 18, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SuperQ SuperQ SuperQ approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dafydd-t @SuperQ