Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgres plugin: option to remove password in tag #486

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
5f3a1fc
Updating system plugins for 0.3.0
sparrc Dec 11, 2015
16932ee
Updating aerospike & apache plugins for 0.3.0
sparrc Dec 14, 2015
d77bdb2
Breakout JSON flattening into internal package, exec & elasticsearch …
sparrc Dec 14, 2015
88631d4
0.3.0: HAProxy
sparrc Dec 14, 2015
deb059e
0.3.0 httpjson
sparrc Dec 14, 2015
c04f37b
0.3.0: leofs & lustre2
sparrc Dec 14, 2015
7a32fe0
0.3.0: mailchimp & memcached
sparrc Dec 14, 2015
a148c9d
0.3.0: ping, mysql, nginx
sparrc Dec 15, 2015
6539090
0.3.0: procstat
sparrc Dec 15, 2015
68c42d4
0.3.0: prometheus & puppetagent
sparrc Dec 15, 2015
3b61bee
0.3.0 redis & rabbitmq
sparrc Dec 15, 2015
931c706
0.3.0: trig and twemproxy
sparrc Dec 15, 2015
c3c6292
backwards compatability for io->diskio change
sparrc Dec 15, 2015
847d2f0
0.3.0: zookeeper and zfs
sparrc Dec 15, 2015
d3c6d87
0.3.0: rethinkdb
sparrc Dec 15, 2015
89fd51e
0.3.0 HAProxy rebase
sparrc Dec 18, 2015
37eeea5
0.3.0: postgresql and phpfpm
sparrc Dec 19, 2015
6bb34fc
0.3.0: mongodb and jolokia
sparrc Dec 19, 2015
963729a
0.3.0 output: datadog and amon
sparrc Dec 19, 2015
bdbba81
0.3.0 output: librato
sparrc Dec 19, 2015
11254e8
0.3.0 outputs: opentsdb
sparrc Dec 19, 2015
06a7851
CHANGELOG update
sparrc Dec 19, 2015
9a5ced8
0.3.0 outputs: riemann
sparrc Dec 19, 2015
2989227
0.3.0 Removing internal parallelism: httpjson and exec
sparrc Dec 19, 2015
a8c1644
0.3.0 Removing internal parallelism: postgresql
sparrc Dec 19, 2015
148b93b
0.3.0 Removing internal parallelism: procstat
sparrc Dec 19, 2015
2800755
0.3.0 Removing internal parallelism: twemproxy and rabbitmq
sparrc Dec 20, 2015
a532d91
Fix httpjson panic for nil request body
sparrc Dec 21, 2015
68c4d1c
0.3.0 unit tests: system plugins
sparrc Jan 5, 2016
cb975ca
0.3.0 unit tests: aerospike, apache, bcache
sparrc Jan 6, 2016
8e38f59
0.3.0 unit tests: disque and elasticsearch
sparrc Jan 6, 2016
4970e2a
0.3.0 unit tests: existing postgresql
Jan 7, 2016
ab00925
Postgres plugin: option to remove password in tag
Jan 6, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,25 @@
## v0.3.0 [unreleased]

### Release Notes
- **breaking change** the `io` plugin has been renamed `diskio`
- **breaking change** Plugin measurements aggregated into a single measurement.
- **breaking change** `jolokia` plugin: must use global tag/drop/pass parameters
for configuration.
- `twemproxy` plugin: `prefix` option removed.
- `procstat` cpu measurements are now prepended with `cpu_time_` instead of
only `cpu_`
- The prometheus plugin schema has not been changed (measurements have not been
aggregated).

### Features
- Plugin measurements aggregated into a single measurement.
- Added ability to specify per-plugin tags
- Added ability to specify per-plugin measurement suffix and prefix.
(`name_prefix` and `name_suffix`)
- Added ability to override base plugin name. (`name_override`)

### Bugfixes

## v0.2.5 [unreleased]

### Features
Expand Down
177 changes: 177 additions & 0 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
# Telegraf Configuration

## Plugin Configuration

There are some configuration options that are configurable per plugin:

* **name_override**: Override the base name of the measurement.
(Default is the name of the plugin).
* **name_prefix**: Specifies a prefix to attach to the measurement name.
* **name_suffix**: Specifies a suffix to attach to the measurement name.
* **tags**: A map of tags to apply to a specific plugin's measurements.

### Plugin Filters

There are also filters that can be configured per plugin:

* **pass**: An array of strings that is used to filter metrics generated by the
current plugin. Each string in the array is tested as a glob match against field names
and if it matches, the field is emitted.
* **drop**: The inverse of pass, if a field name matches, it is not emitted.
* **tagpass**: tag names and arrays of strings that are used to filter
measurements by the current plugin. Each string in the array is tested as a glob
match against the tag name, and if it matches the measurement is emitted.
* **tagdrop**: The inverse of tagpass. If a tag matches, the measurement is not emitted.
This is tested on measurements that have passed the tagpass test.
* **interval**: How often to gather this metric. Normal plugins use a single
global interval, but if one particular plugin should be run less or more often,
you can configure that here.

### Plugin Configuration Examples

This is a full working config that will output CPU data to an InfluxDB instance
at 192.168.59.103:8086, tagging measurements with dc="denver-1". It will output
measurements at a 10s interval and will collect per-cpu data, dropping any
fields which begin with `time_`.

```toml
[tags]
dc = "denver-1"

[agent]
interval = "10s"

# OUTPUTS
[outputs]
[[outputs.influxdb]]
url = "http://192.168.59.103:8086" # required.
database = "telegraf" # required.
precision = "s"

# PLUGINS
[plugins]
[[plugins.cpu]]
percpu = true
totalcpu = false
# filter all fields beginning with 'time_'
drop = ["time_*"]
```

### Plugin Config: tagpass and tagdrop

```toml
[plugins]
[[plugins.cpu]]
percpu = true
totalcpu = false
drop = ["cpu_time"]
# Don't collect CPU data for cpu6 & cpu7
[plugins.cpu.tagdrop]
cpu = [ "cpu6", "cpu7" ]

[[plugins.disk]]
[plugins.disk.tagpass]
# tagpass conditions are OR, not AND.
# If the (filesystem is ext4 or xfs) OR (the path is /opt or /home)
# then the metric passes
fstype = [ "ext4", "xfs" ]
# Globs can also be used on the tag values
path = [ "/opt", "/home*" ]
```

### Plugin Config: pass and drop

```toml
# Drop all metrics for guest & steal CPU usage
[[plugins.cpu]]
percpu = false
totalcpu = true
drop = ["usage_guest", "usage_steal"]

# Only store inode related metrics for disks
[[plugins.disk]]
pass = ["inodes*"]
```

### Plugin config: prefix, suffix, and override

This plugin will emit measurements with the name `cpu_total`

```toml
[[plugins.cpu]]
name_suffix = "_total"
percpu = false
totalcpu = true
```

This will emit measurements with the name `foobar`

```toml
[[plugins.cpu]]
name_override = "foobar"
percpu = false
totalcpu = true
```

### Plugin config: tags

This plugin will emit measurements with two additional tags: `tag1=foo` and
`tag2=bar`

```toml
[[plugins.cpu]]
percpu = false
totalcpu = true
[plugins.cpu.tags]
tag1 = "foo"
tag2 = "bar"
```

### Multiple plugins of the same type

Additional plugins (or outputs) of the same type can be specified,
just define more instances in the config file:

```toml
[[plugins.cpu]]
percpu = false
totalcpu = true

[[plugins.cpu]]
percpu = true
totalcpu = false
drop = ["cpu_time*"]
```

## Output Configuration

Telegraf also supports specifying multiple output sinks to send data to,
configuring each output sink is different, but examples can be
found by running `telegraf -sample-config`.

Outputs also support the same configurable options as plugins
(pass, drop, tagpass, tagdrop), added in 0.2.4

```toml
[[outputs.influxdb]]
urls = [ "http://localhost:8086" ]
database = "telegraf"
precision = "s"
# Drop all measurements that start with "aerospike"
drop = ["aerospike*"]

[[outputs.influxdb]]
urls = [ "http://localhost:8086" ]
database = "telegraf-aerospike-data"
precision = "s"
# Only accept aerospike data:
pass = ["aerospike*"]

[[outputs.influxdb]]
urls = [ "http://localhost:8086" ]
database = "telegraf-cpu0-data"
precision = "s"
# Only store measurements where the tag "cpu" matches the value "cpu0"
[outputs.influxdb.tagpass]
cpu = ["cpu0"]
```
133 changes: 5 additions & 128 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,99 +116,10 @@ unit parser, e.g. "10s" for 10 seconds or "5m" for 5 minutes.
* **debug**: Set to true to gather and send metrics to STDOUT as well as
InfluxDB.

## Plugin Options

There are 5 configuration options that are configurable per plugin:

* **pass**: An array of strings that is used to filter metrics generated by the
current plugin. Each string in the array is tested as a glob match against metric names
and if it matches, the metric is emitted.
* **drop**: The inverse of pass, if a metric name matches, it is not emitted.
* **tagpass**: tag names and arrays of strings that are used to filter metrics by the current plugin. Each string in the array is tested as a glob match against
the tag name, and if it matches the metric is emitted.
* **tagdrop**: The inverse of tagpass. If a tag matches, the metric is not emitted.
This is tested on metrics that have passed the tagpass test.
* **interval**: How often to gather this metric. Normal plugins use a single
global interval, but if one particular plugin should be run less or more often,
you can configure that here.

### Plugin Configuration Examples

This is a full working config that will output CPU data to an InfluxDB instance
at 192.168.59.103:8086, tagging measurements with dc="denver-1". It will output
measurements at a 10s interval and will collect per-cpu data, dropping any
measurements which begin with `cpu_time`.

```toml
[tags]
dc = "denver-1"

[agent]
interval = "10s"

# OUTPUTS
[outputs]
[[outputs.influxdb]]
url = "http://192.168.59.103:8086" # required.
database = "telegraf" # required.
precision = "s"

# PLUGINS
[plugins]
[[plugins.cpu]]
percpu = true
totalcpu = false
drop = ["cpu_time*"]
```

Below is how to configure `tagpass` and `tagdrop` parameters

```toml
[plugins]
[[plugins.cpu]]
percpu = true
totalcpu = false
drop = ["cpu_time"]
# Don't collect CPU data for cpu6 & cpu7
[plugins.cpu.tagdrop]
cpu = [ "cpu6", "cpu7" ]

[[plugins.disk]]
[plugins.disk.tagpass]
# tagpass conditions are OR, not AND.
# If the (filesystem is ext4 or xfs) OR (the path is /opt or /home)
# then the metric passes
fstype = [ "ext4", "xfs" ]
# Globs can also be used on the tag values
path = [ "/opt", "/home*" ]
```

Below is how to configure `pass` and `drop` parameters

```toml
# Drop all metrics for guest CPU usage
[[plugins.cpu]]
drop = [ "cpu_usage_guest" ]

# Only store inode related metrics for disks
[[plugins.disk]]
pass = [ "disk_inodes*" ]
```
## Configuration


Additional plugins (or outputs) of the same type can be specified,
just define more instances in the config file:

```toml
[[plugins.cpu]]
percpu = false
totalcpu = true

[[plugins.cpu]]
percpu = true
totalcpu = false
drop = ["cpu_time*"]
```
See the [configuration guide](CONFIGURATION.md) for a rundown of the more advanced
configuration options.

## Supported Plugins

Expand All @@ -226,7 +137,7 @@ Telegraf currently has support for collecting metrics from:
* haproxy
* httpjson (generic JSON-emitting http service plugin)
* influxdb
* jolokia (remote JMX with JSON over HTTP)
* jolokia
* leofs
* lustre2
* mailchimp
Expand All @@ -249,10 +160,10 @@ Telegraf currently has support for collecting metrics from:
* system
* cpu
* mem
* io
* net
* netstat
* disk
* diskio
* swap

## Supported Service Plugins
Expand All @@ -265,40 +176,6 @@ Telegraf can collect metrics via the following services:
We'll be adding support for many more over the coming months. Read on if you
want to add support for another service or third-party API.

## Output options

Telegraf also supports specifying multiple output sinks to send data to,
configuring each output sink is different, but examples can be
found by running `telegraf -sample-config`.

Outputs also support the same configurable options as plugins
(pass, drop, tagpass, tagdrop), added in 0.2.4

```toml
[[outputs.influxdb]]
urls = [ "http://localhost:8086" ]
database = "telegraf"
precision = "s"
# Drop all measurements that start with "aerospike"
drop = ["aerospike*"]

[[outputs.influxdb]]
urls = [ "http://localhost:8086" ]
database = "telegraf-aerospike-data"
precision = "s"
# Only accept aerospike data:
pass = ["aerospike*"]

[[outputs.influxdb]]
urls = [ "http://localhost:8086" ]
database = "telegraf-cpu0-data"
precision = "s"
# Only store measurements where the tag "cpu" matches the value "cpu0"
[outputs.influxdb.tagpass]
cpu = ["cpu0"]
```


## Supported Outputs

* influxdb
Expand Down
Loading