Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus daemon is not restarting when command-line arguments are changed #382

Closed
sharadgaur opened this issue Nov 13, 2019 · 2 comments · Fixed by #390
Closed

Prometheus daemon is not restarting when command-line arguments are changed #382

sharadgaur opened this issue Nov 13, 2019 · 2 comments · Fixed by #390
Labels
bug Something isn't working

Comments

@sharadgaur
Copy link

sharadgaur commented Nov 13, 2019

Adding or removing any command-line argument to Prometheus class does not trigger daemon restart.

class { 'prometheus':
    configname               => 'prometheus.yml',
    localstorage             => $storage_path,
    manage_user              => false,
    manage_group             => false,
    external_url             => "https://prometheus.test.fake.dns.org",
...
  }

In this case, external_url is a command-line argument to Prometheus

@alexjfisher
Copy link
Member

I think this is only an issue with systemd.

File {
notify => Class['prometheus::run_service'],
}
should be defaulting the notify parameter of all file resources in config.pp
The idea is that some file resources can then notify prometheus::service_reload instead.

Using resource defaults like this is not recommended. 'Area of effect' can be confusing and cause unintentional results. See https://puppet.com/docs/puppet/5.5/lang_defaults.html#behavior

As for systemd...

$systemd_notify = [Exec['prometheus-systemd-reload'], Class['prometheus::run_service']]
hasn't done anything for about 2 years. The variable is not used and the Exec referenced doesn't exist either.

@sharadgaur
Copy link
Author

sharadgaur commented Nov 18, 2019

Should I remove the

File { 
   notify => Class['prometheus::run_service'], 
 } 

from config.pp and make systemd config to use $systemd_notify variable? I really don't want to use Exec.
If you don't mind we can get rid of $system_notify

bastelfreak pushed a commit that referenced this issue Nov 18, 2019
In systemd, the service is currently not being restarted after
configuration changes.  See GH-382

The acceptance test in this commit reconfigures prometheus with
the admin API enabled and then attempts to access it.

See https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-admin-apis
bastelfreak pushed a commit that referenced this issue Nov 18, 2019
The [unreliable](https://puppet.com/docs/puppet/5.5/lang_defaults.html#behavior) resource default
```
File{
  notify => Class['prometheus::run_service']
}
```
is replaced by a `$notify` variable that is set on the relevant file
resources *and* `systemd::unit_file`.  Some care was needed to make sure
the reload behaviour wasn't broken.  ie If the configuration change is
just a new scrape job that is collected, the service should only be
reloaded, not restarted.

Fixes #382
cegeka-jenkins pushed a commit to cegeka/puppet-prometheus that referenced this issue Jan 21, 2021
In systemd, the service is currently not being restarted after
configuration changes.  See voxpupuliGH-382

The acceptance test in this commit reconfigures prometheus with
the admin API enabled and then attempts to access it.

See https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-admin-apis
cegeka-jenkins pushed a commit to cegeka/puppet-prometheus that referenced this issue Jan 21, 2021
The [unreliable](https://puppet.com/docs/puppet/5.5/lang_defaults.html#behavior) resource default
```
File{
  notify => Class['prometheus::run_service']
}
```
is replaced by a `$notify` variable that is set on the relevant file
resources *and* `systemd::unit_file`.  Some care was needed to make sure
the reload behaviour wasn't broken.  ie If the configuration change is
just a new scrape job that is collected, the service should only be
reloaded, not restarted.

Fixes voxpupuli#382
Rovanion pushed a commit to Rovanion/puppet-prometheus that referenced this issue May 5, 2021
In systemd, the service is currently not being restarted after
configuration changes.  See voxpupuliGH-382

The acceptance test in this commit reconfigures prometheus with
the admin API enabled and then attempts to access it.

See https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-admin-apis
Rovanion pushed a commit to Rovanion/puppet-prometheus that referenced this issue May 5, 2021
The [unreliable](https://puppet.com/docs/puppet/5.5/lang_defaults.html#behavior) resource default
```
File{
  notify => Class['prometheus::run_service']
}
```
is replaced by a `$notify` variable that is set on the relevant file
resources *and* `systemd::unit_file`.  Some care was needed to make sure
the reload behaviour wasn't broken.  ie If the configuration change is
just a new scrape job that is collected, the service should only be
reloaded, not restarted.

Fixes voxpupuli#382
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants