Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing parameters in postgresql plugin config #157

Merged
merged 2 commits into from
May 24, 2022

Conversation

jengsala
Copy link
Contributor

@jengsala jengsala commented Apr 15, 2022

This PR will allow to added missing parameters in postgresql plugin config

Reference : https://collectd.org/wiki/index.php/Plugin:PostgreSQL

#
# DO NOT EDIT
#
# This file is managed by salt via salt://collectd/files/postgresql.conf
# Modify the config that generates this file instead
#

LoadPlugin postgresql

<Plugin "postgresql">
  <Query rt36_tickets>
    Statement "mystatement"
    Param "hostname"  
    <Result>
      Type "counter"
      InstancePrefix "rt36_tickets"
      InstancesFrom "type"
      ValuesFrom "count"
    </Result>
  </Query>
  <Query magic>
    Statement "mystatement"  
    <Result>
      Type "counter"
      InstancePrefix "rt36_tickets"
      InstancesFrom "type"
    </Result>
  </Query>
  <Database foo>
    Host "hostname"
    Port "5432"
    User "username"
    Password "secret"
    SSLMode "prefer"
    KRBSrvName "kerberos_service_name"
    Service "service_name"
    Interval "3600"
    Query rt36_tickets
    Query magic
  </Database>

@jengsala jengsala changed the title feat(postgresql): postgresqlplugin feat(postgresql): Custom postgresql plugin Apr 15, 2022
@jengsala jengsala changed the title feat(postgresql): Custom postgresql plugin Custom postgresql plugin for postgresql.conf Apr 15, 2022
@jengsala jengsala changed the title Custom postgresql plugin for postgresql.conf Custom postgresql plugin Apr 15, 2022
@jengsala jengsala force-pushed the DS-IT-CustomPGSQL branch from 040d6ca to d9b93ba Compare April 19, 2022 13:42
@jengsala jengsala changed the title Custom postgresql plugin WIP : Custom postgresql plugin Apr 20, 2022
@jengsala jengsala changed the title WIP : Custom postgresql plugin Generate a custom configuration postgresql plugin Apr 20, 2022
jengsala pushed a commit to jengsala/collectd-formula that referenced this pull request Apr 20, 2022
@jengsala jengsala force-pushed the DS-IT-CustomPGSQL branch from f07e8f7 to f62b8e7 Compare April 20, 2022 14:49
@jengsala jengsala changed the title Generate a custom configuration postgresql plugin Generate a complete configuration postgresql plugin Apr 20, 2022
@jengsala jengsala changed the title Generate a complete configuration postgresql plugin Add missing parameters in postgresql plugin config Apr 20, 2022
@jengsala jengsala force-pushed the DS-IT-CustomPGSQL branch from 9769baf to 6373377 Compare April 26, 2022 14:01
@jengsala jengsala requested a review from a team as a code owner April 29, 2022 13:57
@jengsala jengsala force-pushed the DS-IT-CustomPGSQL branch 2 times, most recently from 6b05f44 to 08b3951 Compare April 29, 2022 14:55
Copy link
Member

@myii myii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your PR, @dsala-it. I've got a review for you, with suggested changes inline.

collectd/files/postgresql.conf Outdated Show resolved Hide resolved
pillar.example Outdated Show resolved Hide resolved
kitchen.yml Outdated Show resolved Hide resolved
test/salt/default/pillar/collectd.sls Outdated Show resolved Hide resolved
Copy link
Member

@daks daks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments about code. Tests are also mostly failing and needs to be fixed.

collectd/files/postgresql.conf Outdated Show resolved Hide resolved
collectd/files/postgresql.conf Outdated Show resolved Hide resolved
collectd/files/postgresql.conf Outdated Show resolved Hide resolved
test/integration/default/controls/service.rb Outdated Show resolved Hide resolved
@jengsala jengsala force-pushed the DS-IT-CustomPGSQL branch 3 times, most recently from 828074b to 7e06c7d Compare May 20, 2022 08:14
Copy link
Member

@myii myii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dsala-it Not a complete review, because I'm short on time at the moment. But a couple of issues to be addressed.

test/integration/default/controls/service.rb Outdated Show resolved Hide resolved
test/salt/states/collectd-deps.sls Outdated Show resolved Hide resolved
Copy link
Member

@daks daks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's OK for me, except for platform ubuntu which has a failing converge https://gitlab.com/saltstack-formulas/collectd-formula/-/jobs/2484292490

test/salt/states/collectd-deps.sls Outdated Show resolved Hide resolved
@jengsala jengsala force-pushed the DS-IT-CustomPGSQL branch from da69cff to 8a796a7 Compare May 20, 2022 09:51
Copy link
Member

@myii myii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dsala-it We're on the final stretch now, just tidying up. Rather than reviewing multiple blocks, I've prepared a commit that contains the rest of the requested changes:

This is the CI run for that:

You can grab that commit from my fork and use it here; if that is difficult, I should be able to force-push it myself. Note, that I've updated the commit message as well, to something more appropriate for the changelog.

I've also mentioned one last change inline (below), that I didn't get around to adding to the commit.


For clarity, this is the overall diff, including the bit below:

diff --git a/collectd/map.jinja b/collectd/map.jinja
index 11d930c..9eb9fae 100644
--- a/collectd/map.jinja
+++ b/collectd/map.jinja
@@ -9,7 +9,7 @@
         'moduledirconfig': '/usr/share/collectd/modules',
         'user': 'root',
         'group': 'root',
-        'required_dependencies': {'pkgs': ['postgresql-client']},
+        'plugin_postgresql': {'pkg': 'postgresql-client'},
     },
     'RedHat': {
         'config': '/etc/collectd.conf',
@@ -20,7 +20,7 @@
         'moduledirconfig': '/usr/share/collectd/modules',
         'user': 'root',
         'group': 'root',
-        'required_dependencies': {'pkgs': ['collectd-postgresql']},
+        'plugin_postgresql': {'pkg': 'collectd-postgresql'},
     },
     'FreeBSD': {
         'config': '/usr/local/etc/collectd.conf',
@@ -31,7 +31,7 @@
         'moduledirconfig': '/usr/local/share/collectd/modules',
         'user': 'root',
         'group': 'wheel',
-        'required_dependencies': {'pkgs': ['postgresql-client']},
+        'plugin_postgresql': {'pkg': 'postgresql-client'},
     },
     'Suse': {
         'config': '/etc/collectd.conf',
@@ -42,7 +42,7 @@
         'moduledirconfig': '/usr/share/collectd/modules',
         'user': 'root',
         'group': 'root',
-        'required_dependencies': {'pkgs': ['collectd-plugin-postgresql']},
+        'plugin_postgresql': {'pkg': 'collectd-plugin-postgresql'},
     },
 }, merge=salt['pillar.get']('collectd:lookup')) %}
 
diff --git a/collectd/postgresql.sls b/collectd/postgresql.sls
index 5a9ec3c..bdfec0e 100644
--- a/collectd/postgresql.sls
+++ b/collectd/postgresql.sls
@@ -1,16 +1,12 @@
 {%- from "collectd/map.jinja" import collectd_settings with context %}
-{%- set postgresql_settings = collectd_settings.get('plugins:postgresql') %}
-
-install_required_dependencies:
-  pkg.installed:
-    - pkgs:
-{% for _pkg in collectd_settings.required_dependencies.pkgs %}
-      - {{ _pkg }}
-{% endfor %}
 
 include:
   - collectd
 
+collectd-postgresql-pkg-installed:
+  pkg.installed:
+    - name: {{ collectd_settings.plugin_postgresql.pkg }}
+
 {{ collectd_settings.plugindirconfig }}/postgresql.conf:
   file.managed:
     - source: salt://collectd/files/postgresql.conf
@@ -21,5 +17,4 @@ include:
     - watch_in:
       - service: collectd-service
     - require:
-      - pkg: install_required_dependencies
-
+      - pkg: collectd-postgresql-pkg-installed
diff --git a/kitchen.yml b/kitchen.yml
index e2d932a..dd10a3a 100644
--- a/kitchen.yml
+++ b/kitchen.yml
@@ -296,9 +296,9 @@ suites:
       state_top:
         base:
           '*':
-            - collectd.postgresql
             - collectd._mapdata
             - collectd
+            - collectd.postgresql
       pillars:
         top.sls:
           base:

Partially reviewed using myii/ssf-formula#446.

kitchen.yml Outdated Show resolved Hide resolved
myii added a commit to myii/ssf-formula that referenced this pull request May 20, 2022
@jengsala jengsala force-pushed the DS-IT-CustomPGSQL branch from 1f4a88d to 2a56b64 Compare May 23, 2022 07:49
@jengsala jengsala force-pushed the DS-IT-CustomPGSQL branch from 9feb6b5 to 82d7104 Compare May 23, 2022 07:59
@jengsala
Copy link
Contributor Author

I redid a little cleaning and a rebase/squash commits. @myii I let you see for one last time and merge if everything is okay

@jengsala
Copy link
Contributor Author

About the PR https://gitlab.com/saltstack-formulas/collectd-formula/-/pipelines/545480720, The converge does not pass in Ubuntu-22* jobs/2491768749.

The problem is that the collectd-core package used here in map.jinja#L5 is not yet available.

See https://pkgs.org/search/?q=collectd

I don't see any other way to solve this problem. Should we ignore it or not?

Otherwise, on the last commit I fixed the linter errors. I let you approve if everything is okay for you ...

@myii myii force-pushed the DS-IT-CustomPGSQL branch from afbf536 to 31fa9fe Compare May 24, 2022 12:09
@myii myii merged commit 9ec2d36 into saltstack-formulas:master May 24, 2022
@myii
Copy link
Member

myii commented May 24, 2022

Merged, thanks @dsala-it. I adjusted the last commit (31fa9fe) in order to fix all the linter issues that we discussed on Slack:

@saltstack-formulas-travis

🎉 This PR is included in version 1.2.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@myii
Copy link
Member

myii commented May 24, 2022

@daks Thanks for the reviews!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants