Use Puppet 4 as provisioner #56

dnsmichi · 2017-04-22T16:23:24Z

Module Analysis

General:

Name	Puppet Version	Path	Url
puppetlabs-stdlib	>= 2.7.20	modules/stdlib	https://github.com/puppetlabs/puppetlabs-stdlib.git
puppetlabs-concat	>= 4.7.0	modules/concat	https://github.com/puppetlabs/puppetlabs-concat.git
puppetlabs-apache	>= 4.7.0	modules/apache	https://github.com/puppetlabs/puppetlabs-apache.git
puppetlabs-mysql	>= 4.7.0	modules/mysql	https://github.com/puppetlabs/puppetlabs-mysql.git
puppetlabs-postgresql	>= 4.7.0	modules/postgresql	https://github.com/puppetlabs/puppetlabs-postgresql.git
puppetlabs-vcsrepo	>= 4.7.0	modules/vcsrepo	https://github.com/puppetlabs/puppetlabs-vcsrepo.git
puppet-module-epel	>= 3.0.0	modules/epel	https://github.com/stahnma/puppet-module-epel.git
puppet-php	>= 4.7.0<5.0.0	modules/php	https://github.com/voxpupuli/puppet-php.git
puppet-selinux	>= 4.7.1	modules/selinux	https://github.com/voxpupuli/puppet-selinux.git
puppetlabs-java	>= 4.7.0	modules/java	https://github.com/puppetlabs/puppetlabs-java.git
puppet-yum	>= 4.6.1	modules/yum	https://github.com/voxpupuli/puppet-yum
puppet-archive	>= 4.7.1	modules/archive	https://github.com/voxpupuli/puppet-archive.git
puppet-vim	>=4.0.0<5.0.0	modules/vim	https://github.com/saz/puppet-vim.git
puppet-datacat	Type for ES	modules/datacat	https://github.com/richardc/puppet-datacat.git
puppet-inifile	>= 4.7.0	modules/inifile	https://github.com/puppetlabs/puppetlabs-inifile.git

Specific projects:

Name	Puppet Version	Path	Url
puppet-elastic-stack	>= 4.6.1	modules/elastic_stack	repo management in v6: https://github.com/elastic/puppet-elastic-stack
puppet-icinga2	4.x	modules/icinga2	https://github.com/icinga/puppet-icinga2.git
puppet-icingaweb2	>= 4.7.0	modules/icingaweb2	https://github.com/icinga/puppet-icingaweb2.git
puppet-graylog	???	modules/graylog	[ ] NEEDS v4 SUPPORT https://github.com/Graylog2/puppet-graylog.git
puppet-elasticsearch	>= 4.5.0	modules/elasticsearch	https://github.com/elasticsearch/puppet-elasticsearch.git
puppet-nginx	>= 4.7.0	modules/nginx	https://github.com/voxpupuli/puppet-nginx.git
puppet-logstash	>= 4.6.1	modules/logstash	https://github.com/elastic/puppet-logstash.git
puppet-kibana	>= 4.5.0	modules/kibana	https://github.com/elastic/puppet-kibana.git
puppet-filebeat	>= 4.0.0	modules/filebeat	https://github.com/pcfens/puppet-filebeat.git
puppetlabs-mongodb	>= 4.7.1	modules/mongodb	https://github.com/puppetlabs/puppetlabs-mongodb.git
golja-influxdb	>= 3.0.0<5.0.0	modules/influxdb	https://github.com/n1tr0g/golja-influxdb.git
puppet-graphite	>= 3.0.0<5.0.0	modules/graphite	[ ] NEEDS TESTS ON UPDATE https://github.com/echocat/puppet-graphite.git
puppet-grafana	>= 4.7.1	modules/grafana	https://github.com/voxpupuli/puppet-grafana

Tasks

Provisioner Pre Script

Inspired by https://github.com/roman-mueller/puppet4-sandbox/blob/master/puppetupgrade.sh

if [ ! -e /etc/yum.repos.d/puppetlabs-pc1.repo ]; then
    echo "Installing Puppet 4 release..."
    yum install -y https://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
fi

if ! rpm -q "puppet-agent" &>/dev/null; then
    echo "Installing puppet..."
    yum install -y puppet-agent
fi

[root@icinga2-puppet4 ~]# puppet -V
4.10.9

Modules

Each module requires an update to the latest stable/git master.

Functional tests

Unclear:

puppet-icinga2 -> sent patch for elasticsearch feature in Add elasticsearch feature voxpupuli/puppet-icinga2#418
puppet-icingaweb2 -> sent patch for changing admin user to icingaadmin in Rename default administrative user to 'icingaadmin' voxpupuli/puppet-icingaweb2#194
puppet-graylog

Boxes

icinga2x (graphite, grafana,dashing,map)
icinga2x-elastic (elasticsearch, kibana, httpproxy)
icinga2x-influxdb (influxdb, grafana)
icinga2x-graylog (graylog)

The text was updated successfully, but these errors were encountered:

dnsmichi · 2017-09-12T08:48:50Z

Guess I will edit this to Puppet 6 again once I finally find the time to do it ;-)

dnsmichi · 2017-11-27T17:21:15Z

Updated the tasks above.

dnsmichi · 2017-11-28T13:36:04Z

Migrate Hiera 3 to 5: https://puppet.com/docs/puppet/4.10/hiera_migrate_v3_yaml.html

dnsmichi · 2017-11-28T13:56:45Z

hiera.yaml

---
version: 5

hierarchy:
  - name: "common"
    path: "common"

defaults:
  datadir: '/vagrant/hiera'
  data_hash: yaml_data

dnsmichi · 2017-11-28T14:33:24Z

Note: puppet-icinga2 & puppet-icingaweb2 is a secondary step, as this involves refactoring of the entire configuration deployment.

dnsmichi · 2017-11-28T14:34:30Z

==> icinga2-puppet4: Warning: This method is deprecated, please use the stdlib validate_legacy function,
==> icinga2-puppet4:                     with Stdlib::Compat::String. There is further documentation for validate_legacy function in the README. at ["/tmp/vagrant-puppet/modules-d5ac0b732b2eabd7898a629229f5ee69/icinga_rpm/manifests/init.pp", 30]:
==> icinga2-puppet4:    (at /tmp/vagrant-puppet/modules-d5ac0b732b2eabd7898a629229f5ee69/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
==> icinga2-puppet4: Warning: This method is deprecated, please use the stdlib validate_legacy function,
==> icinga2-puppet4:                     with Pattern[]. There is further documentation for validate_legacy function in the README. at ["/tmp/vagrant-puppet/modules-d5ac0b732b2eabd7898a629229f5ee69/icinga_rpm/manifests/init.pp", 33]:
==> icinga2-puppet4:    (at /tmp/vagrant-puppet/modules-d5ac0b732b2eabd7898a629229f5ee69/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation')
==> icinga2-puppet4: Warning: ModuleLoader: module 'mysql' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
==> icinga2-puppet4:    (file & line not available)

dnsmichi · 2017-11-28T14:38:03Z

Working branch: feature/puppet-4

dnsmichi · 2017-11-28T14:42:34Z

puppetlabs/mysql lists puppetlabs/staging as dependency while this already encourages to use voxpupuli/archive: https://github.com/voxpupuli/puppet-archive#migrating-from-puppet-staging

dnsmichi · 2017-11-28T15:15:49Z

Mysql, Apache work standalone.
Icinga 2 with the old foo interestingly works too.

dnsmichi · 2017-11-28T15:16:34Z

michi@mbmif ~/coding/icinga/icinga-vagrant/puppet4 (feature/puppet-4 *>) $ curl -k -s -u root:icinga https://192.168.33.50:5665/v1
<html><head><title>Icinga 2</title></head><h1>Hello from Icinga 2 (Version: v2.8.0-35-g6b39319)!</h1><p>You are authenticated as <b>root</b>. Your user has the following permissions:</p> <ul><li>*</li></ul><p>More information about API requests is available in the <a href="https://docs.icinga.com/icinga2/latest" target="_blank">documentation</a>.</p></html>

dnsmichi · 2017-11-28T15:48:44Z

Dashing works.

dnsmichi · 2017-11-28T17:34:25Z

influxdb, grafana works.

dnsmichi · 2017-11-29T10:34:05Z

graphite, grafana works.

dnsmichi · 2017-11-29T15:06:21Z

Elasticsearch 5, Kibana 5, HTTP Proxy works.

dnsmichi · 2017-11-29T15:35:07Z

The custom foo with icingaweb2 does not work, and I am not willing to fix it. Going the route to puppet-icingaweb2 directly.

dnsmichi · 2017-11-29T18:11:42Z

Need a workaround with a symlink /var/opt/rh/rh-php71/log/php-fpm -> /var/log/php-fpm since the PHP Puppet module does not yet support it as param.

  # Workaround for PHP module not allowing to configure log path
  file { '/var/opt/rh/rh-php71/log/php-fpm':
    ensure => directory,
    owner  => 'apache',
    group  => 'apache',
    mode   => '0750'
  }
  ->
  file { '/var/log/php-fpm':
    ensure => link,
    source => '/var/opt/rh/rh-php71/log/php-fpm'
  }

dnsmichi · 2017-11-29T21:02:01Z

puppet-icingaweb2 works just fine, it even creates a new default user.

The module handling is super easy, the examples/ directory is really awesome. All previous defined modules in the profile have been updated.

Need to figure out how to deploy businessprocess config, and ensure that menu.ini is updated with concat::fragment. Director, NagVis, etc. is still pending.

Elasticsearch 6.x support is pending too.

dnsmichi · 2017-11-29T21:03:29Z

dnsmichi · 2017-12-05T16:41:38Z

Update to Elastic Stack 6.x

[root@icinga2-puppet4 ~]# curl -k -u icinga:icinga http://192.168.33.50:9200
{
  "name" : "icinga2-puppet4-elastic-es",
  "cluster_name" : "elastic",
  "cluster_uuid" : "Jt9nktwtQNOVJprFRO6kKw",
  "version" : {
    "number" : "6.0.0",
    "build_hash" : "8f0685b",
    "build_date" : "2017-11-10T18:41:22.859Z",
    "build_snapshot" : false,
    "lucene_version" : "7.0.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Icingabeat and 6.x

Does not work, needs upgrades in Icinga/icingabeat#19

dnsmichi · 2017-12-05T16:53:37Z

Custom index pattern for filebeat cannot be used. Only if you ensure to provide additional setup parameters.

pcfens/puppet-filebeat#146

dnsmichi · 2017-12-06T14:42:15Z

Sticking with Elastic Stack 5.x for now, only testing stuff until Icinga/icinga2#5795 is resolved.

dnsmichi · 2017-12-11T12:46:32Z

Continues in CW 51

dnsmichi · 2017-12-21T13:56:15Z

puppet-icinga2 is fully integrated, awesome. Now for the missing features and their configuration.

dnsmichi · 2018-02-08T17:30:21Z

Director, Graphite 1.x, Elasticsearch 6, Graylog 2.4 have been added. Any required module supports Puppet 4.

dnsmichi · 2018-02-09T17:17:09Z

Cluster scenario built-in for the profile with master and satellite nodes.

dnsmichi · 2018-02-09T17:47:06Z

Renamed all boxes to short names.

Open TODOs

menu.ini with concat (separate task)
better demo config

Will create follow-up issues.

dnsmichi added the area/puppet label Apr 22, 2017

dnsmichi added this to the 2.0.0 milestone Apr 22, 2017

dnsmichi mentioned this issue Apr 22, 2017

Replace bfraser/grafana with voxpupuli/grafana #55

Closed

dnsmichi added the enhancement label Apr 22, 2017

dnsmichi changed the title ~~Use Puppet 4 as provisioner~~ Use Puppet 5 as provisioner Sep 12, 2017

dnsmichi mentioned this issue Nov 27, 2017

Update icinga2x-elastic to Elastic Stack 6.0 #113

Closed

dnsmichi changed the title ~~Use Puppet 5 as provisioner~~ Use Puppet 4 or 5 as provisioner Nov 27, 2017

dnsmichi mentioned this issue Nov 27, 2017

puppet4 voxpupuli/puppet-icinga2#402

Closed

dnsmichi self-assigned this Nov 28, 2017

This was referenced Nov 29, 2017

Use puppet-icingaweb2 instead of custom module #13

Closed

Use puppet-icinga2 instead of custom module #12

Closed

Use Puppet profiles #15

Closed

Provide release and snapshot boxes #20

Closed

dnsmichi mentioned this issue Dec 5, 2017

ElasticsearchWriter gives "Unexpected response code 400" with Elasticsearch 6.x Icinga/icinga2#5795

Closed

dnsmichi changed the title ~~Use Puppet 4 or 5 as provisioner~~ Use Puppet 4 as provisioner Dec 21, 2017

dnsmichi mentioned this issue Feb 12, 2018

Use Puppet 4 as provisioner and in profiles, use official Icinga Puppet modules, rename scenarios #116

Merged

dnsmichi closed this as completed in #116 Feb 12, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use Puppet 4 as provisioner #56

Use Puppet 4 as provisioner #56

dnsmichi commented Apr 22, 2017 •

edited

Loading

dnsmichi commented Sep 12, 2017

dnsmichi commented Nov 27, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 29, 2017

dnsmichi commented Nov 29, 2017 •

edited

Loading

dnsmichi commented Nov 29, 2017

dnsmichi commented Nov 29, 2017

dnsmichi commented Nov 29, 2017

dnsmichi commented Nov 29, 2017

dnsmichi commented Dec 5, 2017

dnsmichi commented Dec 5, 2017

dnsmichi commented Dec 6, 2017

dnsmichi commented Dec 11, 2017

dnsmichi commented Dec 21, 2017

dnsmichi commented Feb 8, 2018

dnsmichi commented Feb 9, 2018

dnsmichi commented Feb 9, 2018

Use Puppet 4 as provisioner #56

Use Puppet 4 as provisioner #56

Comments

dnsmichi commented Apr 22, 2017 • edited Loading

Module Analysis

Tasks

Provisioner Pre Script

Modules

Functional tests

Unclear:

Boxes

dnsmichi commented Sep 12, 2017

dnsmichi commented Nov 27, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 28, 2017

dnsmichi commented Nov 29, 2017

dnsmichi commented Nov 29, 2017 • edited Loading

dnsmichi commented Nov 29, 2017

dnsmichi commented Nov 29, 2017

dnsmichi commented Nov 29, 2017

dnsmichi commented Nov 29, 2017

dnsmichi commented Dec 5, 2017

Update to Elastic Stack 6.x

Icingabeat and 6.x

dnsmichi commented Dec 5, 2017

dnsmichi commented Dec 6, 2017

dnsmichi commented Dec 11, 2017

dnsmichi commented Dec 21, 2017

dnsmichi commented Feb 8, 2018

dnsmichi commented Feb 9, 2018

dnsmichi commented Feb 9, 2018

dnsmichi commented Apr 22, 2017 •

edited

Loading

dnsmichi commented Nov 29, 2017 •

edited

Loading