[BUG] docker.containers requires, but does not install, yum-plugin-versionlock

[anderbubble]

Your setup

Formula commit hash / release tag

master

Versions reports (master & minion)

$ salt --versions-report
Salt Version:
           Salt: 2018.3.4
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.7.3
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: 0.27.7
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.27.4
         Python: 3.7.3 (default, Dec 20 2019, 18:57:59)
   python-gnupg: Not Installed
         PyYAML: 3.13
          PyZMQ: 17.1.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.1
 
System Versions:
           dist: debian 10.3 
         locale: UTF-8
        machine: x86_64
        release: 4.19.0-8-amd64
         system: Linux
        version: debian 10.3

$ salt-minion --versions-report
Salt Version:
           Salt: 3001
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: 4.2.1
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.11.1
        libgit2: Not Installed
       M2Crypto: 0.35.2
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: Not Installed
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.8 (default, Apr  2 2020, 13:34:55)
   python-gnupg: Not Installed
         PyYAML: 3.13
          PyZMQ: 17.0.0
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.1.4
 
System Versions:
           dist: centos 7 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-1127.10.1.el7.x86_64
         system: Linux
        version: CentOS Linux 7 Core

Pillar / config used

base:
  '[hostname]':
    - docker.containers

Bug details

Describe the bug

          ID: docker-package      
    Function: pkg.installed
        Name: docker-ce                                                                                                                         
      Result: False
     Comment: Cannot proceed, yum-plugin-versionlock is not installed.   
     Started: 23:36:22.960722
    Duration: 39598.016 ms
     Changes:   
              ----------
              installed:                             
                  ----------
                  docker-ce:                            
                      ----------
                      new:                                                        
                          3:19.03.12-3.el7
                      old:
                          3:19.03.11-3.el7

Steps to reproduce the bug

• CentOS 7 install with yum-plugin-versionlock not installed.
• Use docker.containers

Expected behaviour

yum-plugin-versionlock should either be a dependency of a salt package (whatever provides pkg) or installed by the docker.containers formula.

Attempts to fix the bug

Installing yum-plugin-versionlock is successful.

[myii]

@anderbubble Thanks for the detailed report. Just an important side-point for now: you appear to be running a vulnerable version of Salt on your master. You really need to update immediately:

• https://community.saltstack.com/blog/active-saltstack-cve-critical-updates/

A secondary note is that it's good practice to have the master on the same or higher version of Salt than all of your minions.

[anderbubble]

you appear to be running a vulnerable version of Salt on your master. You really need to update immediately

@myii Thanks for pointing this out to me. I actually just got done re-building this system (and several of its minions) specifically after getting compromised by that CVE. I guess I assumed that running the latest salt-master available in the latest Debian image from my vps would have a patched version; but I was negligent in checking specifically, and it seems I may never have actually run an apt upgrade after deploying the instance.

That said, I had moved my salt-master from a public interface to a VPN-only interface. So at least I wasn't just flapping in the breeze again.

After an apt upgrade:

$ salt --versions-report
Salt Version:
           Salt: 2018.3.4
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.7.3
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: 0.27.7
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.27.4
         Python: 3.7.3 (default, Dec 20 2019, 18:57:59)
   python-gnupg: Not Installed
         PyYAML: 3.13
          PyZMQ: 17.1.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.1
 
System Versions:
           dist: debian 10.4 
         locale: UTF-8
        machine: x86_64
        release: 4.19.0-8-amd64
         system: Linux
        version: debian 10.4

I note that Salt: 2018.3.4 and release: 4.19.0-8-amd64 are both unchanged; but it did deploy new versions of salt packages.

The following NEW packages will be installed:
  linux-headers-4.19.0-9-amd64 linux-headers-4.19.0-9-common linux-image-4.19.0-9-amd64
The following packages will be upgraded:
  apt apt-utils base-files bind9-host ca-certificates distro-info-data iputils-ping libapt-inst2.0 libapt-pkg5.0 libbind9-161 libdns-export1104 libdns1104 libfuse2 libisc-export1100 libisc1100 libisccc161 libisccfg163
  liblwres161 libnss-systemd libpam-systemd libsystemd0 libudev1 linux-compiler-gcc-8-x86 linux-headers-amd64 linux-image-amd64 linux-kbuild-4.19 linux-libc-dev salt-common salt-master salt-minion salt-ssh systemd systemd-sysv
  tzdata udev wireguard wireguard-dkms wireguard-tools
38 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 77.4 MB of archives.
After this operation, 325 MB of additional disk space will be used.

So I'm confused how I'm meant to discern whether the Debian package I'm using is patched or not from this command.

These are the actual packages that are currently installed:

$ dpkg -l | grep salt
ii  salt-common                     2018.3.4+dfsg1-6+deb10u1     all          shared libraries that salt requires for all packages
ii  salt-master                     2018.3.4+dfsg1-6+deb10u1     all          remote manager to administer servers via salt
ii  salt-minion                     2018.3.4+dfsg1-6+deb10u1     all          client package for salt, the distributed remote execution system
ii  salt-ssh                        2018.3.4+dfsg1-6+deb10u1     all          remote manager to administer servers via Salt SSH

According to https://metadata.ftp-master.debian.org/changelogs//main/s/salt/salt_2018.3.4+dfsg1-6+deb10u1_changelog, dfsg1-6+deb10u1 contains the fix, and is the latest version; so since I got an upgrade I definitely didn't have it before.

(I'm only putting all this all here in case it's useful to someone else who finds this. I'm sure you already know this better than I do.)

A secondary note is that it's good practice to have the master on the same or higher version of Salt than all of your minions.

Yep, and I hadn't noticed it was downrev until filing this report. I intended to resolve that anyway; but I assessed it likely wasn't the cause of this issue. This is just a side-effect of me using salt from Buster on my Debian host(s), and salt from upstream on my CentOS host(s).

Thanks again!

[myii]

@anderbubble From discussions I've had on Slack, the only way to get the patches for EOL versions of Salt (i.e. < 2019.2) is by completing the request form linked on the page I mentioned above. In fact, there's a paragraph there that says just that:

To receive the patches for versions prior to 2019.2.4, please complete this request form: https://www.saltstack.com/lp/request-patch-april-2020/ Tracking these requests helps ensure that those who are running older versions of Salt receive adequate support and attention.

Unfortunately, I don't think you're out of dangerous waters just yet... the alternative is to upgrade to at least the oldest supported version, which would be salt-master_2019.2.5+ds-1_all.deb from http://repo.saltstack.com/py3/debian/10/amd64/2019.2/.

[anderbubble]

@myii I believe you are incorrect. Even though Salt 2018.3.4 is unsupported upstream, it is supported by Debian as part of Debian Stable (Buster).

https://packages.debian.org/buster/salt-master

As such, the Debian security team has applied the patch in Debian, even though upstream isn't delivering updated packages for versions prior to 2019.2.4.

https://metadata.ftp-master.debian.org/changelogs//main/s/salt/salt_2018.3.4+dfsg1-6+deb10u1_changelog

salt (2018.3.4+dfsg1-6+deb10u1) buster-security; urgency=high

• Non-maintainer upload by the Security Team.
• Fix CVE-2020-11651: Resolve issue which allows access to un-intended
  methods in the ClearFuncs class of the salt-master process
  (Closes: #959684)
• Fix CVE-2020-11652: Sanitize paths in ClearFuncs methods provided by
  salt-master (Closes: #959684)
• Add note about log messages to hardening salt docs
• salt-api NET API with the ssh client enabled is vulnerable to command
  injection (CVE-2019-17361) (Closes: #949222)

[myii]

As such, the Debian security team has applied the patch in Debian, even though upstream isn't delivering updated packages for versions prior to 2019.2.4.

@anderbubble Good to hear that the patch is being applied by Debian itself, that should mitigate some of the fallout.

Don't overlook the other point I made either:

A secondary note is that it's good practice to have the master on the same or higher version of Salt than all of your minions.

---

Back on-topic, I've been able to reproduce this bug in Travis.

https://travis-ci.org/github/myii/docker-formula/jobs/702927636#L2069-L2073

                 ID: docker-package
           Function: pkg.installed
               Name: docker-ce
             Result: False
            Comment: Cannot proceed, yum-plugin-versionlock is not installed.

[myii]

@anderbubble Forgot to mention, I got most of this working with the following commit:

• myii@37738c9

It only fails to start some of the services now:

• https://travis-ci.org/github/myii/docker-formula/jobs/702975433#L2212-L2216
• https://travis-ci.org/github/myii/docker-formula/jobs/702975433#L2257-L2261

Perhaps you can take what I've done and finalise a fix.

---

Note: That's a very centos-7-centric commit; the final solution can't have python36-docker for all of the RedHat platforms. I was just trying the quickest solution at the time.

[anderbubble]

@myii why was changing from python3-docker (from the left) to python36-docker (in your commit) necessary or useful, if it can't be in the final solution?

[myii]

@myii why was changing from python3-docker (from the left) to python36-docker (in your commit) necessary or useful, if it can't be in the final solution?

@anderbubble Since there's no osfingermap.yaml in this repo, I wanted a quick way to test the idea. It could be possible to use this file instead:

• https://github.com/saltstack-formulas/docker-formula/blob/master/docker/codenamemap.yaml

However, I just needed something quick to see if any progress could be made. Did you manage to try anything out based on what I shared?

[noelmcloughlin]

Fixed in merged #256