diff --git a/doc/topics/venafi/index.rst b/doc/topics/venafi/index.rst index b756d4f271ea..729a0ae91aee 100644 --- a/doc/topics/venafi/index.rst +++ b/doc/topics/venafi/index.rst @@ -5,256 +5,147 @@ Venafi Tools for Salt Introduction ~~~~~~~~~~~~ -Before using these modules you need to register an account with Venafi, and -configure it in your ``master`` configuration file. +First, you need to configure the ``master`` file. This is because +all module functions require either a configured ``api_key`` (for Cloud) or +``a ttp_user`` with a ``tpp_password`` and a ``base_url`` (for Trust Platform). -First, you need to add a placeholder to the ``master`` file. This is because -the module will not load unless it finds an ``api_key`` setting, valid or not. -Open up ``/etc/salt/master`` and add: +For Venafi Cloud: .. code-block:: yaml venafi: - api_key: None - -Then register your email address with Venafi using the following command: + api_key: abcdef01-2345-6789-abcd-ef0123456789 + base_url: "https://cloud.venafi.example.com/" (optional) -.. code-block:: bash +If you don't have a Venafi Cloud account, you can sign up for one on the `enrollment page`_. - salt-run venafi.register +.. _enrollment page: https://www.venafi.com/platform/cloud/devops -This command will not return an ``api_key`` to you; that will be send to you -via email from Venafi. Once you have received that key, open up your ``master`` -file and set the ``api_key`` to it: +For Venafi Platform: .. code-block:: yaml venafi: - api_key: abcdef01-2345-6789-abcd-ef0123456789 + base_url: "https://tpp.example.com/" + tpp_user: admin + tpp_password: "Str0ngPa$$w0rd" + trust_bundle: "/opt/venafi/bundle.pem" + +*It is not common for the Venafi Platform's REST API (WebSDK) to be secured using a certificate issued by a publicly trusted CA, therefore establishing trust for that server certificate is a critical part of your configuration. Ideally this is done by obtaining the root CA certificate in the issuing chain in PEM format and copying that file to your Salt Master (e.g. /opt/venafi/bundle.pem). You then reference that file using the 'trust_bundle' parameter as shown above.* -To enable the ability for creating keys and certificates it is necessary to enable the -external pillars. Open the ``/etc/salt/master`` file and add: +For the Venafi module to create keys and certificates it is necessary to enable external pillars. This is done by adding the following to the ``/etc/salt/master`` file: .. code-block:: yaml ext_pillar: - venafi: True -To modify the URL being used for the Venafi Certificate issuance modify the file -in ``/etc/salt/master`` and add the base_url information following under the venafi tag: - -.. code-block:: yaml - - venafi: - base_url: http://newurl.venafi.com - - -Example Usage -~~~~~~~~~~~~~ -Generate a CSR and submit it to Venafi for issuance, using the 'Internet' zone: -salt-run venafi.request minion.example.com minion.example.com zone=Internet - -Retrieve a certificate for a previously submitted request with request ID -aaa-bbb-ccc-dddd: -salt-run venafi.pickup aaa-bbb-ccc-dddd Runner Functions ~~~~~~~~~~~~~~~~ -gen_key -------- - -Generate and return a ``private_key``. If a ``dns_name`` is passed in, the -``private_key`` will be cached under that name. - -The key will be generated based on the policy values that were configured -by the Venafi administrator. A default Certificate Use Policy is associated -with a zone; the key type and key length parameters associated with this value -will be used. - -.. code-block:: bash - - salt-run venafi.gen_key minion.example.com minion.example.com zone=Internet \ - password=SecretSauce - -:param str minion_id: Required. The name of the minion which hosts the domain - name in question. - -:param str dns_name: Required. The FQDN of the domain that will be hosted on - the minion. - -:param str zone: Required. Default value is "default". The zone on Venafi that - the domain belongs to. - -:param str password: Optional. If specified, the password to use to access the - generated key. - - -gen_csr -------- - -Generate a csr using the host's private_key. Analogous to: - -.. code-block:: bash - - salt-run venafi.gen_csr minion.example.com minion.example.com country=US \ - state=California loc=Sacramento org=CompanyName org_unit=DevOps \ - zone=Internet password=SecretSauce - -:param str minion_id: Required. - -:param str dns_name: Required. - -:param str zone: Optional. Default value is "default". The zone on Venafi that - the domain belongs to. - -:param str country=None: Optional. The two-letter ISO abbreviation for your - country. - -:param str state=None: Optional. The state/county/region where your - organisation is legally located. Must not be abbreviated. - -:param str loc=None: Optional. The city where your organisation is legally - located. - -:param str org=None: Optional. The exact legal name of your organisation. Do - not abbreviate your organisation name. - -:param str org_unit=None: Optional. Section of the organisation, can be left - empty if this does not apply to your case. - -:param str password=None: Optional. Password for the CSR. - - request ------- +This command is used to enroll a certificate from Venafi Cloud or Venafi Platform. -Request a new certificate. Analogous to: - -.. code-block:: bash - - salt-run venafi.request minion.example.com minion.example.com country=US \ - state=California loc=Sacramento org=CompanyName org_unit=DevOps \ - zone=Internet password=SecretSauce +``minion_id`` + ID of the minion for which the certificate is being issued. Required. -:param str minion_id: Required. +``dns_name`` + DNS subject name for the certificate. Required if ``csr_path`` is not specified. -:param str dns_name: Required. +``csr_path`` + Full path name of certificate signing request file to enroll. Required if ``dns_name`` is not specified. -:param str zone: Required. Default value is "default". The zone on Venafi that - the certificate request will be submitted to. +``zone`` + Venafi Cloud zone ID or Venafi Platform folder that specify key and certificate policy. Defaults to "Default". For Venafi Cloud, the Zone ID can be found in the Zone page for your Venafi Cloud project. -:param str country=None: Optional. The two-letter ISO abbreviation for your - country. +``org_unit`` + Business Unit, Department, etc. Do not specify if it does not apply. -:param str state=None: Optional. The state/county/region where your - organisation is legally located. Must not be abbreviated. +``org`` + Exact legal name of your organization. Do not abbreviate. -:param str loc=None: Optional. The city where your organisation is legally - located. +``loc`` + City/locality where your organization is legally located. -:param str org=None: Optional. The exact legal name of your organisation. Do - not abbreviate your organisation name. +``state`` + State or province where your organization is legally located. Must not be abbreviated. -:param str org_unit=None: Optional. Section of the organisation, can be left - empty if this does not apply to your case. +``country`` + Country where your organization is legally located; two-letter ISO code. -:param str password=None: Optional. Password for the CSR. +``key_password`` + Password for encrypting the private key. -:param str company_id=None: Optional, but may be configured in ``master`` file - instead. - -register --------- - -Register a new user account +The syntax for requesting a new certificate with private key generation looks like this: .. code-block:: bash - salt-run venafi.register username@example.com - -:param str email: Required. The email address to use for the new Venafi account. - - -show_company ------------- + salt-run venafi.request minion.example.com dns_name=www.example.com \ + country=US state=California loc=Sacramento org="Company Name" org_unit=DevOps \ + zone=Internet key_password=SecretSauce -Show company information, especially the company id +And the syntax for requesting a new certificate using a previously generated CSR looks like this: .. code-block:: bash - salt-run venafi.show_company example.com + salt-run venafi.request minion.example.com csr_path=/tmp/minion.req zone=Internet -:param str domain: Required. The domain name to look up information for. - -show_csrs +show_cert --------- +This command is used to show last issued certificate for domain. -Show certificate requests for the configured API key. +``dns_name`` + DNS subject name of the certificate to look up. .. code-block:: bash - salt-run venafi.show_csrs - - -show_zones ----------- + salt-run venafi.show_cert www.example.com -Show zones for the specified company id. - -.. code-block:: bash - salt-run venafi.show_zones - -:param str company_id: Optional. The company id to show the zones for. - - -pickup, show_cert +list_domain_cache ----------------- - -Show certificate requests for the specified certificate id. Analogous to the -VCert pickup command. +This command lists domains that have been cached on this Salt Master. .. code-block:: bash - salt-run venafi.pickup 4295ebc0-14bf-11e7-b965-1df050017ec1 - -:param str id\_: Required. The id of the certificate to look up. + salt-run venafi.list_domain_cache -show_rsa --------- +del_cached_domain +----------------- +This command deletes a domain from the Salt Master's cache. -Show a private RSA key. +``domains`` + A domain name, or a comma-separated list of domain names, to delete from this master's cache. .. code-block:: bash - salt-run venafi.show_rsa minion.example.com minion.example.com - -:param str minion_id: The name of the minion to display the key for. - -:param str dns_name: The domain name to display the key for. - - -list_domain_cache ------------------ + salt-run venafi.del_cached_domain www.example.com -List domains that have been cached on this master. -.. code-block:: bash +Transfer certificate to a minion +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - salt-run venafi.list_domain_cache +To transfer a cached certificate to a minion, you can use Venafi pillar. +Example state (SLS) file: -del_cached_domain ------------------ - -Delete a domain from this master's cache. +.. code-block:: yaml -.. code-block:: bash + /etc/ssl/cert/www.example.com.crt: + file.managed: + - contents_pillar: venafi:www.example.com:cert + - replace: True - salt-run venafi.delete_domain_cache example.com + /etc/ssl/cert/www.example.com.key: + file.managed: + - contents_pillar: venafi:www.example.com:pkey + - replace: True -:param str domains: A domain name, or a comma-separated list of domain names, - to delete from this master's cache. + /etc/ssl/cert/www.example.com-chain.pem: + file.managed: + - contents_pillar: venafi:www.example.com:chain + - replace: True \ No newline at end of file diff --git a/requirements/static/darwin.in b/requirements/static/darwin.in index abaeb66f9876..31e4613aaa4d 100644 --- a/requirements/static/darwin.in +++ b/requirements/static/darwin.in @@ -34,3 +34,5 @@ Genshi Cheetah3==3.1.0 Mako wempy; python_version <'3' + +vcert~=0.7.0 \ No newline at end of file diff --git a/requirements/static/linux.in b/requirements/static/linux.in index 731d37d53969..622e6f38c92c 100644 --- a/requirements/static/linux.in +++ b/requirements/static/linux.in @@ -42,6 +42,7 @@ timelib tornado<5.0 virtualenv watchdog +vcert~=0.7.0 # Available template libraries that can be used Genshi diff --git a/requirements/static/py2.7/darwin.txt b/requirements/static/py2.7/darwin.txt index 130ecce6fee3..48ec2b3a1c0a 100644 --- a/requirements/static/py2.7/darwin.txt +++ b/requirements/static/py2.7/darwin.txt @@ -137,3 +137,17 @@ zc.lockfile==1.4 # via cherrypy zipp==0.6.0 # via importlib-metadata # Passthrough dependencies from pkg/osx/req.txt pyobjc==5.1.2 +# Passthrough dependencies from pkg/osx/req.txt +pyobjc==5.1.2 +# Passthrough dependencies from pkg/osx/req.txt +pyobjc==5.1.2 +# Passthrough dependencies from pkg/osx/req.txt +pyobjc==5.1.2 +# Passthrough dependencies from pkg/osx/req.txt +pyobjc==5.1.2 +# Passthrough dependencies from pkg/osx/req.txt +pyobjc==5.1.2 +# Passthrough dependencies from pkg/osx/req.txt +pyobjc==5.1.2 +# Passthrough dependencies from pkg/osx/req.txt +pyobjc==5.1.2 diff --git a/requirements/static/py3.4/linux-crypto.txt b/requirements/static/py3.4/linux-crypto.txt index a83659e76786..d0634bbd83f4 100644 --- a/requirements/static/py3.4/linux-crypto.txt +++ b/requirements/static/py3.4/linux-crypto.txt @@ -6,4 +6,3 @@ # m2crypto==0.35.2 pycryptodomex==3.9.3 -typing==3.7.4.1 # via m2crypto diff --git a/requirements/static/py3.4/linux.txt b/requirements/static/py3.4/linux.txt index 3e41ad8ffaa6..29333b77f0bc 100644 --- a/requirements/static/py3.4/linux.txt +++ b/requirements/static/py3.4/linux.txt @@ -6,19 +6,19 @@ # apache-libcloud==2.0.0 argh==0.26.2 # via watchdog -asn1crypto==0.24.0 # via cryptography +asn1crypto==1.2.0 # via certvalidator, cryptography, oscrypto atomicwrites==1.3.0 # via pytest attrs==19.1.0 # via pytest aws-xray-sdk==0.95 # via moto -backports-abc==0.5 # via tornado backports.functools-lru-cache==1.5 # via cheroot -backports.ssl-match-hostname==3.7.0.1 # via docker, websocket-client +backports.ssl-match-hostname==3.7.0.1 # via docker bcrypt==3.1.6 # via paramiko boto3==1.9.132 boto==2.49.0 botocore==1.12.132 # via boto3, moto, s3transfer cachetools==3.1.0 # via google-auth certifi==2019.3.9 +certvalidator==0.11.1 # via vcert cffi==1.12.2 chardet==3.0.4 # via requests cheetah3==3.1.0 @@ -26,7 +26,7 @@ cheroot==6.5.4 # via cherrypy cherrypy==17.3.0 contextlib2==0.5.5 # via cherrypy croniter==0.3.29 -cryptography==2.6.1 # via moto, paramiko, pyopenssl +cryptography==2.6.1 # via moto, paramiko, pyopenssl, vcert dnspython==1.16.0 docker-pycreds==0.4.0 # via docker docker==3.7.2 @@ -61,6 +61,7 @@ moto==1.3.7 msgpack==0.5.6 ncclient==0.6.4 # via junos-eznc netaddr==0.7.19 # via junos-eznc +oscrypto==1.1.0 # via certvalidator packaging==19.2 # via pytest paramiko==2.4.2 pathlib2==2.3.3 # via importlib-metadata, pytest @@ -85,7 +86,7 @@ pytest-salt-runtests-bridge==2019.7.10 pytest-salt==2019.12.27 pytest-tempdir==2019.10.12 pytest==4.6.6 -python-dateutil==2.8.0 # via botocore, croniter, kubernetes, moto +python-dateutil==2.8.0 # via botocore, croniter, kubernetes, moto, vcert python-etcd==0.4.5 python-gnupg==0.4.4 python-jose==2.0.2 # via moto @@ -104,14 +105,14 @@ scp==0.13.2 # via junos-eznc selectors2==2.0.1 # via ncclient setproctitle==1.1.10 setuptools-scm==3.2.0 -singledispatch==3.4.0.3 # via tornado -six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pathlib2, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client +six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pathlib2, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, tempora, vcert, websocket-client smmap2==2.0.5 # via gitdb2 strict-rfc3339==0.7 tempora==1.14.1 # via portend timelib==0.2.4 tornado==4.5.3 ; python_version >= "3.4" urllib3==1.24.2 # via botocore, kubernetes, python-etcd, requests +vcert==0.7.3 virtualenv==16.4.3 watchdog==0.9.0 wcwidth==0.1.7 # via pytest diff --git a/requirements/static/py3.5/darwin-crypto.txt b/requirements/static/py3.5/darwin-crypto.txt index a6a86036f2e7..492a8bd18558 100644 --- a/requirements/static/py3.5/darwin-crypto.txt +++ b/requirements/static/py3.5/darwin-crypto.txt @@ -6,4 +6,3 @@ # m2crypto==0.35.2 pycryptodomex==3.9.0 -typing==3.7.4.1 # via m2crypto diff --git a/requirements/static/py3.5/darwin.txt b/requirements/static/py3.5/darwin.txt index 9ed4e6aa34aa..2e22c1cf5398 100644 --- a/requirements/static/py3.5/darwin.txt +++ b/requirements/static/py3.5/darwin.txt @@ -6,19 +6,21 @@ # apache-libcloud==2.4.0 argh==0.26.2 # via watchdog -asn1crypto==0.24.0 # via cryptography +asn1crypto==1.3.0 # via certvalidator, cryptography, oscrypto atomicwrites==1.3.0 # via pytest attrs==19.1.0 # via pytest aws-xray-sdk==0.95 # via moto -backports-abc==0.5 backports.functools-lru-cache==1.5 # via cheroot backports.ssl_match_hostname==3.7.0.1 +backports_abc==0.5 bcrypt==3.1.6 # via paramiko boto3==1.9.132 boto==2.49.0 botocore==1.12.132 # via boto3, moto, s3transfer cachetools==3.1.0 # via google-auth certifi==2019.3.9 +certvalidator==0.11.1 # via vcert +certvalidator==0.11.1 # via vcert cffi==1.12.2 chardet==3.0.4 # via requests cheetah3==3.1.0 @@ -65,6 +67,7 @@ msgpack-python==0.5.6 msgpack==0.5.6 ncclient==0.6.4 # via junos-eznc netaddr==0.7.19 # via junos-eznc +oscrypto==1.2.0 # via certvalidator packaging==19.2 # via pytest paramiko==2.4.2 # via junos-eznc, ncclient, scp pathlib2==2.3.3 # via pytest @@ -104,7 +107,7 @@ salttesting==2017.6.1 scp==0.13.2 # via junos-eznc setproctitle==1.1.10 singledispatch==3.4.0.3 -six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kubernetes, mock, more-itertools, moto, ncclient, packaging, pathlib2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client +six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kubernetes, mock, more-itertools, moto, ncclient, packaging, pathlib2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, vcert, websocket-client smmap2==2.0.5 # via gitdb2 smmap==0.9.0 strict-rfc3339==0.7 @@ -112,6 +115,7 @@ tempora==1.14.1 # via portend timelib==0.2.4 tornado==4.5.3 ; python_version >= "3.4" urllib3==1.24.2 # via botocore, kubernetes, python-etcd, requests +vcert==0.7.3 virtualenv==16.4.3 vultr==1.0.1 watchdog==0.9.0 diff --git a/requirements/static/py3.5/linux-crypto.txt b/requirements/static/py3.5/linux-crypto.txt index 59c45171e55c..6cdeaecccc4d 100644 --- a/requirements/static/py3.5/linux-crypto.txt +++ b/requirements/static/py3.5/linux-crypto.txt @@ -6,4 +6,3 @@ # m2crypto==0.35.2 pycryptodomex==3.9.3 -typing==3.7.4.1 # via m2crypto diff --git a/requirements/static/py3.5/linux.txt b/requirements/static/py3.5/linux.txt index c231a6a3651a..040e6b1120fa 100644 --- a/requirements/static/py3.5/linux.txt +++ b/requirements/static/py3.5/linux.txt @@ -6,19 +6,18 @@ # apache-libcloud==2.0.0 argh==0.26.2 # via watchdog -asn1crypto==0.24.0 # via cryptography +asn1crypto==1.2.0 # via certvalidator, cryptography, oscrypto atomicwrites==1.3.0 # via pytest attrs==19.1.0 # via pytest aws-xray-sdk==0.95 # via moto -backports-abc==0.5 # via tornado backports.functools-lru-cache==1.5 # via cheroot -backports.ssl-match-hostname==3.7.0.1 # via websocket-client bcrypt==3.1.6 # via paramiko boto3==1.9.132 boto==2.49.0 botocore==1.12.132 # via boto3, moto, s3transfer cachetools==3.1.0 # via google-auth certifi==2019.3.9 +certvalidator==0.11.1 # via vcert cffi==1.12.2 chardet==3.0.4 # via requests cheetah3==3.1.0 @@ -26,7 +25,7 @@ cheroot==6.5.4 # via cherrypy cherrypy==17.3.0 contextlib2==0.5.5 # via cherrypy croniter==0.3.29 -cryptography==2.6.1 # via moto, paramiko, pyopenssl +cryptography==2.6.1 # via moto, paramiko, pyopenssl, vcert dnspython==1.16.0 docker-pycreds==0.4.0 # via docker docker==3.7.2 @@ -61,6 +60,7 @@ moto==1.3.7 msgpack==0.5.6 ncclient==0.6.4 # via junos-eznc netaddr==0.7.19 # via junos-eznc +oscrypto==1.1.0 # via certvalidator packaging==19.2 # via pytest paramiko==2.4.2 pathlib2==2.3.3 # via pytest @@ -85,7 +85,7 @@ pytest-salt-runtests-bridge==2019.7.10 pytest-salt==2019.12.27 pytest-tempdir==2019.10.12 pytest==4.6.6 -python-dateutil==2.8.0 # via botocore, croniter, kubernetes, moto +python-dateutil==2.8.0 # via botocore, croniter, kubernetes, moto, vcert python-etcd==0.4.5 python-gnupg==0.4.4 python-jose==2.0.2 # via moto @@ -102,14 +102,14 @@ salttesting==2017.6.1 scp==0.13.2 # via junos-eznc setproctitle==1.1.10 setuptools-scm==3.2.0 -singledispatch==3.4.0.3 # via tornado -six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pathlib2, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client +six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pathlib2, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, tempora, vcert, websocket-client smmap2==2.0.5 # via gitdb2 strict-rfc3339==0.7 tempora==1.14.1 # via portend timelib==0.2.4 tornado==4.5.3 ; python_version >= "3.4" urllib3==1.24.2 # via botocore, kubernetes, python-etcd, requests +vcert==0.7.3 virtualenv==16.4.3 watchdog==0.9.0 wcwidth==0.1.7 # via pytest diff --git a/requirements/static/py3.5/windows-crypto.txt b/requirements/static/py3.5/windows-crypto.txt index 1e209d42238b..091858543a9c 100644 --- a/requirements/static/py3.5/windows-crypto.txt +++ b/requirements/static/py3.5/windows-crypto.txt @@ -6,4 +6,3 @@ # m2crypto==0.35.2 pycryptodomex==3.9.0 -typing==3.7.4.1 # via m2crypto diff --git a/requirements/static/py3.6/darwin-crypto.txt b/requirements/static/py3.6/darwin-crypto.txt index 9c11d937f084..3cc68e7cab08 100644 --- a/requirements/static/py3.6/darwin-crypto.txt +++ b/requirements/static/py3.6/darwin-crypto.txt @@ -6,4 +6,3 @@ # m2crypto==0.35.2 pycryptodomex==3.9.0 -typing==3.7.4.1 # via m2crypto diff --git a/requirements/static/py3.6/darwin.txt b/requirements/static/py3.6/darwin.txt index c1fa12beaace..3f4973562189 100644 --- a/requirements/static/py3.6/darwin.txt +++ b/requirements/static/py3.6/darwin.txt @@ -6,19 +6,20 @@ # apache-libcloud==2.4.0 argh==0.26.2 # via watchdog -asn1crypto==0.24.0 # via cryptography +asn1crypto==1.3.0 # via certvalidator, cryptography, oscrypto atomicwrites==1.3.0 # via pytest attrs==19.1.0 # via pytest aws-xray-sdk==0.95 # via moto -backports-abc==0.5 backports.functools-lru-cache==1.5 # via cheroot backports.ssl_match_hostname==3.7.0.1 +backports_abc==0.5 bcrypt==3.1.6 # via paramiko boto3==1.9.132 boto==2.49.0 botocore==1.12.132 # via boto3, moto, s3transfer cachetools==3.1.0 # via google-auth certifi==2019.3.9 +certvalidator==0.11.1 # via vcert cffi==1.12.2 chardet==3.0.4 # via requests cheetah3==3.1.0 @@ -65,6 +66,7 @@ msgpack-python==0.5.6 msgpack==0.5.6 ncclient==0.6.4 # via junos-eznc netaddr==0.7.19 # via junos-eznc +oscrypto==1.2.0 # via certvalidator packaging==19.2 # via pytest paramiko==2.4.2 # via junos-eznc, ncclient, scp pathtools==0.1.2 # via watchdog @@ -104,7 +106,7 @@ salttesting==2017.6.1 scp==0.13.2 # via junos-eznc setproctitle==1.1.10 singledispatch==3.4.0.3 -six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kubernetes, mock, more-itertools, moto, ncclient, packaging, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client +six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kubernetes, mock, more-itertools, moto, ncclient, packaging, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, vcert, websocket-client smmap2==2.0.5 # via gitdb2 smmap==0.9.0 strict-rfc3339==0.7 @@ -112,6 +114,7 @@ tempora==1.14.1 # via portend timelib==0.2.4 tornado==4.5.3 ; python_version >= "3.4" urllib3==1.24.2 # via botocore, kubernetes, python-etcd, requests +vcert==0.7.3 virtualenv==16.4.3 vultr==1.0.1 watchdog==0.9.0 diff --git a/requirements/static/py3.6/linux-crypto.txt b/requirements/static/py3.6/linux-crypto.txt index 3b92d88ffb20..efb7b5dc6eaa 100644 --- a/requirements/static/py3.6/linux-crypto.txt +++ b/requirements/static/py3.6/linux-crypto.txt @@ -6,4 +6,3 @@ # m2crypto==0.35.2 pycryptodomex==3.9.3 -typing==3.7.4.1 # via m2crypto diff --git a/requirements/static/py3.6/linux.txt b/requirements/static/py3.6/linux.txt index 5e2a6071162d..3b4eee1bd7c2 100644 --- a/requirements/static/py3.6/linux.txt +++ b/requirements/static/py3.6/linux.txt @@ -6,19 +6,18 @@ # apache-libcloud==2.0.0 argh==0.26.2 # via watchdog -asn1crypto==0.24.0 # via cryptography +asn1crypto==1.2.0 # via certvalidator, cryptography, oscrypto atomicwrites==1.3.0 # via pytest attrs==19.1.0 # via pytest aws-xray-sdk==0.95 # via moto -backports-abc==0.5 # via tornado backports.functools-lru-cache==1.5 # via cheroot -backports.ssl-match-hostname==3.7.0.1 # via websocket-client bcrypt==3.1.6 # via paramiko boto3==1.9.132 boto==2.49.0 botocore==1.12.132 # via boto3, moto, s3transfer cachetools==3.1.0 # via google-auth certifi==2019.3.9 +certvalidator==0.11.1 # via vcert cffi==1.12.2 chardet==3.0.4 # via requests cheetah3==3.1.0 @@ -26,7 +25,7 @@ cheroot==6.5.4 # via cherrypy cherrypy==17.3.0 contextlib2==0.5.5 # via cherrypy croniter==0.3.29 -cryptography==2.6.1 # via moto, paramiko, pyopenssl +cryptography==2.6.1 # via moto, paramiko, pyopenssl, vcert dnspython==1.16.0 docker-pycreds==0.4.0 # via docker docker==3.7.2 @@ -61,6 +60,7 @@ moto==1.3.7 msgpack==0.5.6 ncclient==0.6.4 # via junos-eznc netaddr==0.7.19 # via junos-eznc +oscrypto==1.1.0 # via certvalidator packaging==19.2 # via pytest paramiko==2.4.2 pathtools==0.1.2 # via watchdog @@ -85,7 +85,7 @@ pytest-salt-runtests-bridge==2019.7.10 pytest-salt==2019.12.27 pytest-tempdir==2019.10.12 pytest==4.6.6 -python-dateutil==2.8.0 # via botocore, croniter, kubernetes, moto +python-dateutil==2.8.0 # via botocore, croniter, kubernetes, moto, vcert python-etcd==0.4.5 python-gnupg==0.4.4 python-jose==2.0.2 # via moto @@ -102,14 +102,14 @@ salttesting==2017.6.1 scp==0.13.2 # via junos-eznc setproctitle==1.1.10 setuptools-scm==3.2.0 -singledispatch==3.4.0.3 # via tornado -six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client +six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, tempora, vcert, websocket-client smmap2==2.0.5 # via gitdb2 strict-rfc3339==0.7 tempora==1.14.1 # via portend timelib==0.2.4 tornado==4.5.3 ; python_version >= "3.4" urllib3==1.24.2 # via botocore, kubernetes, python-etcd, requests +vcert==0.7.3 virtualenv==16.4.3 watchdog==0.9.0 wcwidth==0.1.7 # via pytest diff --git a/requirements/static/py3.6/windows-crypto.txt b/requirements/static/py3.6/windows-crypto.txt index 2e4eaf54bcc0..6f6886495685 100644 --- a/requirements/static/py3.6/windows-crypto.txt +++ b/requirements/static/py3.6/windows-crypto.txt @@ -6,4 +6,3 @@ # m2crypto==0.35.2 pycryptodomex==3.9.0 -typing==3.7.4.1 # via m2crypto diff --git a/requirements/static/py3.7/darwin-crypto.txt b/requirements/static/py3.7/darwin-crypto.txt index 370308dc087b..0a4ffcf4c8d4 100644 --- a/requirements/static/py3.7/darwin-crypto.txt +++ b/requirements/static/py3.7/darwin-crypto.txt @@ -6,4 +6,3 @@ # m2crypto==0.35.2 pycryptodomex==3.9.0 -typing==3.7.4.1 # via m2crypto diff --git a/requirements/static/py3.7/darwin.txt b/requirements/static/py3.7/darwin.txt index ce207e364643..925d78b2e28b 100644 --- a/requirements/static/py3.7/darwin.txt +++ b/requirements/static/py3.7/darwin.txt @@ -6,19 +6,20 @@ # apache-libcloud==2.4.0 argh==0.26.2 # via watchdog -asn1crypto==0.24.0 # via cryptography +asn1crypto==1.3.0 # via certvalidator, cryptography, oscrypto atomicwrites==1.3.0 # via pytest attrs==19.1.0 # via pytest aws-xray-sdk==0.95 # via moto -backports-abc==0.5 backports.functools-lru-cache==1.5 # via cheroot backports.ssl_match_hostname==3.7.0.1 +backports_abc==0.5 bcrypt==3.1.6 # via paramiko boto3==1.9.132 boto==2.49.0 botocore==1.12.132 # via boto3, moto, s3transfer cachetools==3.1.0 # via google-auth certifi==2019.3.9 +certvalidator==0.11.1 # via vcert cffi==1.12.2 chardet==3.0.4 # via requests cheetah3==3.1.0 @@ -65,6 +66,7 @@ msgpack-python==0.5.6 msgpack==0.5.6 ncclient==0.6.4 # via junos-eznc netaddr==0.7.19 # via junos-eznc +oscrypto==1.2.0 # via certvalidator packaging==19.2 # via pytest paramiko==2.4.2 # via junos-eznc, ncclient, scp pathtools==0.1.2 # via watchdog @@ -104,7 +106,7 @@ salttesting==2017.6.1 scp==0.13.2 # via junos-eznc setproctitle==1.1.10 singledispatch==3.4.0.3 -six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kubernetes, mock, more-itertools, moto, ncclient, packaging, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client +six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kubernetes, mock, more-itertools, moto, ncclient, packaging, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, vcert, websocket-client smmap2==2.0.5 # via gitdb2 smmap==0.9.0 strict-rfc3339==0.7 @@ -112,6 +114,7 @@ tempora==1.14.1 # via portend timelib==0.2.4 tornado==4.5.3 ; python_version >= "3.4" urllib3==1.24.2 # via botocore, kubernetes, python-etcd, requests +vcert==0.7.3 virtualenv==16.4.3 vultr==1.0.1 watchdog==0.9.0 diff --git a/requirements/static/py3.7/linux-crypto.txt b/requirements/static/py3.7/linux-crypto.txt index a376a05e786c..1139152146b9 100644 --- a/requirements/static/py3.7/linux-crypto.txt +++ b/requirements/static/py3.7/linux-crypto.txt @@ -6,4 +6,3 @@ # m2crypto==0.35.2 pycryptodomex==3.9.3 -typing==3.7.4.1 # via m2crypto diff --git a/requirements/static/py3.7/linux.txt b/requirements/static/py3.7/linux.txt index 8080ddf178bf..d59336649eb8 100644 --- a/requirements/static/py3.7/linux.txt +++ b/requirements/static/py3.7/linux.txt @@ -6,19 +6,18 @@ # apache-libcloud==2.0.0 argh==0.26.2 # via watchdog -asn1crypto==0.24.0 # via cryptography +asn1crypto==1.2.0 # via certvalidator, cryptography, oscrypto atomicwrites==1.3.0 # via pytest attrs==19.1.0 # via pytest aws-xray-sdk==0.95 # via moto -backports-abc==0.5 # via tornado backports.functools-lru-cache==1.5 # via cheroot -backports.ssl-match-hostname==3.7.0.1 # via websocket-client bcrypt==3.1.6 # via paramiko boto3==1.9.132 boto==2.49.0 botocore==1.12.132 # via boto3, moto, s3transfer cachetools==3.1.0 # via google-auth certifi==2019.3.9 +certvalidator==0.11.1 # via vcert cffi==1.12.2 chardet==3.0.4 # via requests cheetah3==3.1.0 @@ -26,7 +25,7 @@ cheroot==6.5.4 # via cherrypy cherrypy==17.3.0 contextlib2==0.5.5 # via cherrypy croniter==0.3.29 -cryptography==2.6.1 # via moto, paramiko, pyopenssl +cryptography==2.6.1 # via moto, paramiko, pyopenssl, vcert dnspython==1.16.0 docker-pycreds==0.4.0 # via docker docker==3.7.2 @@ -61,6 +60,7 @@ moto==1.3.7 msgpack==0.5.6 ncclient==0.6.4 # via junos-eznc netaddr==0.7.19 # via junos-eznc +oscrypto==1.1.0 # via certvalidator packaging==19.2 # via pytest paramiko==2.4.2 pathtools==0.1.2 # via watchdog @@ -85,7 +85,7 @@ pytest-salt-runtests-bridge==2019.7.10 pytest-salt==2019.12.27 pytest-tempdir==2019.10.12 pytest==4.6.6 -python-dateutil==2.8.0 # via botocore, croniter, kubernetes, moto +python-dateutil==2.8.0 # via botocore, croniter, kubernetes, moto, vcert python-etcd==0.4.5 python-gnupg==0.4.4 python-jose==2.0.2 # via moto @@ -102,14 +102,14 @@ salttesting==2017.6.1 scp==0.13.2 # via junos-eznc setproctitle==1.1.10 setuptools-scm==3.2.0 -singledispatch==3.4.0.3 # via tornado -six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client +six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, tempora, vcert, websocket-client smmap2==2.0.5 # via gitdb2 strict-rfc3339==0.7 tempora==1.14.1 # via portend timelib==0.2.4 tornado==4.5.3 ; python_version >= "3.4" urllib3==1.24.2 # via botocore, kubernetes, python-etcd, requests +vcert==0.7.3 virtualenv==16.4.3 watchdog==0.9.0 wcwidth==0.1.7 # via pytest diff --git a/requirements/static/py3.7/windows-crypto.txt b/requirements/static/py3.7/windows-crypto.txt index ee941ca2d08e..82aeec9d3bfd 100644 --- a/requirements/static/py3.7/windows-crypto.txt +++ b/requirements/static/py3.7/windows-crypto.txt @@ -6,4 +6,3 @@ # m2crypto==0.35.2 pycryptodomex==3.9.0 -typing==3.7.4.1 # via m2crypto diff --git a/salt/pillar/venafi.py b/salt/pillar/venafi.py index 5b4d728eb40b..74cb89514c46 100644 --- a/salt/pillar/venafi.py +++ b/salt/pillar/venafi.py @@ -36,9 +36,8 @@ def ext_pillar(minion_id, pillar, conf): cache = salt.cache.Cache(__opts__, syspaths.CACHE_DIR) ret = {} - dns_names = cache.fetch('venafi/minions', minion_id) - for dns_name in dns_names: + for dns_name in cache.list('venafi/domains'): data = cache.fetch('venafi/domains', dns_name) - ret[dns_name] = data - del ret[dns_name]['csr'] + if data['minion_id'] == minion_id: + ret[dns_name] = data return {'venafi': ret} diff --git a/salt/runners/venafiapi.py b/salt/runners/venafiapi.py index 1da6b7d59b65..bc584165372a 100644 --- a/salt/runners/venafiapi.py +++ b/salt/runners/venafiapi.py @@ -1,50 +1,36 @@ # -*- coding: utf-8 -*- -''' +""" Support for Venafi -Before using this module you need to register an account with Venafi, and -configure it in your ``master`` configuration file. +:depends: - vcert Python module -First, you need to add a placeholder to the ``master`` file. This is because -the module will not load unless it finds an ``api_key`` setting, valid or not. -Open up ``/etc/salt/master`` and add: +:configuration: In order to connect to Venafi services you need to specify it in + Salt master configuration. + Example for Venafi Cloud (using env variables): -.. code-block:: yaml + .. code-block:: yaml venafi: - api_key: None + api_key: "sdb://osenv/CLOUDAPIKEY" -Then register your email address with Venafi using the following command: + Example for Venafi Platform (using env variables): -.. code-block:: bash - - salt-run venafi.register - -This command will not return an ``api_key`` to you; that will be sent to you -via email from Venafi. Once you have received that key, open up your ``master`` -file and set the ``api_key`` to it: - -.. code-block:: yaml + .. code-block:: yaml venafi: - api_key: abcdef01-2345-6789-abcd-ef0123456789 -''' + base_url: "https://tpp.example.com/" + tpp_user: admin + tpp_password: "sdb://osenv/TPP_PASSWORD" + trust_bundle: "/opt/venafi/bundle.pem" + +""" from __future__ import absolute_import, print_function, unicode_literals import logging -import os -import tempfile +import time -try: - from M2Crypto import RSA - HAS_M2 = True -except ImportError: - HAS_M2 = False - try: - from Cryptodome.PublicKey import RSA - except ImportError: - from Crypto.PublicKey import RSA # Import Salt libs +import sys import salt.cache import salt.syspaths as syspaths import salt.utils.files @@ -54,311 +40,131 @@ # Import 3rd-party libs from salt.ext import six +try: + import vcert + from vcert.common import CertificateRequest + HAS_VCERT = True +except ImportError: + HAS_VCERT = False +CACHE_BANK_NAME = 'venafi/domains' __virtualname__ = 'venafi' log = logging.getLogger(__name__) +def _init_connection(): + log.info("Initializing Venafi Trust Platform or Venafi Cloud connection") + api_key = __opts__.get('venafi', {}).get('api_key', '') + base_url = __opts__.get('venafi', {}).get('base_url', '') + log.info("Using base_url: %s", base_url) + tpp_user = __opts__.get('venafi', {}).get('tpp_user', '') + tpp_password = __opts__.get('venafi', {}).get('tpp_password', '') + trust_bundle = __opts__.get('venafi', {}).get('trust_bundle', '') + fake = __opts__.get('venafi', {}).get('fake', '') + log.info("Finished config processing") + if fake: + return vcert.Connection(fake=True) + elif trust_bundle: + log.info("Will use trust bundle from file %s", trust_bundle) + return vcert.Connection(url=base_url, token=api_key, user=tpp_user, password=tpp_password, + http_request_kwargs={"verify": trust_bundle}) + else: + return vcert.Connection(url=base_url, token=api_key, user=tpp_user, password=tpp_password) + + def __virtual__(): - ''' - Only load the module if venafi is installed - ''' - if __opts__.get('venafi', {}).get('api_key'): + """ + Only load the module if vcert module is installed + """ + if HAS_VCERT: return __virtualname__ return False -def _base_url(): - ''' - Return the base_url - ''' - return __opts__.get('venafi', {}).get( - 'base_url', 'https://api.venafi.cloud/v1' - ) - - -def _api_key(): - ''' - Return the API key - ''' - return __opts__.get('venafi', {}).get('api_key', '') - - -def gen_key(minion_id, dns_name=None, zone='default', password=None): - ''' - Generate and return an private_key. If a ``dns_name`` is passed in, the - private_key will be cached under that name. The type of key and the - parameters used to generate the key are based on the default certificate - use policy associated with the specified zone. - - CLI Example: - - .. code-block:: bash - - salt-run venafi.gen_key [dns_name] [zone] [password] - ''' - # Get the default certificate use policy associated with the zone - # so we can generate keys that conform with policy - - # The /v1/zones/tag/{name} API call is a shortcut to get the zoneID - # directly from the name - - qdata = __utils__['http.query']( - '{0}/zones/tag/{1}'.format(_base_url(), zone), - method='GET', - decode=True, - decode_type='json', - header_dict={ - 'tppl-api-key': _api_key(), - 'Content-Type': 'application/json', - }, - ) - - zone_id = qdata['dict']['id'] - - # the /v1/certificatepolicies?zoneId API call returns the default - # certificate use and certificate identity policies - - qdata = __utils__['http.query']( - '{0}/certificatepolicies?zoneId={1}'.format(_base_url(), zone_id), - method='GET', - decode=True, - decode_type='json', - header_dict={ - 'tppl-api-key': _api_key(), - 'Content-Type': 'application/json', - }, - ) - - policies = qdata['dict']['certificatePolicies'] - - # Extract the key length and key type from the certificate use policy - # and generate the private key accordingly - - for policy in policies: - if policy['certificatePolicyType'] == "CERTIFICATE_USE": - keyTypes = policy['keyTypes'] - # in case multiple keytypes and key lengths are supported - # always use the first key type and key length - keygen_type = keyTypes[0]['keyType'] - key_len = keyTypes[0]['keyLengths'][0] - - if int(key_len) < 2048: - key_len = 2048 - - if keygen_type == "RSA": - if HAS_M2: - gen = RSA.gen_key(key_len, 65537) - private_key = gen.as_pem(cipher='des_ede3_cbc', callback=lambda x: six.b(password)) - else: - gen = RSA.generate(bits=key_len) - private_key = gen.exportKey('PEM', password) - if dns_name is not None: - bank = 'venafi/domains' - cache = salt.cache.Cache(__opts__, syspaths.CACHE_DIR) - try: - data = cache.fetch(bank, dns_name) - data['private_key'] = private_key - data['minion_id'] = minion_id - except TypeError: - data = {'private_key': private_key, - 'minion_id': minion_id} - cache.store(bank, dns_name, data) - return private_key - - -def gen_csr( - minion_id, - dns_name, - zone='default', - country=None, - state=None, - loc=None, - org=None, - org_unit=None, - password=None, - ): - ''' - Generate a csr using the host's private_key. - Analogous to: - - .. code-block:: bash - - VCert gencsr -cn [CN Value] -o "Beta Organization" -ou "Beta Group" \ - -l "Palo Alto" -st "California" -c US - - CLI Example: - - .. code-block:: bash - - salt-run venafi.gen_csr - ''' - tmpdir = tempfile.mkdtemp() - os.chmod(tmpdir, 0o700) - - bank = 'venafi/domains' - cache = salt.cache.Cache(__opts__, syspaths.CACHE_DIR) - data = cache.fetch(bank, dns_name) - if data is None: - data = {} - if 'private_key' not in data: - data['private_key'] = gen_key(minion_id, dns_name, zone, password) - - tmppriv = '{0}/priv'.format(tmpdir) - tmpcsr = '{0}/csr'.format(tmpdir) - with salt.utils.files.fopen(tmppriv, 'w') as if_: - if_.write(salt.utils.stringutils.to_str(data['private_key'])) - - if country is None: - country = __opts__.get('venafi', {}).get('country') - - if state is None: - state = __opts__.get('venafi', {}).get('state') - - if loc is None: - loc = __opts__.get('venafi', {}).get('loc') - - if org is None: - org = __opts__.get('venafi', {}).get('org') - - if org_unit is None: - org_unit = __opts__.get('venafi', {}).get('org_unit') - - subject = '/C={0}/ST={1}/L={2}/O={3}/OU={4}/CN={5}'.format( - country, - state, - loc, - org, - org_unit, - dns_name, - ) - - cmd = "openssl req -new -sha256 -key {0} -out {1} -subj '{2}'".format( - tmppriv, - tmpcsr, - subject - ) - if password is not None: - cmd += ' -passin pass:{0}'.format(password) - output = __salt__['salt.cmd']('cmd.run', cmd) - - if 'problems making Certificate Request' in output: - raise CommandExecutionError( - 'There was a problem generating the CSR. Please ensure that you ' - 'have the following variables set either on the command line, or ' - 'in the venafi section of your master configuration file: ' - 'country, state, loc, org, org_unit' - ) - - with salt.utils.files.fopen(tmpcsr, 'r') as of_: - csr = salt.utils.stringutils.to_unicode(of_.read()) - - data['minion_id'] = minion_id - data['csr'] = csr - cache.store(bank, dns_name, data) - return csr - - def request( - minion_id, - dns_name=None, - zone='default', - request_id=None, - country='US', - state='California', - loc='Palo Alto', - org='Beta Organization', - org_unit='Beta Group', - password=None, - zone_id=None, - ): - ''' + minion_id, + dns_name=None, + zone=None, + country=None, + state=None, + loc=None, + org=None, + org_unit=None, + key_password=None, + csr_path=None, + pkey_path=None, +): + """ Request a new certificate - Uses the following command: - - .. code-block:: bash - - VCert enroll -z -k -cn - CLI Example: .. code-block:: bash salt-run venafi.request - ''' - if password is not None: - if password.startswith('sdb://'): - password = __salt__['sdb.get'](password) - - if zone_id is None: - zone_id = __opts__.get('venafi', {}).get('zone_id') - - if zone_id is None and zone is not None: - zone_id = get_zone_id(zone) - - if zone_id is None: - raise CommandExecutionError( - 'Either a zone or a zone_id must be passed in or ' - 'configured in the master file. This id can be retreived using ' - 'venafi.show_company ' - ) - - private_key = gen_key(minion_id, dns_name, zone, password) - - csr = gen_csr( - minion_id, - dns_name, - zone=zone, - country=country, - state=state, - loc=loc, - org=org, - org_unit=org_unit, - password=password, - ) - - pdata = salt.utils.json.dumps({ - 'zoneId': zone_id, - 'certificateSigningRequest': csr, - }) - - qdata = __utils__['http.query']( - '{0}/certificaterequests'.format(_base_url()), - method='POST', - data=pdata, - decode=True, - decode_type='json', - header_dict={ - 'tppl-api-key': _api_key(), - 'Content-Type': 'application/json', - }, - ) - - request_id = qdata['dict']['certificateRequests'][0]['id'] - ret = { - 'request_id': request_id, - 'private_key': private_key, - 'csr': csr, - 'zone': zone, - } + """ + + log.info("Requesting Venafi certificate") + if zone is None: + log.error(msg=str("Missing zone parameter")) + sys.exit(1) + + if key_password is not None: + if key_password.startswith('sdb://'): + key_password = __salt__['sdb.get'](key_password) + conn = _init_connection() + + if csr_path is None: + request = CertificateRequest(common_name=dns_name, country=country, province=state, locality=loc, + organization=org, organizational_unit=org_unit, key_password=key_password) + zone_config = conn.read_zone_conf(zone) + log.info("Updating request from zone %s", zone_config) + request.update_from_zone_config(zone_config) + else: + log.info("Will use generated CSR from %s", csr_path) + log.info("Using CN %s", dns_name) + try: + with salt.utils.files.fopen(csr_path) as csr_file: + csr = csr_file.read() + request = CertificateRequest(csr=csr, common_name=dns_name) + except Exception as e: + raise Exception( + 'Unable to open file {file}: {excp}'.format(file=csr_path, excp=e) + ) + conn.request_cert(request, zone) + + # TODO: add timeout parameter here + timeout_seconds = 300 + timeout = time.time() + timeout_seconds + cert = None + while cert is None and time.time() < timeout: + cert = conn.retrieve_cert(request) + if cert is None: + time.sleep(5) + + if csr_path is None: + private_key = request.private_key_pem + else: + if pkey_path: + try: + with salt.utils.files.fopen(pkey_path) as pkey_file: + private_key = pkey_file.read() + except Exception as e: + raise Exception( + 'Unable to open file {file}: {excp}'.format(file=pkey_path, excp=e) + ) + else: + private_key = None - bank = 'venafi/domains' cache = salt.cache.Cache(__opts__, syspaths.CACHE_DIR) - data = cache.fetch(bank, dns_name) - if data is None: - data = {} - data.update({ + data = { 'minion_id': minion_id, - 'request_id': request_id, - 'private_key': private_key, - 'zone': zone, - 'csr': csr, - }) - cache.store(bank, dns_name, data) - _id_map(minion_id, dns_name) - - return ret + 'cert': cert.cert, + 'chain': cert.chain, + 'pkey': private_key + } + cache.store(CACHE_BANK_NAME, dns_name, data) + return cert.cert, private_key # Request and renew are the same, so far as this module is concerned @@ -366,268 +172,38 @@ def request( def _id_map(minion_id, dns_name): - ''' - Maintain a relationship between a minion and a dns name - ''' - bank = 'venafi/minions' + """ + Maintain a relationship between a minion and a DNS name + """ + cache = salt.cache.Cache(__opts__, syspaths.CACHE_DIR) - dns_names = cache.fetch(bank, minion_id) + dns_names = cache.fetch(CACHE_BANK_NAME, minion_id) if not isinstance(dns_names, list): dns_names = [] if dns_name not in dns_names: dns_names.append(dns_name) - cache.store(bank, minion_id, dns_names) + cache.store(CACHE_BANK_NAME, minion_id, dns_names) -def register(email): - ''' - Register a new user account +def show_cert(dns_name): + """ + Show issued certificate for domain CLI Example: .. code-block:: bash - salt-run venafi.register email@example.com - ''' - data = __utils__['http.query']( - '{0}/useraccounts'.format(_base_url()), - method='POST', - data=salt.utils.json.dumps({ - 'username': email, - 'userAccountType': 'API', - }), - status=True, - decode=True, - decode_type='json', - header_dict={ - 'Content-Type': 'application/json', - }, - ) - status = data['status'] - if six.text_type(status).startswith('4') or six.text_type(status).startswith('5'): - raise CommandExecutionError( - 'There was an API error: {0}'.format(data['error']) - ) - return data.get('dict', {}) - - -def show_company(domain): - ''' - Show company information, especially the company id - - CLI Example: - - .. code-block:: bash - - salt-run venafi.show_company example.com - ''' - data = __utils__['http.query']( - '{0}/companies/domain/{1}'.format(_base_url(), domain), - status=True, - decode=True, - decode_type='json', - header_dict={ - 'tppl-api-key': _api_key(), - }, - ) - status = data['status'] - if six.text_type(status).startswith('4') or six.text_type(status).startswith('5'): - raise CommandExecutionError( - 'There was an API error: {0}'.format(data['error']) - ) - return data.get('dict', {}) - - -def show_csrs(): - ''' - Show certificate requests for this API key - - CLI Example: - - .. code-block:: bash - - salt-run venafi.show_csrs - ''' - data = __utils__['http.query']( - '{0}/certificaterequests'.format(_base_url()), - status=True, - decode=True, - decode_type='json', - header_dict={ - 'tppl-api-key': _api_key(), - }, - ) - status = data['status'] - if six.text_type(status).startswith('4') or six.text_type(status).startswith('5'): - raise CommandExecutionError( - 'There was an API error: {0}'.format(data['error']) - ) - return data.get('dict', {}) - - -def get_zone_id(zone_name): - ''' - Get the zone ID for the given zone name - - CLI Example: - - .. code-block:: bash - - salt-run venafi.get_zone_id default - ''' - data = __utils__['http.query']( - '{0}/zones/tag/{1}'.format(_base_url(), zone_name), - status=True, - decode=True, - decode_type='json', - header_dict={ - 'tppl-api-key': _api_key(), - }, - ) - - status = data['status'] - if six.text_type(status).startswith('4') or six.text_type(status).startswith('5'): - raise CommandExecutionError( - 'There was an API error: {0}'.format(data['error']) - ) - return data['dict']['id'] - - -def show_policies(): - ''' - Show zone details for the API key owner's company - - CLI Example: - - .. code-block:: bash - - salt-run venafi.show_zones - ''' - data = __utils__['http.query']( - '{0}/certificatepolicies'.format(_base_url()), - status=True, - decode=True, - decode_type='json', - header_dict={ - 'tppl-api-key': _api_key(), - }, - ) - status = data['status'] - if six.text_type(status).startswith('4') or six.text_type(status).startswith('5'): - raise CommandExecutionError( - 'There was an API error: {0}'.format(data['error']) - ) - return data['dict'] - - -def show_zones(): - ''' - Show zone details for the API key owner's company - - CLI Example: - - .. code-block:: bash - - salt-run venafi.show_zones - ''' - data = __utils__['http.query']( - '{0}/zones'.format(_base_url()), - status=True, - decode=True, - decode_type='json', - header_dict={ - 'tppl-api-key': _api_key(), - }, - ) - status = data['status'] - if six.text_type(status).startswith('4') or six.text_type(status).startswith('5'): - raise CommandExecutionError( - 'There was an API error: {0}'.format(data['error']) - ) - return data['dict'] - - -def show_cert(id_): - ''' - Show certificate requests for this API key - - CLI Example: - - .. code-block:: bash - - salt-run venafi.show_cert 01234567-89ab-cdef-0123-456789abcdef - ''' - data = __utils__['http.query']( - '{0}/certificaterequests/{1}/certificate'.format(_base_url(), id_), - params={ - 'format': 'PEM', - 'chainOrder': 'ROOT_FIRST' - }, - status=True, - text=True, - header_dict={'tppl-api-key': _api_key()}, - ) - status = data['status'] - if six.text_type(status).startswith('4') or six.text_type(status).startswith('5'): - raise CommandExecutionError( - 'There was an API error: {0}'.format(data['error']) - ) - data = data.get('body', '') - csr_data = __utils__['http.query']( - '{0}/certificaterequests/{1}'.format(_base_url(), id_), - status=True, - decode=True, - decode_type='json', - header_dict={'tppl-api-key': _api_key()}, - ) - status = csr_data['status'] - if six.text_type(status).startswith('4') or six.text_type(status).startswith('5'): - raise CommandExecutionError( - 'There was an API error: {0}'.format(csr_data['error']) - ) - csr_data = csr_data.get('dict', {}) - certs = _parse_certs(data) - dns_name = '' - for item in csr_data['certificateName'].split(','): - if item.startswith('cn='): - dns_name = item.split('=')[1] - #certs['CSR Data'] = csr_data + salt-run venafi.show_cert example.com + """ cache = salt.cache.Cache(__opts__, syspaths.CACHE_DIR) - domain_data = cache.fetch('venafi/domains', dns_name) - if domain_data is None: - domain_data = {} - certs['private_key'] = domain_data.get('private_key') - domain_data.update(certs) - cache.store('venafi/domains', dns_name, domain_data) - - certs['request_id'] = id_ - return certs - - -pickup = show_cert - - -def show_rsa(minion_id, dns_name): - ''' - Show a private RSA key - - CLI Example: - - .. code-block:: bash - - salt-run venafi.show_rsa myminion domain.example.com - ''' - cache = salt.cache.Cache(__opts__, syspaths.CACHE_DIR) - bank = 'venafi/domains' - data = cache.fetch( - bank, dns_name - ) - return data['private_key'] + domain_data = cache.fetch(CACHE_BANK_NAME, dns_name) or {} + cert = domain_data.get('cert') + return cert def list_domain_cache(): - ''' + """ List domains that have been cached CLI Example: @@ -635,13 +211,13 @@ def list_domain_cache(): .. code-block:: bash salt-run venafi.list_domain_cache - ''' + """ cache = salt.cache.Cache(__opts__, syspaths.CACHE_DIR) return cache.list('venafi/domains') def del_cached_domain(domains): - ''' + """ Delete cached domains from the master CLI Example: @@ -649,7 +225,7 @@ def del_cached_domain(domains): .. code-block:: bash salt-run venafi.del_cached_domain domain1.example.com,domain2.example.com - ''' + """ cache = salt.cache.Cache(__opts__, syspaths.CACHE_DIR) if isinstance(domains, six.string_types): domains = domains.split(',') @@ -662,51 +238,8 @@ def del_cached_domain(domains): failed = [] for domain in domains: try: - cache.flush('venafi/domains', domain) + cache.flush(CACHE_BANK_NAME, domain) success.append(domain) except CommandExecutionError: failed.append(domain) return {'Succeeded': success, 'Failed': failed} - - -def _parse_certs(data): - cert_mode = False - cert = '' - certs = [] - rsa_key = '' - for line in data.splitlines(): - if not line.strip(): - continue - if 'Successfully posted request' in line: - comps = line.split(' for ') - request_id = comps[-1].strip() - continue - if 'END CERTIFICATE' in line or 'END RSA private_key' in line: - if 'RSA' in line: - rsa_key = rsa_key + line - else: - cert = cert + line - certs.append(cert) - cert_mode = False - continue - if 'BEGIN CERTIFICATE' in line or 'BEGIN RSA private_key' in line: - if 'RSA' in line: - rsa_key = line + '\n' - else: - cert = line + '\n' - cert_mode = True - continue - if cert_mode is True: - cert = cert + line + '\n' - continue - - rcert = certs.pop(0) - eecert = certs.pop(-1) - ret = { - 'end_entity_certificate': eecert, - 'private_key': rsa_key, - 'root_certificate': rcert, - 'intermediate_certificates': certs - } - - return ret diff --git a/tests/integration/externalapi/test_venafiapi.py b/tests/integration/externalapi/test_venafiapi.py index ee1a9edd495d..9c2bb8e4d2af 100644 --- a/tests/integration/externalapi/test_venafiapi.py +++ b/tests/integration/externalapi/test_venafiapi.py @@ -8,10 +8,16 @@ import random import string + # Import Salt Testing libs from tests.support.case import ShellCase -from tests.support.helpers import destructiveTest, expensiveTest from salt.ext.six.moves import range +from salt.ext.six import text_type +from cryptography import x509 +from cryptography.hazmat.backends import default_backend +from cryptography.x509.oid import NameOID +from cryptography.hazmat.primitives import serialization +import tempfile def _random_name(prefix=''): @@ -25,70 +31,117 @@ def with_random_name(func): ''' generate a randomized name for a container ''' + @functools.wraps(func) def wrapper(self, *args, **kwargs): name = _random_name(prefix='salt_') - return func(self, _random_name(prefix='salt_test_'), *args, **kwargs) + return func(self, _random_name(prefix='salt-test-'), *args, **kwargs) + return wrapper -@destructiveTest -@expensiveTest class VenafiTest(ShellCase): ''' Test the venafi runner ''' @with_random_name - def test_gen_key_password(self, name): - ''' - venafi.gen_key - ''' - ret = self.run_run_plus(fun='venafi.gen_key', - minion_id='{0}.test.saltstack.com'.format(name), - dns_name='{0}.test.saltstack.com'.format(name), - zone='Internet', - password='SecretSauce') - self.assertEqual(ret['out'][0], '-----BEGIN RSA PRIVATE KEY-----') - self.assertEqual(ret['out'][1], 'Proc-Type: 4,ENCRYPTED') - self.assertEqual(ret['out'][-1], '-----END RSA PRIVATE KEY-----') + def test_request(self, name): + cn = '{0}.example.com'.format(name) - @with_random_name - def test_gen_key_without_password(self, name): - ''' - venafi.gen_key - ''' - ret = self.run_run_plus(fun='venafi.gen_key', - minion_id='{0}.test.saltstack.com'.format(name), - dns_name='{0}.test.saltstack.com'.format(name), - zone='Internet') - self.assertEqual(ret['out'][0], '-----BEGIN RSA PRIVATE KEY-----') - self.assertNotEqual(ret['out'][1], 'Proc-Type: 4,ENCRYPTED') - self.assertEqual(ret['out'][-1], '-----END RSA PRIVATE KEY-----') + # Provide python27 compatibility + if not isinstance(cn, text_type): + cn = cn.decode() - @with_random_name - def test_gen_csr(self, name): - ''' - venafi.gen_csr - ''' - ret = self.run_run_plus(fun='venafi.gen_csr', - minion_id='{0}.test.saltstack.com'.format(name), - dns_name='{0}.test.saltstack.com'.format(name), - country='US', state='Utah', loc='Salt Lake City', - org='Salt Stack Inc.', org_unit='Testing', - zone='Internet', password='SecretSauce') - self.assertEqual(ret['out'][0], '-----BEGIN CERTIFICATE REQUEST-----') - self.assertEqual(ret['out'][-1], '-----END CERTIFICATE REQUEST-----') + ret = self.run_run_plus(fun='venafi.request', + minion_id=cn, + dns_name=cn, + key_password='secretPassword', + zone='fake') + cert_output = ret['return'][0] + assert cert_output is not None, 'venafi_certificate not found in `output_value`' + + cert = x509.load_pem_x509_certificate(cert_output.encode(), default_backend()) + assert isinstance(cert, x509.Certificate) + assert cert.subject.get_attributes_for_oid(NameOID.COMMON_NAME) == [ + x509.NameAttribute( + NameOID.COMMON_NAME, cn + ) + ] + + pkey_output = ret['return'][1] + assert pkey_output is not None, 'venafi_private key not found in output_value' + + pkey = serialization.load_pem_private_key(pkey_output.encode(), password=b'secretPassword', + backend=default_backend()) + + pkey_public_key_pem = pkey.public_key().public_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PublicFormat.SubjectPublicKeyInfo + ) + cert_public_key_pem = cert.public_key().public_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PublicFormat.SubjectPublicKeyInfo + ) + assert pkey_public_key_pem == cert_public_key_pem @with_random_name - def test_request(self, name): - ''' - venafi.request - ''' - ret = self.run_run_plus(fun='venafi.request', - minion_id='{0}.example.com'.format(name), - dns_name='{0}.example.com'.format(name), - country='US', state='Utah', loc='Salt Lake City', - org='Salt Stack Inc.', org_unit='Testing', - zone='Internet', password='SecretSauce') - self.assertTrue('request_id' in ret['return']) + def test_sign(self, name): + + csr_pem = """-----BEGIN CERTIFICATE REQUEST----- +MIIFbDCCA1QCAQAwgbQxCzAJBgNVBAYTAlVTMQ0wCwYDVQQIDARVdGFoMRIwEAYD +VQQHDAlTYWx0IExha2UxFDASBgNVBAoMC1ZlbmFmaSBJbmMuMRQwEgYDVQQLDAtJ +bnRlZ3JhdGlvbjEnMCUGCSqGSIb3DQEJARYYZW1haWxAdmVuYWZpLmV4YW1wbGUu +Y29tMS0wKwYDVQQDDCR0ZXN0LWNzci0zMjMxMzEzMS52ZW5hZmkuZXhhbXBsZS5j +b20wggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC4T0bdjq+mF+DABhF+ +XWCwOXXUWbPNWa72VVhxoelbyTS0iIeZEe64AvNGykytFdOuT/F9pdkZa+Io07R1 +ZMp6Ak8dp2Wjt4c5rayVZus6ZK+0ZwBRJO7if/cqhEpxy8Wz1RMfVLf2AE1u/xZS +QSYY0BTRWGmPqrFJrIGbnyQfvmGVPk3cA0RfdrwYJZXtZ2/4QNrbNCoSoSmqTHzt +NAtZhvT2dPU9U48Prx4b2460x+ck3xA1OdJNXV7n5u53QbxOIcjdGT0lJ62ml70G +5gvEHmdPcg+t5cw/Sm5cfDSUEDtNEXvD4oJXfP98ty6f1cYsZpcrgxRwk9RfGain +hvoweXhZP3NWnU5nRdn2nOfExv+xMeQOyB/rYv98zqzK6LvwKhwI5UB1l/n9KTpg +jgaNCP4x/KAsrPecbHK91oiqGSbPn4wtTYOmPkDxSzATN317u7fE20iqvVAUy/O+ +7SCNNKEDPX2NP9LLz0IPK0roQxLiwd2CVyN6kEXuzs/3psptkNRMSlhyeAZdfrOE +CNOp46Pam9f9HGBqzXxxoIlfzLqHHL584kgFlBm7qmivVrgp6zdLPDa+UayXEl2N +O17SnGS8nkOTmfg3cez7lzX/LPLO9X/Y1xKYqx5hoGZhh754K8mzDWCVCYThWgou +yBOYY8uNXiX6ldqzQUHpbxxQgwIDAQABoHIwcAYJKoZIhvcNAQkOMWMwYTBfBgNV +HREEWDBWgilhbHQxLXRlc3QtY3NyLTMyMzEzMTMxLnZlbmFmaS5leGFtcGxlLmNv +bYIpYWx0Mi10ZXN0LWNzci0zMjMxMzEzMS52ZW5hZmkuZXhhbXBsZS5jb20wDQYJ +KoZIhvcNAQELBQADggIBAJd87BIdeh0WWoyQ4IX+ENpNqmm/sLmdfmUB/hj9NpBL +qbr2UTWaSr1jadoZ+mrDxtm1Z0YJDTTIrEWxkBOW5wQ039lYZNe2tfDXSJZwJn7u +2keaXtWQ2SdduK1wOPDO9Hra6WnH7aEq5D1AyoghvPsZwTqZkNynt/A1BZW5C/ha +J9/mwgWfL4qXBGBOhLwKN5GUo3erUkJIdH0TlMqI906D/c/YAuJ86SRdQtBYci6X +bJ7C+OnoiV6USn1HtQE6dfOMeS8voJuixpSIvHZ/Aim6kSAN1Za1f6FQAkyqbF+o +oKTJHDS1CPWikCeLdpPUcOCDIbsiISTsMZkEvIkzZ7dKBIlIugauxw3vaEpk47jN +Wq09r639RbSv/Qs8D6uY66m1IpL4zHm4lTAknrjM/BqihPxc8YiN76ssajvQ4SFT +DHPrDweEVe4KL1ENw8nv4wdkIFKwJTDarV5ZygbETzIhfa2JSBZFTdN+Wmd2Mh5h +OTu+vuHrJF2TO8g1G48EB/KWGt+yvVUpWAanRMwldnFX80NcUlM7GzNn6IXTeE+j +BttIbvAAVJPG8rVCP8u3DdOf+vgm5macj9oLoVP8RBYo/z0E3e+H50nXv3uS6JhN +xlAKgaU6i03jOm5+sww5L2YVMi1eeBN+kx7o94ogpRemC/EUidvl1PUJ6+e7an9V +-----END CERTIFICATE REQUEST----- + """ + + with tempfile.NamedTemporaryFile('w+') as f: + f.write(csr_pem) + f.flush() + csr_path = f.name + cn = "test-csr-32313131.venafi.example.com" + + # Provide python27 compatibility + if not isinstance(cn, text_type): + cn = cn.decode() + + ret = self.run_run_plus(fun='venafi.request', + minion_id=cn, + csr_path=csr_path, + zone='fake') + cert_output = ret['return'][0] + assert cert_output is not None, 'venafi_certificate not found in `output_value`' + + cert = x509.load_pem_x509_certificate(cert_output.encode(), default_backend()) + assert isinstance(cert, x509.Certificate) + assert cert.subject.get_attributes_for_oid(NameOID.COMMON_NAME) == [ + x509.NameAttribute( + NameOID.COMMON_NAME, cn + ) + ] diff --git a/tests/integration/files/conf/master b/tests/integration/files/conf/master index 1b639a522d63..25b4378bef51 100644 --- a/tests/integration/files/conf/master +++ b/tests/integration/files/conf/master @@ -128,3 +128,5 @@ peer_run: - vault.generate_token sdbvault: driver: vault +venafi: + fake: "true" \ No newline at end of file