Merge branch 'master' into backport_49981

CHANGELOG.md

@@ -21,6 +21,7 @@ Versions are `MAJOR.PATCH`.
 
 ### Added
 - [#56627](https://github.com/saltstack/salt/pull/56627) - Add new salt-ssh set_path option
+- [#51379](https://github.com/saltstack/salt/pull/56792) - Backport 51379 : Adds .set_domain_workgroup to win_system
 
 ## 3000.1
 

conf/master

@@ -677,7 +677,9 @@
 
 # The master_roots setting configures a master-only copy of the file_roots dictionary,
 # used by the state compiler.
-#master_roots: /srv/salt-master
+#master_roots:
+#  base:
+#    - /srv/salt-master
 
 # When using multiple environments, each with their own top file, the
 # default behaviour is an unordered merge. To prevent top files from

doc/man/salt.7

@@ -284897,7 +284897,7 @@ new
 all
 .TP
 .B note
-If you see the following error, you\(aqll need to upgrade \fBrequests\fP to atleast 2.4.2
+If you see the following error, you\(aqll need to upgrade \fBrequests\fP to at least 2.4.2
 .UNINDENT
 .INDENT 0.0
 .INDENT 3.5

doc/ref/configuration/master.rst

@@ -2654,14 +2654,18 @@ nothing is ignored.
 ``master_roots``
 ----------------
 
-Default: ``/srv/salt-master``
+Default: ``''``
 
 A master-only copy of the :conf_master:`file_roots` dictionary, used by the
 state compiler.
 
+Example:
+
 .. code-block:: yaml
 
-    master_roots: /srv/salt-master
+    master_roots:
+      base:
+        - /srv/salt-master
 
 roots: Master's Local File Server
 ---------------------------------

doc/topics/development/modules/index.rst

@@ -69,16 +69,6 @@ dynamic modules when states are run. To disable this behavior set
 When dynamic modules are autoloaded via states, only the modules defined in the
 same saltenvs as the states currently being run.
 
-Also it is possible to use the explicit ``saltutil.sync_*`` :py:mod:`state functions <salt.states.saltutil>`
-to sync the modules (previously it was necessary to use the ``module.run`` state):
-
-.. code-block::yaml
-
-   synchronize_modules:
-     saltutil.sync_modules:
-       - refresh: True
-
-
 Sync Via the saltutil Module
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -350,7 +340,7 @@ SDB
 
 * :ref:`Writing SDB Modules <sdb-writing-modules>`
 
-SDB is a way to store data that's not associated with a minion. See 
+SDB is a way to store data that's not associated with a minion. See
 :ref:`Storing Data in Other Databases <sdb>`.
 
 Serializer
@@ -394,6 +384,12 @@ pkgfiles modules handle the actual installation.
 SSH Wrapper
 -----------
 
+.. toctree::
+    :maxdepth: 1
+    :glob:
+
+    ssh_wrapper
+
 Replacement execution modules for :ref:`Salt SSH <salt-ssh>`.
 
 Thorium
@@ -420,7 +416,7 @@ the state system.
 Util
 ----
 
-Just utility modules to use with other modules via ``__utils__`` (see 
+Just utility modules to use with other modules via ``__utils__`` (see
 :ref:`Dunder Dictionaries <dunder-dictionaries>`).
 
 Wheel

doc/topics/development/modules/ssh_wrapper.rst

@@ -0,0 +1,63 @@
+.. _ssh-wrapper:
+
+===========
+SSH Wrapper
+===========
+
+Salt-SSH Background
+===================
+
+Salt-SSH works by creating a tar ball of salt, a bunch of python modules, and a generated
+short minion config. It then copies this onto the destination host over ssh, then
+uses that host's local python install to run ``salt-client --local`` with any requested modules.
+It does not automatically copy over states or cache files and since it is uses a local file_client,
+modules that rely on :py:func:`cp.cache* <salt.modules.cp>` functionality do not work.
+
+SSH Wrapper modules
+===================
+
+To support cp modules or other functionality which might not otherwise work in the remote environment,
+a wrapper module can be created. These modules are run from the salt-master initiating the salt-ssh
+command and can include logic to support the needed functionality. SSH Wrapper modules are located in
+/salt/client/ssh/wrapper/ and are named the same as the execution module being extended. Any functions
+defined inside of the wrapper module are called from the ``salt-ssh module.function argument``
+command rather than executing on the minion.
+
+State Module example
+--------------------
+
+Running salt states on an salt-ssh minion, obviously requires the state files themselves. To support this,
+a state module wrapper script exists at salt/client/ssh/wrapper/state.py, and includes standard state
+functions like :py:func:`apply <salt.modules.state.apply>`, :py:func:`sls <salt.modules.state.sls>`,
+and :py:func:`highstate <salt.modules.state.highstate>`. When executing ``salt-ssh minion state.highstate``,
+these wrapper functions are used and include the logic to walk the low_state output for that minion to
+determine files used, gather needed files, tar them together, transfer the tar file to the minion over
+ssh, and run a state on the ssh minion. This state then extracts the tar file, applies the needed states
+and data, and cleans up the transferred files.
+
+Wrapper Handling
+----------------
+
+From the wrapper script any invocations of ``__salt__['some.module']()`` do not run on the master
+which is running the wrapper, but instead magically are invoked on the minion over ssh.
+Should the function being called exist in the wrapper, the wrapper function will be
+used instead.
+
+One way of supporting this workflow may be to create a wrapper function which performs the needed file
+copy operations. Now that files are resident on the ssh minion, the next step is to run the original
+execution module function. But since that function name was already overridden by the wrapper, a
+function alias can be created in the original execution module, which can then be called from the
+wrapper.
+
+Example
+```````
+
+The saltcheck module needs sls and tst files on the minion to function. The invocation of
+:py:func:`saltcheck.run_state_tests <salt.modules.saltcheck.run_state_tests>` is run from
+the wrapper module, and is responsible for performing the needed file copy. The
+:py:func:`saltcheck <salt.modules.saltcheck>` execution module includes an alias line of
+``run_state_tests_ssh = salt.utils.functools.alias_function(run_state_tests, 'run_state_tests_ssh')``
+which creates an alias of ``run_state_tests`` with the name ``run_state_tests_ssh``. At the end of
+the ``run_state_tests`` function in the wrapper module, it then calls
+``__salt__['saltcheck.run_state_tests_ssh']()``. Since this function does not exist in the wrapper script,
+the call is made on the remote minion, which then having the needed files, runs as expected.

doc/topics/event/events.rst

@@ -250,7 +250,7 @@ done at the CLI:
 
     caller = salt.client.Caller()
 
-    ret = called.cmd('event.send',
+    ret = caller.cmd('event.send',
                      'myco/event/success'
                      { 'success': True,
                        'message': "It works!" })

doc/topics/troubleshooting/master.rst

@@ -180,6 +180,7 @@ Results can then be analyzed with `kcachegrind`_ or similar tool.
 
 .. _`kcachegrind`: http://kcachegrind.sourceforge.net/html/Home.html
 
+Make sure you have yappi installed.
 
 On Windows, in the absense of kcachegrind, a simple file-based workflow to create
 profiling graphs could use `gprof2dot`_, `graphviz`_ and this batch file:

doc/topics/tutorials/intro_scale.rst

@@ -272,7 +272,7 @@ system, such as a database.
   data using a returner (instead of the local job cache on disk).
 
 If a master has many accepted keys, it may take a long time to publish a job
-because the master much first determine the matching minions and deliver
+because the master must first determine the matching minions and deliver
 that information back to the waiting client before the job can be published.
 
 To mitigate this, a key cache may be enabled. This will reduce the load

salt/client/__init__.py

@@ -1620,7 +1620,12 @@ def get_cli_event_returns(
                             yield {
                                 id_: {
                                     "out": "no_return",
-                                    "ret": "Minion did not return. [No response]",
+                                    "ret": "Minion did not return. [No response]"
+                                    "\nThe minions may not have all finished running and any "
+                                    "remaining minions will return upon completion. To look "
+                                    "up the return data for this job later, run the following "
+                                    "command:\n\n"
+                                    "salt-run jobs.lookup_jid {0}".format(jid),
                                     "retcode": salt.defaults.exitcodes.EX_GENERIC,
                                 }
                             }

salt/modules/acme.py

@@ -58,6 +58,9 @@
 )
 LE_LIVE = "/etc/letsencrypt/live/"
 
+if salt.utils.platform.is_freebsd():
+    LE_LIVE = "/usr/local" + LE_LIVE
+
 
 def __virtual__():
     """

salt/modules/deb_postgres.py

@@ -34,7 +34,15 @@ def __virtual__():
 
 
 def cluster_create(
-    version, name="main", port=None, locale=None, encoding=None, datadir=None
+    version,
+    name="main",
+    port=None,
+    locale=None,
+    encoding=None,
+    datadir=None,
+    allow_group_access=None,
+    data_checksums=None,
+    wal_segsize=None,
 ):
     """
     Adds a cluster to the Postgres server.
@@ -53,7 +61,9 @@ def cluster_create(
 
         salt '*' postgres.cluster_create '9.3' locale='fr_FR'
 
+        salt '*' postgres.cluster_create '11' data_checksums=True wal_segsize='32'
     """
+
     cmd = [salt.utils.path.which("pg_createcluster")]
     if port:
         cmd += ["--port", six.text_type(port)]
@@ -64,6 +74,15 @@ def cluster_create(
     if datadir:
         cmd += ["--datadir", datadir]
     cmd += [version, name]
+    # initdb-specific options are passed after '--'
+    if allow_group_access or data_checksums or wal_segsize:
+        cmd += ["--"]
+    if allow_group_access is True:
+        cmd += ["--allow-group-access"]
+    if data_checksums is True:
+        cmd += ["--data-checksums"]
+    if wal_segsize:
+        cmd += ["--wal-segsize", wal_segsize]
     cmdstr = " ".join([pipes.quote(c) for c in cmd])
     ret = __salt__["cmd.run_all"](cmdstr, python_shell=False)
     if ret.get("retcode", 0) != 0: