Creation of ceph_test_radios_io_sequence application

PendingReleaseNotes

@@ -279,6 +279,10 @@ CephFS: Disallow delegating preallocated inode ranges to clients. Config
 * RGW: in bucket notifications, the `principalId` inside `ownerIdentity` now contains
   complete user id, prefixed with tenant id
 
+* NFS: The export create/apply of CephFS based exports will now have a additional parameter `cmount_path` under the FSAL block,
+  which specifies the path within the CephFS to mount this export on. If this and the other
+  `EXPORT { FSAL {} }` options are the same between multiple exports, those exports will share a single CephFS client. If not specified, the default is `/`.
+
 >=18.0.0
 
 * The RGW policy parser now rejects unknown principals by default. If you are

README.md

@@ -81,15 +81,15 @@ To build Ceph, follow this procedure:
    contains `do_cmake.sh` and `CONTRIBUTING.rst`.
 2. Run the `do_cmake.sh` script:
 
-    ``./do_cmake.sh``
+       ./do_cmake.sh
 
    ``do_cmake.sh`` by default creates a "debug build" of Ceph, which can be 
    up to five times slower than a non-debug build. Pass 
    ``-DCMAKE_BUILD_TYPE=RelWithDebInfo`` to ``do_cmake.sh`` to create a 
    non-debug build.
 3. Move into the `build` directory:
 
-    ``cd build``
+       cd build
 4. Use the `ninja` buildsystem to build the development environment:
 
        ninja -j3
@@ -120,11 +120,11 @@ To build Ceph, follow this procedure:
 
    To build only certain targets, run a command of the following form:
 
-	``ninja [target name]``
+       ninja [target name]
 
 5. Install the vstart cluster:
 
-	``ninja install``
+       ninja install
 
 ### CMake Options
 

doc/cephfs/fs-volumes.rst

@@ -276,7 +276,7 @@ Use a command of the following form to create a subvolume:
 
 .. prompt:: bash #
 
-   ceph fs subvolume create <vol_name> <subvol_name> [--size <size_in_bytes>] [--group_name <subvol_group_name>] [--pool_layout <data_pool_name>] [--uid <uid>] [--gid <gid>] [--mode <octal_mode>] [--namespace-isolated]
+   ceph fs subvolume create <vol_name> <subvol_name> [--size <size_in_bytes>] [--group_name <subvol_group_name>] [--pool_layout <data_pool_name>] [--uid <uid>] [--gid <gid>] [--mode <octal_mode>] [--namespace-isolated] [--earmark <earmark>]
 
 
 The command succeeds even if the subvolume already exists.
@@ -289,6 +289,15 @@ The subvolume can be created in a separate RADOS namespace by specifying the
 default subvolume group with an octal file mode of ``755``, a uid of its
 subvolume group, a gid of its subvolume group, a data pool layout of its parent
 directory, and no size limit.
+You can also assign an earmark to a subvolume using the ``--earmark`` option.
+The earmark is a unique identifier that tags the subvolume for specific purposes,
+such as NFS or SMB services. By default, no earmark is set, allowing for flexible
+assignment based on administrative needs. An empty string ("") can be used to remove
+any existing earmark from a subvolume.
+
+The earmarking mechanism ensures that subvolumes are correctly tagged and managed,
+helping to avoid conflicts and ensuring that each subvolume is associated
+with the intended service or use case.
 
 Removing a subvolume
 ~~~~~~~~~~~~~~~~~~~~
@@ -418,6 +427,7 @@ The output format is JSON and contains the following fields.
 * ``pool_namespace``: RADOS namespace of the subvolume
 * ``features``: features supported by the subvolume
 * ``state``: current state of the subvolume
+* ``earmark``: earmark of the subvolume
 
 If a subvolume has been removed but its snapshots have been retained, the
 output contains only the following fields.
@@ -522,6 +532,33 @@ subvolume using the metadata key:
 Using the ``--force`` flag allows the command to succeed when it would
 otherwise fail (if the metadata key did not exist).
 
+Getting earmark of a subvolume
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Use a command of the following form to get the earmark of a subvolume:
+
+.. prompt:: bash #
+
+   ceph fs subvolume earmark get <vol_name> <subvol_name> [--group_name <subvol_group_name>]
+
+Setting earmark of a subvolume
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Use a command of the following form to set the earmark of a subvolume:
+
+.. prompt:: bash #
+
+   ceph fs subvolume earmark set <vol_name> <subvol_name> [--group_name <subvol_group_name>] <earmark>
+
+Removing earmark of a subvolume
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Use a command of the following form to remove the earmark of a subvolume:
+
+.. prompt:: bash #
+
+   ceph fs subvolume earmark rm <vol_name> <subvol_name> [--group_name <subvol_group_name>]
+
 Creating a Snapshot of a Subvolume
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

doc/dev/developer_guide/testing_integration_tests/tests-integration-testing-teuthology-intro.rst

@@ -317,16 +317,16 @@ directory tree within the ``suites/`` subdirectory of the `ceph/qa sub-directory
 The set of all tests defined by a given subdirectory of ``suites/`` is
 called an "integration test suite", or a "teuthology suite".
 
-Combination of yaml facets is controlled by special files (``%`` and
-``+``) that are placed within the directory tree and can be thought of as
-operators.  The ``%`` file is the "convolution" operator and ``+``
-signifies concatenation.
+Combination of YAML facets is controlled by special files (``%``, ``+`` and ``$``)
+that are placed within the directory tree and can be thought of as
+operators.  The ``%`` file is the "convolution" operator, ``+`` signifies
+concatenation and ``$`` is the "random selection" operator.
 
-Convolution operator
-^^^^^^^^^^^^^^^^^^^^
+Convolution operator - ``%``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The convolution operator, implemented as a (typically empty) file called ``%``,
-tells teuthology to construct a test matrix from yaml facets found in
+tells teuthology to construct a test matrix from YAML facets found in
 subdirectories below the directory containing the operator.
 
 For example, the `ceph-deploy suite
@@ -421,8 +421,8 @@ tests will still preserve the correct numerator (subset of subsets).
 You can disable nested subsets using the ``--no-nested-subset`` argument to
 ``teuthology-suite``.
 
-Concatenation operator
-^^^^^^^^^^^^^^^^^^^^^^
+Concatenation operator - ``+``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 For even greater flexibility in sharing yaml files between suites, the
 special file plus (``+``) can be used to concatenate files within a
@@ -561,6 +561,15 @@ rest of the cluster (``5-finish-upgrade.yaml``).
 The last stage is requiring the updated release (``ceph require-osd-release quincy``,
 ``ceph osd set-require-min-compat-client quincy``) and running the ``final-workload``.
 
+Random Selection Operator - ``$``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The presence of a file named ``$`` provides a hint to teuthology to randomly
+include one of the YAML fragments in the test. Such a scenario is typically
+seen when we need to choose one of the flavors of the features/options to be
+tested randomly.
+
+
 Position Independent Linking
 ----------------------------
 

doc/man/8/rbd.rst

@@ -543,8 +543,9 @@ Commands
 
 :command:`mirror pool info` [*pool-name*]
   Show information about the pool or namespace mirroring configuration.
-  For a pool, it includes mirroring mode, peer UUID, remote cluster name,
-  and remote client name. For a namespace, it includes only mirroring mode.
+  For both pools and namespaces, it includes the mirroring mode, mirror UUID
+  and remote namespace. For pools, it additionally includes the site name,
+  peer UUID, remote cluster name, and remote client name.
 
 :command:`mirror pool peer add` [*pool-name*] *remote-cluster-spec*
   Add a mirroring peer to a pool.
@@ -555,7 +556,7 @@ Commands
   This requires mirroring to be enabled on the pool.
 
 :command:`mirror pool peer remove` [*pool-name*] *uuid*
-  Remove a mirroring peer from a pool. The peer uuid is available
+  Remove a mirroring peer from a pool. The peer UUID is available
   from ``mirror pool info`` command.
 
 :command:`mirror pool peer set` [*pool-name*] *uuid* *key* *value*

doc/mgr/nfs.rst

@@ -283,7 +283,7 @@ Create CephFS Export
 
 .. code:: bash
 
-    $ ceph nfs export create cephfs --cluster-id <cluster_id> --pseudo-path <pseudo_path> --fsname <fsname> [--readonly] [--path=/path/in/cephfs] [--client_addr <value>...] [--squash <value>] [--sectype <value>...]
+    $ ceph nfs export create cephfs --cluster-id <cluster_id> --pseudo-path <pseudo_path> --fsname <fsname> [--readonly] [--path=/path/in/cephfs] [--client_addr <value>...] [--squash <value>] [--sectype <value>...] [--cmount_path <value>]
 
 This creates export RADOS objects containing the export block, where
 
@@ -318,6 +318,12 @@ values may be separated by a comma (example: ``--sectype krb5p,krb5i``). The
 server will negotatiate a supported security type with the client preferring
 the supplied methods left-to-right.
 
+``<cmount_path>`` specifies the path within the CephFS to mount this export on. It is
+allowed to be any complete path hierarchy between ``/`` and the ``EXPORT {path}``. (i.e. if ``EXPORT { Path }`` parameter is ``/foo/bar`` then cmount_path could be ``/``, ``/foo`` or ``/foo/bar``).
+
+.. note:: If this and the other ``EXPORT { FSAL {} }`` options are the same between multiple exports, those exports will share a single CephFS client.
+          If not specified, the default is ``/``.
+
 .. note:: Specifying values for sectype that require Kerberos will only function on servers
           that are configured to support Kerberos. Setting up NFS-Ganesha to support Kerberos
           is outside the scope of this document.
@@ -477,9 +483,9 @@ For example,::
      ],
      "fsal": {
        "name": "CEPH",
-       "user_id": "nfs.mynfs.1",
        "fs_name": "a",
-       "sec_label_xattr": ""
+       "sec_label_xattr": "",
+       "cmount_path": "/"
      },
      "clients": []
    }
@@ -494,6 +500,9 @@ as when creating a new export), with the exception of the
 authentication credentials, which will be carried over from the
 previous state of the export where possible.
 
+!! NOTE: The ``user_id`` in the ``fsal`` block should not be modified or mentioned in the JSON file as it is auto-generated for CephFS exports.
+It's auto-generated in the format ``nfs.<cluster_id>.<fs_name>.<hash_id>``.
+
 ::
 
    $ ceph nfs export apply mynfs -i update_cephfs_export.json
@@ -514,9 +523,9 @@ previous state of the export where possible.
      ],
      "fsal": {
        "name": "CEPH",
-       "user_id": "nfs.mynfs.1",
        "fs_name": "a",
-       "sec_label_xattr": ""
+       "sec_label_xattr": "",
+       "cmount_path": "/"
      },
      "clients": []
    }

qa/suites/orch/cephadm/workunits/task/test_monitoring_stack_basic.yaml

@@ -61,6 +61,6 @@ tasks:
         curl -s http://${PROM_IP}:9095/api/v1/alerts
         curl -s http://${PROM_IP}:9095/api/v1/alerts | jq -e '.data | .alerts | .[] | select(.labels | .alertname == "CephMonDown") | .state == "firing"'
         # check alertmanager endpoints are responsive and mon down alert is active
-        curl -s http://${ALERTM_IP}:9093/api/v1/status
-        curl -s http://${ALERTM_IP}:9093/api/v1/alerts
-        curl -s http://${ALERTM_IP}:9093/api/v1/alerts | jq -e '.data | .[] | select(.labels | .alertname == "CephMonDown") | .status | .state == "active"'
+        curl -s http://${ALERTM_IP}:9093/api/v2/status
+        curl -s http://${ALERTM_IP}:9093/api/v2/alerts
+        curl -s http://${ALERTM_IP}:9093/api/v2/alerts | jq -e '.[] | select(.labels | .alertname == "CephMonDown") | .status | .state == "active"'

qa/tasks/cephfs/test_backtrace.py

@@ -100,3 +100,29 @@ def test_backtrace(self):
         # we don't update the layout in all the old pools whenever it changes
         old_pool_layout = self.fs.read_layout(file_ino, pool=old_data_pool_name)
         self.assertEqual(old_pool_layout['object_size'], 4194304)
+
+    def test_backtrace_flush_on_deleted_data_pool(self):
+        """
+        that the MDS does not go read-only when handling backtrace update errors
+        when backtrace updates are batched and flushed to RADOS (during journal trim)
+        and some of the pool have been removed.
+        """
+        data_pool = self.fs.get_data_pool_name()
+        extra_data_pool_name_1 = data_pool + '_extra1'
+        self.fs.add_data_pool(extra_data_pool_name_1)
+
+        self.mount_a.run_shell(["mkdir", "dir_x"])
+        self.mount_a.setfattr("dir_x", "ceph.dir.layout.pool", extra_data_pool_name_1)
+        self.mount_a.run_shell(["touch", "dir_x/file_x"])
+        self.fs.flush()
+
+        extra_data_pool_name_2 = data_pool + '_extra2'
+        self.fs.add_data_pool(extra_data_pool_name_2)
+        self.mount_a.setfattr("dir_x/file_x", "ceph.file.layout.pool", extra_data_pool_name_2)
+        self.mount_a.run_shell(["setfattr", "-x", "ceph.dir.layout", "dir_x"])
+        self.run_ceph_cmd("fs", "rm_data_pool", self.fs.name, extra_data_pool_name_1)
+        self.fs.flush()
+
+        # quick test to check if the mds has handled backtrace update failure
+        # on the deleted data pool without going read-only.
+        self.mount_a.run_shell(["mkdir", "dir_y"])

qa/tasks/cephfs/test_mirroring.py

@@ -1505,9 +1505,22 @@ def test_get_set_mirror_dirty_snap_id(self):
         """
         That get/set ceph.mirror.dirty_snap_id attribute succeeds in a remote filesystem.
         """
+        log.debug('reconfigure client auth caps')
+        self.get_ceph_cmd_result(
+            'auth', 'caps', "client.{0}".format(self.mount_b.client_id),
+                'mds', 'allow rw',
+                'mon', 'allow r',
+                'osd', 'allow rw pool={0}, allow rw pool={1}'.format(
+                    self.backup_fs.get_data_pool_name(),
+                    self.backup_fs.get_data_pool_name()))
+        log.debug(f'mounting filesystem {self.secondary_fs_name}')
+        self.mount_b.umount_wait()
+        self.mount_b.mount_wait(cephfs_name=self.secondary_fs_name)
+        log.debug('setting ceph.mirror.dirty_snap_id attribute')
         self.mount_b.run_shell(["mkdir", "-p", "d1/d2/d3"])
         attr = str(random.randint(1, 10))
         self.mount_b.setfattr("d1/d2/d3", "ceph.mirror.dirty_snap_id", attr)
+        log.debug('getting ceph.mirror.dirty_snap_id attribute')
         val = self.mount_b.getfattr("d1/d2/d3", "ceph.mirror.dirty_snap_id")
         self.assertEqual(attr, val, f"Mismatch for ceph.mirror.dirty_snap_id value: {attr} vs {val}")