[confcom] adding fragment support (#8238)

* read only logic for some mount types (#62)

* offloading error checking and updating tests

* Gracefully handle broken base64 policies in non-diff mode (#64)

* handle broken base64
* address style fixes

* adding flag to omit ID from policy

* adding ability to not use sidecars via ARM tag

* adding workload identity support for vn2

* Add user prompt to confirm policy overwrite for VN2 YAMLs

* support for image attached fragments

* updating locations where executables are found

* updating version to 1.1.0

* updating test value

* taking out unused dependency

* fixing errors in docs and types

* getting rid of whitespace

* updating kata tests for linux

* updating kata tests for windows

* can't have binary files

---------

Co-authored-by: Khalil Sayid <30742855+ksayid@users.noreply.github.com>
Co-authored-by: Khalil Sayid <khalilsayid@microsoft.com>

src/confcom/HISTORY.rst

@@ -2,7 +2,20 @@
 
 Release History
 ===============
+1.1.0
+++++++
+* adding support for image-attached fragments via `acifragmentgen`
+* adding workload identity support for VN2
+* adding `--exclude-default-fragments` to disallow sidecars from policy
+* adding `--omit-id` for policy stability across multiple image registries
+* better handle broken base64 policies in templates
+* improve error handling structure
+* make some mount types in VN2 required readonly
+* prompt users if they want to overwrite their policy in VN2
+* changing where dmverity-vhd and sign1util binaries are fetched from. This includes a significant speedup in dmverity-vhd hashing
+
 1.0.1
+++++++
 * getting rid of msrestazure dependency in _validators.py
 
 1.0.0

src/confcom/azext_confcom/README.md

@@ -29,8 +29,8 @@
     - [allow_environment_variable_dropping](#allow_environment_variable_dropping)
     - [allow_unencrypted_scratch](#allow_unencrypted_scratch)
     - [allow_capabilities_dropping](#allow_capabilities_dropping)
+- [Microsoft Azure CLI 'confcom acifragmentgen' Extension Examples](#microsoft-azure-cli-confcom-acifragmentgen-extension-examples)
 - [Microsoft Azure CLI 'confcom katapolicygen' Extension Examples](#microsoft-azure-cli-confcom-katapolicygen-extension-examples)
-  - [Microsoft Azure CLI 'confcom katapolicygen' Extension Examples]
 
 ## Microsoft Azure CLI 'confcom acipolicygen' Extension Examples
 
@@ -279,16 +279,16 @@ Mixed-mode policy generation is available in the `confcom` tooling, meaning imag
 
 ## AKS Virtual Node
 
-Azure Kubernetes Service (AKS) allows pods to be scheduled on Azure Container Instances (ACI) 
-using the [AKS Virtual Node](https://learn.microsoft.com/en-us/azure/aks/virtual-nodes) feature. The `confcom` tooling can generate security policies for these ACI-based pods in the same way as for standalone ACI container groups. The key difference is that the `confcom` tooling will ingest an AKS pod specification (`pod.yaml`) instead of an ARM Template. 
+Azure Kubernetes Service (AKS) allows pods to be scheduled on Azure Container Instances (ACI)
+using the [AKS Virtual Node](https://learn.microsoft.com/en-us/azure/aks/virtual-nodes) feature. The `confcom` tooling can generate security policies for these ACI-based pods in the same way as for standalone ACI container groups. The key difference is that the `confcom` tooling will ingest an AKS pod specification (`pod.yaml`) instead of an ARM Template.
 
 Use the following command to generate and print a security policy for an AKS pod running on ACI:
 
 ```bash
 az confcom acipolicygen --virtual-node-yaml ./pod.yaml --print-policy
 ```
 
-> [!NOTE]  
+> [!NOTE]
 > The `acipolicygen` command is specific to generating policies for ACI-based containers. For generating security policies for the [Confidential Containers on AKS](https://learn.microsoft.com/en-us/azure/aks/confidential-containers-overview) feature, use the `katapolicygen` command.
 
 ## Security Policy Information Sources
@@ -659,6 +659,81 @@ This rule determines whether unencrypted writable storage from the UVM to the co
 
 Whether to allow capabilities to be dropped in the same manner as allow_environment_variable_dropping.
 
+## Microsoft Azure CLI 'confcom acifragmentgen' Extension Examples
+
+Run `az confcom acifragmentgen --help` to see a list of supported arguments along with explanations. The following commands demonstrate the usage of different arguments to generate confidential computing security fragments.
+
+For information on what a policy fragment is, see [policy fragments](#policy-fragments). For a full walkthrough on how to generate a policy fragment and use it in a policy, see [Create a Key and Cert for Signing](../samples/certs/README.md).
+
+**Examples:**
+
+Example 1: The following command creates a security fragment and prints it to stdout as well as saving it to a file `contoso.rego`:
+
+```bash
+az confcom acifragmentgen --config ./fragment_config.json --svn 1 --namespace contoso
+```
+
+The config file is a JSON file that contains the following information:
+
+```json
+{
+ "containers": [
+  {
+   "name": "my-image",
+   "properties": {
+    "image": "mcr.microsoft.com/acc/samples/aci/helloworld:2.8",
+    "environmentVariables": [
+     {
+      "name": "PATH",
+      "value": "/customized/path/value"
+     },
+     {
+      "name": "TEST_REGEXP_ENV",
+      "value": "test_regexp_env(.*)",
+      "regex": true
+     }
+    ],
+    "command": [
+     "python3",
+     "main.py"
+    ]
+   }
+  }
+ ]
+}
+```
+
+The `--svn` argument is used to specify the security version number of the fragment and should be an integer. The `--namespace` argument is used to specify the namespace of the fragment and cannot conflict with some built-in names. If a conflicting name occurs, there will be an error message. [This list of reserved names can be found here under 'reserved_fragment_namespaces'](./data/internal_config.json). The format of the config file generally follows that of the [ACI resource in an ARM template](https://learn.microsoft.com/en-us/azure/templates/microsoft.containerinstance/containergroups?pivots=deployment-language-arm-template).
+
+Example 2: This command creates a signed security fragment and attaches it to a container image in an ORAS-compliant registry:
+
+```bash
+az confcom acifragmentgen --chain ./samples/certs/intermediateCA/certs/www.contoso.com.chain.cert.pem --key ./samples/certs/intermediateCA/private/ec_p384_private.pem --svn 1 --namespace contoso --config ./samples/config.json --upload-fragment
+```
+
+Example 3: This command creates a file to be used by `acipolicygen` that says which fragments should be included in the policy. Note that the policy must be [COSE](https://www.iana.org/assignments/cose/cose.xhtml) signed:
+
+```bash
+az confcom acifragmentgen --generate-import -p ./contoso.rego.cose --minimum-svn 1 --fragments-json fragments.json
+```
+
+This outputs a file `fragments.json` that contains the following information:
+
+```json
+{
+ "path": "./contoso.rego.cose",
+ "feed": "contoso.azurecr.io/example",
+ "includes": [
+  "containers",
+  "fragments"
+ ],
+ "issuer": "did:x509:0:sha256:mLzv0uyBNQvC6hi4y9qy8hr6NSZuYFv6gfCwAEWBNqc::subject:CN:Contoso",
+ "minimum_svn": "1"
+}
+```
+
+This file is then used by `acipolicygen` to generate a policy that includes custom fragments.
+
 ## Microsoft Azure CLI 'confcom katapolicygen' Extension Examples
 
 Run `az confcom katapolicygen --help` to see a list of supported arguments along with explanations. The following commands demonstrate the usage of different arguments to generate confidential computing security policies.

src/confcom/azext_confcom/_help.py

@@ -5,7 +5,7 @@
 # --------------------------------------------------------------------------------------------
 
 from knack.help_files import helps  # pylint: disable=unused-import
-
+from azext_confcom.config import SUPPORTED_ALGOS
 
 helps[
     "confcom"
@@ -51,23 +51,23 @@
 
         - name: --debug-mode
           type: boolean
-          short-summary: 'When enabled, the generated security policy adds the ability to use /bin/sh or /bin/bash to debug the container. It also enabled stdio access, ability to dump stack traces, and enables runtime logging. It is recommended to only use this option for debugging purposes.'
+          short-summary: 'When enabled, the generated security policy adds the ability to use /bin/sh or /bin/bash to debug the container. It also enabled stdio access, ability to dump stack traces, and enables runtime logging. It is recommended to only use this option for debugging purposes'
 
         - name: --approve-wildcards -y
           type: boolean
-          short-summary: 'When enabled, all prompts for using wildcards in environment variables are automatically approved.'
+          short-summary: 'When enabled, all prompts for using wildcards in environment variables are automatically approved'
 
         - name: --disable-stdio
           type: boolean
-          short-summary: 'When enabled, the containers in the container group do not have access to stdio.'
+          short-summary: 'When enabled, the containers in the container group do not have access to stdio'
 
         - name: --print-existing-policy
           type: boolean
-          short-summary: 'When enabled, the existing security policy that is present in the ARM Template is printed to the command line, and no new security policy is generated.'
+          short-summary: 'When enabled, the existing security policy that is present in the ARM Template is printed to the command line, and no new security policy is generated'
 
         - name: --diff -d
           type: boolean
-          short-summary: 'When combined with an input ARM Template, verifies the policy present in the ARM Template under "ccePolicy" and the containers within the ARM Template are compatible. If they are incompatible, a list of reasons is given and the exit status code will be 2.'
+          short-summary: 'When combined with an input ARM Template file (or YAML file for Virtual Node policy generation), verifies the policy present in the ARM Template under "ccePolicy" and the containers within the file are compatible. If they are incompatible, a list of reasons is given and the exit status code will be 2'
 
         - name: --outraw
           type: boolean
@@ -79,7 +79,7 @@
 
         - name: --save-to-file -s
           type: string
-          short-summary: 'Save output policy to given file path.'
+          short-summary: 'Save output policy to given file path'
 
         - name: --print-policy
           type: boolean
@@ -89,6 +89,22 @@
           type: boolean
           short-summary: 'When enabled, the hashing algorithm used to generate the policy is faster but less memory efficient'
 
+        - name: --omit-id
+          type: boolean
+          short-summary: 'When enabled, the generated policy will not contain the ID field. This will keep the policy from being tied to a specific image name and tag'
+
+        - name: --include-fragments -f
+          type: boolean
+          short-summary: 'When enabled, the path specified by --fragments-json will be used to pull fragments from an OCI registry or locally and include them in the generated policy'
+
+        - name: --fragments-json -j
+          type: string
+          short-summary: 'Path to JSON file containing fragment information to use for generating a policy. This requires --include-fragments to be enabled'
+
+        - name: --exclude-default-fragments -e
+          type: boolean
+          short-summary: 'When enabled, the default fragments are not included in the generated policy. This includes containers needed to mount azure files, mount secrets, mount git repos, and other common ACI features'
+
     examples:
         - name: Input an ARM Template file to inject a base64 encoded Confidential Container Security Policy into the ARM Template
           text: az confcom acipolicygen --template-file "./template.json"
@@ -98,6 +114,96 @@
           text: az confcom acipolicygen --template-file "./template.json" -s "./output-file.txt" --print-policy
         - name: Input an ARM Template file and use a tar file as the image source instead of the Docker daemon
           text: az confcom acipolicygen --template-file "./template.json" --tar "./image.tar"
+        - name: Input an ARM Template file and use a fragments JSON file to generate a policy
+          text: az confcom acipolicygen --template-file "./template.json" --fragments-json "./fragments.json" --include-fragments
+"""
+
+helps[
+    "confcom acifragmentgen"
+] = f"""
+    type: command
+    short-summary: Create a Confidential Container Policy Fragment for ACI.
+
+    parameters:
+        - name: --image
+          type: string
+          short-summary: 'Image to use for the generated policy fragment'
+
+        - name: --input -i
+          type: string
+          short-summary: 'Path to a JSON file containing the configuration for the generated policy fragment'
+
+        - name: --tar
+          type: string
+          short-summary: 'Path to either a tarball containing image layers or a JSON file containing paths to tarballs of image layers'
+
+        - name: --namespace -n
+          type: string
+          short-summary: 'Namespace to use for the generated policy fragment'
+
+        - name: --svn
+          type: string
+          short-summary: 'Minimum Allowed Software Version Number for the generated policy fragment. This should be a monotonically increasing integer'
+
+        - name: --feed -f
+          type: string
+          short-summary: 'Feed to use for the generated policy fragment. This is typically the same as the image name when using image-attached fragments. It is the location in the remote repository where the fragment will be stored'
+
+        - name: --key -k
+          type: string
+          short-summary: 'Path to .pem formatted key file to use for signing the generated policy fragment. This must be used with --chain'
+
+        - name: --chain
+          type: string
+          short-summary: 'Path to .pem formatted certificate chain file to use for signing the generated policy fragment. This must be used with --key'
+
+        - name: --algo
+          type: string
+          short-summary: |
+            Algorithm used for signing the generated policy fragment. This must be used with --key and --chain.
+            Supported algorithms are {SUPPORTED_ALGOS}
+
+        - name: --fragment-path -p
+          type: string
+          short-summary: 'Path to an existing policy fragment file to be used with --generate-import. This option allows you to create import statements for the specified fragment without needing to pull it from an OCI registry'
+
+        - name: --generate-import -g
+          type: boolean
+          short-summary: 'Generate an import statement for a policy fragment'
+
+        - name: --disable-stdio
+          type: boolean
+          short-summary: 'When enabled, the containers in the container group do not have access to stdio'
+
+        - name: --debug-mode
+          type: boolean
+          short-summary: 'When enabled, the generated security policy adds the ability to use /bin/sh or /bin/bash to debug the container. It also enabled stdio access, ability to dump stack traces, and enables runtime logging. It is recommended to only use this option for debugging purposes'
+
+        - name: --output-filename
+          type: string
+          short-summary: 'Save output policy to given file path'
+
+        - name: --outraw
+          type: boolean
+          short-summary: 'Output policy in clear text compact JSON instead of default pretty print format'
+
+        - name: --upload-fragment -u
+          type: boolean
+          short-summary: 'When enabled, the generated policy fragment will be uploaded to the registry of the image being used'
+
+        - name: --fragments-json -j
+          type: string
+          short-summary: 'Path to a JSON file that will store the fragment import information generated when using --generate-import. This file can later be fed into the policy generation command (acipolicygen) to include the fragment in a new or existing policy. If not specified, the import statement will be printed to the console instead of being saved to a file'
+
+    examples:
+        - name: Input an image name to generate a simple fragment
+          text: az confcom acifragmentgen --image mcr.microsoft.com/azuredocs/aci-helloworld
+        - name: Input a config file to generate a fragment with a custom namespace and debug mode enabled
+          text: az confcom acifragmentgen --input "./config.json" --namespace "my-namespace" --debug-mode
+        - name: Generate an import statement for a signed local fragment
+          text: az confcom acifragmentgen --fragment-path "./fragment.json" --generate-import --minimum-svn 1
+        - name: Generate a fragment and COSE sign it with a key and chain
+          text: az confcom acifragmentgen --image mcr.microsoft.com/azuredocs/aci-helloworld --key "./key.pem" --chain "./chain.pem" --svn 1 --namespace contoso --no-print
 """
 
 helps[