feat: release v5.33.0 (#942)

* test: add tests for enable/disable all feature (#906)

* chore: add data-testid to interactAllPrompt

* chore: initial attempt for interactAllPrompt

* chore: add tests and backend for enable disable all

* chore: decrease sleep for tests

* chore: increase sleep for disable_enable_all test

* fix: update urllib3 (#922)

This PR updates urllib3 to v1.26.18 to resolve CVE (already done for main branch).

* Feat/create stories for existing components (#916)

feat: move components to ts and create stories

* ci: build storybook to docs (#888)

* ci: refactor ui workflows

* ci: refactor ui workflows

* ci: refactor ui workflows

* ci: refactor ui workflows

* add shell bash

* add cd

* composite action to reusable workflow

* composite action to reusable workflow

* composite action to reusable workflow

* composite action to reusable workflow

* add storybook build

* feat: add stories for components

Checkbox Component, AcceptModal, FileInout

* feat: add component stories, move components to ts

* fix: rebase

* feat: other components moved to ts and stories

* feat: move components to ts and create stories

* feat: handlers for stories, TextComponent to ts
move TextComponent to ts
add correct handlers to stories
fix Meta and meta missmatches in stories

* fix: review

* fix: review

* fix: lint

---------

Co-authored-by: vtsvetkov-splunk <142901247+vtsvetkov-splunk@users.noreply.github.com>

* feat: additional validations for the groups feature (#926)

This PR adds one more additional validation for the "Groups Feature" (https://splunk.github.io/addonfactory-ucc-generator/advanced/groups_feature/): there can not be a group field which is not defined in the entity.

* chore: get rid of nesting of UI directory (#917)

* chore: get rid of nesting folders in UI

* rev

* reapply changes

* ci: don't rebuild add-on one more time for UI tests (#937)

* ci: add script for injecting Splunk RUM to test addon (#934)

* test: add textarea component tests (#928)

* chore: update global_config_everything globalConfig.json with text area

* chore: add textarea component to ExampleInputOne and update tests for textarea component

* chore: pre-commit fixes

* chore: update smoke tests and add new test cases for test area

* chore: pre-commit fixes

* chore: fix smoke tests

* chore: fix smoke tests

* chore: fix smoke tests whitespaces

* chore(deps): bump smartx version

* chore: poetry lock --no-update

---------

Co-authored-by: Artem Rys <rysartem@gmail.com>

* chore(deps): update ui dependencies (#925)

* docs: adjust UI build commands (#931)

* feat: add optional label for helpLink component (#939)

* feat: add optional label for helpLink component

* fix: align new lines

* test: update smoke test

---------

Co-authored-by: Artem Rys <rysartem@gmail.com>

* feat: ADDON-64844 better spacing in modal (#935)

feat: ADDON-64844 better spacing in modal

* feat: ADDON-65107 implement download button for openapi (#932)

feat: ADDON-65107 implement download button for openapi

* refactor: change location of download openApi button (#940)

* ci: returntocorp -> semgrep (#941)

`returntocorp` organization name was changed to `semgrep`.

* chore(ui): fix TypeScript types resolution (#944)

* chore: update dynatrace TA for smoke, altered globalConfig to pass checks (#946)

* feat(schema.json): make entity schemas more specific (#938)

* feat: extract helpLink

* feat: extract TextEntity

* feat(schema.json): merge ConfigurationEntity into GenericEntity

* feat(schema.json): extract TextArea entity

* feat(schema.json): extract SingleSelect entity

* feat(schema.json): extract Checkbox entity

* feat(schema.json): extract MultipleSelect entity

* feat(schema.json): extract RadioEntity

* feat(schema.json): extract FileEntity

* feat(schema.json): extract OAuth entity

* fix issues

* remove GenericEntity

* test

* Revert "test"

This reverts commit c057860.

* refactor(schema.json): add display option for LinkEntity

* chore(storybook): ADDON-65440 add theme switcher (#950)

---------

Co-authored-by: mkolasinski-splunk <105011638+mkolasinski-splunk@users.noreply.github.com>
Co-authored-by: soleksy-splunk <143183665+soleksy-splunk@users.noreply.github.com>
Co-authored-by: vtsvetkov-splunk <142901247+vtsvetkov-splunk@users.noreply.github.com>

.github/workflows/build-test-release.yml

@@ -59,7 +59,7 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - id: semgrep
-        uses: returntocorp/semgrep-action@v1
+        uses: semgrep/semgrep-action@v1
         with:
           publishToken: ${{ secrets.SEMGREP_PUBLISH_TOKEN }}
 
@@ -104,7 +104,7 @@ jobs:
         run: poetry run mkdocs build --strict
       - name: Build UCC
         run: |
-          cp -R built-ui/package splunk_add_on_ucc_framework/
+          cp -R built-ui/. splunk_add_on_ucc_framework/package/appserver/static/js/
           poetry build
       - uses: actions/upload-artifact@v3
         with:
@@ -157,7 +157,7 @@ jobs:
           path: built-ui/
       - run: curl -sSL https://install.python-poetry.org | python3 - --version 1.5.1
       - run: |
-          cp -R built-ui/package splunk_add_on_ucc_framework/
+          cp -R built-ui/. splunk_add_on_ucc_framework/package/appserver/static/js/
           poetry install
           poetry run pytest tests/smoke
 
@@ -176,9 +176,12 @@ jobs:
           path: built-ui/
       - run: curl -sSL https://install.python-poetry.org | python3 - --version 1.5.1
       - run: |
-          cp -R built-ui/package splunk_add_on_ucc_framework/
+          cp -R built-ui/. splunk_add_on_ucc_framework/package/appserver/static/js/
           poetry install
       - run: poetry run ucc-gen build --source tests/testdata/test_addons/package_global_config_everything/package
+      - run: ./scripts/include-rum.sh output/Splunk_TA_UCCExample/appserver/templates/base.html scripts/rum-script.html "$RUM_ACCESS_TOKEN"
+        env:
+          RUM_ACCESS_TOKEN: ${{ secrets.RUM_ACCESS_TOKEN}}
       - run: poetry run ucc-gen package --path output/Splunk_TA_UCCExample
       - uses: actions/upload-artifact@v3
         with:
@@ -227,20 +230,14 @@ jobs:
         with:
           name: Splunk_TA_UCCExample-raw-output
           path: output/
-      - uses: actions/download-artifact@v3
-        with:
-          name: UCC-UI-build
-          path: built-ui/
-      - run: curl -sSL https://install.python-poetry.org | python3 - --version 1.5.1
       - run: |
-          cp -R built-ui/package splunk_add_on_ucc_framework/
+          curl -sSL https://install.python-poetry.org | python3 - --version 1.5.1
           poetry install
       - name: Link chromedriver
         # Use installed chromedriver https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2204-Readme.md
         run: |
           export PATH=$PATH:$CHROMEWEBDRIVER
           chromedriver --version
-      - run: poetry run ucc-gen build --source tests/testdata/test_addons/package_global_config_everything/package
       - run: |
           ./run_splunk.sh ${{ matrix.splunk.version }}
           until curl -Lsk "https://localhost:8088/services/collector/health" &>/dev/null ; do echo -n "Waiting for HEC-" && sleep 5 ; done
@@ -323,7 +320,7 @@ jobs:
       - run: poetry install
       - name: Copy built UCC UI into UCC generator folder
         run: |
-          cp -R built-ui/package splunk_add_on_ucc_framework/
+          cp -R built-ui/. splunk_add_on_ucc_framework/package/appserver/static/js/
       - id: semantic
         uses: splunk/semantic-release-action@v1.3
         with:

.github/workflows/build-ui.yml

@@ -29,7 +29,7 @@ jobs:
       - name: Build UCC UI
         run: yarn run build
       - name: List deps into dependencies.txt
-        run: yarn list --prod --depth 0 | python ../.github/scripts/format_yarn_deps.py --output-file=dist/package/appserver/static/js/dependencies.txt
+        run: yarn list --prod --depth 0 | python ../.github/scripts/format_yarn_deps.py --output-file=dist/dependencies.txt
       - name: Upload dist with name UCC-UI-build
         uses: actions/upload-artifact@v3
         with:

.gitignore

@@ -41,3 +41,8 @@ Splunk_TA_Dynatrace_ucc/*
 demo_addon_for_splunk/
 demo_addon_for_splunk_already_exists/
 init_addon_for_ucc_package/
+
+# UI build
+# ignore everything except redirect_page.js
+splunk_add_on_ucc_framework/package/appserver/static/js/build/
+!splunk_add_on_ucc_framework/package/appserver/static/js/build/redirect_page.js

.licenserc.yaml

@@ -46,4 +46,7 @@ header:
     - "splunk_add_on_ucc_framework/commands/import_from_aob.sh"
     - "splunk_add_on_ucc_framework/commands/imports.py"
     - "scripts/compare_different_ucc_versions_output.sh"
+    - "scripts/include-rum.sh"
+    - "build-ui.sh"
+    - "scripts/rum-script.html"
     - "mypy.ini"

CONTRIBUTING.md

@@ -1,3 +1,3 @@
 # Contributing Guidelines
 
-Read about Contributing Guidelines [here](https://splunk.github.io/addonfactory-ucc-generator/contributing/).
+Read about Contributing Guidelines [here](docs/contributing.md).

README.md

@@ -20,4 +20,4 @@ If you are a part of the community use: https://splunk-usergroups.slack.com/arch
 
 ## Contributing Guidelines
 
-Go to [Contributing Guidelines](https://splunk.github.io/addonfactory-ucc-generator/contributing/).
+Go to [contributing Guidelines in the repo](docs/contributing.md) or [published version](https://splunk.github.io/addonfactory-ucc-generator/contributing/).

build-ui.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+
+cd ui && yarn run setup
+cd ..
+
+cp -n -R ui/dist/build/ splunk_add_on_ucc_framework/package/appserver/static/js/build/

docs/contributing.md

@@ -1,19 +1,22 @@
 # Contributing Guidelines
 
+Default development branch is `develop`. Please use it for PR
+
 ## Build and Test
 
-Download UCC UI and unpack it using the following commands:
+Prerequisites:
+- Node.js >= 18 ([NodeJS](https://nodejs.org/en/download) or use [nvm](https://github.com/nvm-sh/nvm))
+- Yarn Classic (`npm install --global yarn`)
+- Poetry 1.5.1. [Installation guide](https://python-poetry.org/docs/#installing-with-the-official-installer)
+
+If you are interested in contributing to UI, `ui` folder has separate README.md.
 
+Building UI in `ui/dist` folder and copying files to static folder of UCC.
 ```
-cd ui/
-yarn
-yarn run build
-cp -R dist/ ../splunk_add_on_ucc_framework/
+./build-ui.sh
 ```
 
-This project uses [`poetry 1.5.1`](https://python-poetry.org/).
-
-To build a new local version of `ucc-gen`:
+Building a new local version of `ucc-gen`:
 
 ```
 poetry build

docs/openapi.md

@@ -7,6 +7,8 @@ There has to be defined valid `globalConfig.json` and `app.manifest` to have the
 
 Once `ucc-gen` command is executed, OpenAPI description document is located in output `static` subdirectory.
 
+One of ways is to download it via button displayed in top right corner of configuration page.
+
 When add-on is installed to Splunk instance, it is exposed via web and management interface, so is available under following addresses accordingly:
 
 * \[protocol\]://\[domain\]:\[port\]/en-GB/splunkd/__raw/servicesNS/\[user\]/\[appname\]/static/openapi.json

pyproject.toml

@@ -20,7 +20,7 @@ version = "5.32.0"
 description = "Splunk Add-on SDK formerly UCC is a build and code generation framework"
 license = "Apache-2.0"
 authors = ["Splunk <addonfactory@splunk.com>"]
-include = ["NOTICE"]
+include = ["NOTICE", "splunk_add_on_ucc_framework/package/appserver/**/*"]
 readme = "README.md"
 homepage = "https://github.com/splunk/addonfactory-ucc-generator"
 repository = "https://github.com/splunk/addonfactory-ucc-generator"
@@ -55,7 +55,7 @@ mkdocs = "^1.4.2"
 importlib-metadata = {version="*", python="<3.8"}
 pytest = "^7.2.1"
 pytest-splunk-addon = "^5.0.0"
-pytest-splunk-addon-ui-smartx = "2.3.4"
+pytest-splunk-addon-ui-smartx = "2.5.1"
 pytest-rerunfailures = "^11.1.1"
 mkdocs-material = "^9.1.3"
 pytest-cov = "^4.0.0"

scripts/include-rum.sh

@@ -0,0 +1,24 @@
+#!/bin/bash
+
+# Ensure there are exactly two arguments provided
+if [[ "$#" -ne 3 ]]; then
+  echo "Usage: $0 <path to base HTML file> <path to script HTML file> <rum_access_token>"
+  exit 1
+fi
+
+source_html_file=$1
+target_html_file=$2
+rum_access_token=$3
+
+# Ensure both files exist
+if [[ ! -f "$target_html_file" ]] || [[ ! -f "$source_html_file" ]]; then
+  echo "Both files must exist."
+  exit 1
+fi
+
+# Fill token in script
+sed -i "s/RUM_ACCESS_TOKEN/$rum_access_token/g" "$target_html_file"
+
+# Fill base.html with script
+sed -i "\|<head>|r $target_html_file" "$source_html_file"
+

scripts/rum-script.html

@@ -0,0 +1,17 @@
+<script src="https://cdn.signalfx.com/o11y-gdi-rum/v0.16.5/splunk-otel-web.js" integrity="sha384-zd5QhD1H5BOwLjGzeHqnEpZRkpWp1TwNbJyEedzArs+6Tq/JejIutyQ3rTndqygI" crossorigin="anonymous"></script>
+<script>
+  SplunkRum.init({
+    realm: "us1",
+    rumAccessToken: "RUM_ACCESS_TOKEN",
+    applicationName: "UCC",
+    deploymentEnvironment: "CI"
+  });
+</script>
+<script src="https://cdn.signalfx.com/o11y-gdi-rum/v0.16.5/splunk-otel-web-session-recorder.js" integrity="sha384-k6kMobPGxYompyccnaa88srGuLjk6Yb4d7yEt98LhR5zaP0zxjl3ELA5C4LRxvJX" crossorigin="anonymous"></script>
+<script>
+  SplunkSessionRecorder.init({
+    app: "UCC",
+    realm: "us1",
+    rumAccessToken: "RUM_ACCESS_TOKEN"
+  });
+</script>

splunk_add_on_ucc_framework/global_config_validator.py

@@ -331,7 +331,8 @@ def _validate_entity_duplicates(self, entity: List[Dict[str, Any]]) -> None:
         fields, labels = [], []
         for _entity in entity:
             fields.append(_entity["field"].lower())
-            labels.append(_entity["label"].lower())
+            if "label" in _entity:
+                labels.append(_entity["label"].lower())
             options = _entity.get("options")
             if options and options.get("autoCompleteFields"):
                 self._validate_autoCompleteFields_duplicates(
@@ -521,7 +522,7 @@ def _validate_checkbox_group(self) -> None:
                                 )
                             group_used_field_names.append(group_field_name)
 
-    def _validate_group_labels(self) -> None:
+    def _validate_groups(self) -> None:
         pages = self._config["pages"]
         inputs = pages.get("inputs")
         if inputs is None:
@@ -531,6 +532,10 @@ def _validate_group_labels(self) -> None:
             groups = service.get("groups")
             if groups is None:
                 continue
+            entities = service["entity"]
+            entity_fields = []
+            for entity in entities:
+                entity_fields.append(entity["field"])
             service_group_labels = []
             for group in groups:
                 group_label = group["label"]
@@ -539,6 +544,13 @@ def _validate_group_labels(self) -> None:
                         f"Service {service['name']} has duplicate labels in groups"
                     )
                 service_group_labels.append(group_label)
+            for group in groups:
+                group_fields = group["fields"]
+                for group_field in group_fields:
+                    if group_field not in entity_fields:
+                        raise GlobalConfigValidatorException(
+                            f"Service {service['name']} uses group field {group_field} which is not defined in entity"
+                        )
 
     def validate(self) -> None:
         self._validate_config_against_schema()
@@ -552,4 +564,4 @@ def validate(self) -> None:
         self._validate_panels()
         self._warn_on_placeholder_usage()
         self._validate_checkbox_group()
-        self._validate_group_labels()
+        self._validate_groups()