feat: validate CSpell dictionaries (#159)

# Description

This PR adds a validation of the custom CSpell dictionaries whenever
they are changed. We validate, that they

- are sorted
- contain no orphan words

Closes #157 

# Verification

Local tests.

.github/workflows/default_spelling_callable.yml

@@ -19,3 +19,20 @@ jobs:
       - uses: streetsidesoftware/cspell-action@ef95dc49d631fc2a9e9ea089ae2b2127b7c4588e # v6.10.0
         with:
           config: .config/cspell.json
+
+  validate-dictionaries:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        id: changes-for-shell
+        with:
+          list-files: "shell"
+          filters: |
+            dictionaries:
+            - added|modified: '.config/dictionaries/*.txt'
+
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+
+      - run: .github/workflows/scripts/validate-dictionaries.sh
+        continue-on-error: true
+        if: steps.changes-for-shell.outputs.dictionaries == 'true'

.github/workflows/scripts/check_dictionaries.sh

@@ -0,0 +1,47 @@
+#!/bin/bash
+
+#
+# Based on the idea outlined in https://github.com/streetsidesoftware/cspell/issues/2536#issuecomment-1126077282
+# but with a few modifications to fit the needs of our project.
+#
+
+set -euo pipefail
+
+MISSPELLED_WORDS_PATH="misspelled-words.txt"
+
+CSPELL_CONFIGURATION_FILE=$(find . -name "cspell.json" | head -n 1)
+DICTIONARIES_PATH=".config/dictionaries"
+mapfile -t DICTIONARY_FILES_TO_CHECK < <(ls .config/dictionaries/*.txt)
+
+# Make a list of every misspelled word without any custom dictionaries and configuration file
+mv "$CSPELL_CONFIGURATION_FILE" "${CSPELL_CONFIGURATION_FILE}.temp"
+npx cspell . --dot --no-progress --no-summary --unique --words-only --no-exit-code --exclude ".git/**" --exclude "$DICTIONARIES_PATH/**" | sort --ignore-case --unique > "$MISSPELLED_WORDS_PATH"
+
+# Check the custom dictionaries
+ONE_OR_MORE_FAILURES=0
+for DICTIONARY_NAME in "${DICTIONARY_FILES_TO_CHECK[@]}"; do
+  echo "Checking for orphaned words in dictionary: $DICTIONARY_NAME"
+
+  # ensure the dictionary is sorted and unique
+  sort --ignore-case --unique --check "$DICTIONARY_NAME" > /dev/null
+
+  # Check that each word in the dictionary is actually being used
+  while IFS= read -r line; do
+    # Remove any trailing newline characters
+    line=$(echo "$line" | tr -d '\r\n')
+
+    if ! grep "$line" "$MISSPELLED_WORDS_PATH" --ignore-case --silent ; then
+      echo "The following word in the $DICTIONARY_NAME dictionary is not being used: ($line)"
+      ONE_OR_MORE_FAILURES=1
+    fi
+  done < "$DICTIONARY_NAME"
+done
+
+rm -f "$MISSPELLED_WORDS_PATH"
+
+if [ $ONE_OR_MORE_FAILURES -ne "0" ]; then
+  echo "Dictionary check failed."
+  exit 1
+fi
+
+echo "All dictionaries are valid."

README.md

@@ -27,14 +27,15 @@ all files in case of major changes in the templates.
 - linters for all files
 - PRs are checked for semantic commit titles to ensure an automatic release
 - ChatOps to run workflows from comments
+- a `.config/dictionaries/project.txt` file for the spell checker exceptions
 
 ## For Developers - Repository Layout
 
 1. Add all workflows to `.github/workflows/`, otherwise they can't be referenced from the repositories.
 2. Workflows with `this_` prefix are used for this repository only.
-3. Workflows with `default_` prefix are added to every new repository. Otherwise use the correct prefix for the project type.
+3. Workflows with `default_` prefix are added to every new repository. Otherwise, use the correct prefix for the project type.
 
-The script to set up the workflows for new repositories is `setup-workflows.sh`. It copies the necessary files to the new
+The script to set up the workflows for new repositories is `update-workflows.sh`. It copies the necessary files to the new
 repository. It starts with the default workflows and adds the specific ones based on the project type. In case of a filename clash,
 the specific template overwrites the default one (exception: `.gitignore` These files are concatenated).
 
@@ -50,3 +51,9 @@ Use
 
 in the file to describe the triggers which should be used in the repository. The script will automatically replace the triggers
 marked with `USE_WORKFLOW` which are valid within this repository only.
+
+### Spell Checker
+
+1. Add the words to the `.config/dictionaries/workflow.txt` file.
+2. `.config/dictionaries/project.txt` file is used for the project specific words of the project being set up.
+3. All other dictionaries are managed by the `Repository-Template-*` repositories.