Skip to content

Commit

Permalink
feat: add nbqa linter for jupyter notebooks (#6240)
Browse files Browse the repository at this point in the history
  • Loading branch information
@holtskinner
holtskinner authored Dec 20, 2024
1 parent 85f7611 commit 469f30e
Show file tree
Hide file tree
Showing 33 changed files with 478 additions and 4 deletions.
1 change: 1 addition & 0 deletions .github/dependabot.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ updates:
- flake8
- isort
- mypy
- nbqa
- pyink
- pylint
- ruff
Expand Down
1 change: 1 addition & 0 deletions Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,7 @@ ENV PATH="${PATH}:/venvs/cpplint/bin"
ENV PATH="${PATH}:/venvs/flake8/bin"
ENV PATH="${PATH}:/venvs/isort/bin"
ENV PATH="${PATH}:/venvs/mypy/bin"
ENV PATH="${PATH}:/venvs/nbqa/bin"
ENV PATH="${PATH}:/venvs/pyink/bin"
ENV PATH="${PATH}:/venvs/pylint/bin"
ENV PATH="${PATH}:/venvs/ruff/bin"
Expand Down
11 changes: 11 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ Super-linter supports the following tools:
| **JSON** | [eslint-plugin-jsonc (configured for JSON)](https://www.npmjs.com/package/eslint-plugin-jsonc) (default), [eslint-plugin-json](https://www.npmjs.com/package/eslint-plugin-json) | [Prettier](https://prettier.io/) |
| **JSONC**, **JSON5** | [eslint-plugin-jsonc](https://www.npmjs.com/package/eslint-plugin-jsonc) | [Prettier](https://prettier.io/) |
| **JSX**, **TSX** | [eslint-plugin-jsx-a11y](https://github.com/jsx-eslint/eslint-plugin-jsx-a11y), [eslint-plugin-react](https://github.com/jsx-eslint/eslint-plugin-react) | [Prettier](https://prettier.io/) |
| **Jupyter Notebook** | [nbqa](https://nbqa.readthedocs.io/en/latest/index.html) | [nbqa](https://nbqa.readthedocs.io/en/latest/index.html) |
| **Kubernetes** | [kubeconform](https://github.com/yannh/kubeconform), [Checkov](https://www.checkov.io/) | See YAML formatters |
| **Kotlin** | [ktlint](https://github.com/pinterest/ktlint) | |
| **LaTeX** | [ChkTex](https://www.nongnu.org/chktex/) | |
Expand Down Expand Up @@ -249,6 +250,9 @@ You can configure Super-linter using the following environment variables:
| **FIX_JSONC_PRETTIER** | `true` | Flag to enable or disable the formatting of JSONC and JSON5 files with Prettier. |
| **FIX_JSX_PRETTIER** | `true` | Flag to enable or disable the formatting of JSX files with Prettier. |
| **FIX_JSX** | `false` | Option to enable fix mode for `JSX`. |
| **FIX_JUPYTER_NBQA_BLACK** | `false` | Option to enable fix mode for `NBQA_BLACK`. |
| **FIX_JUPYTER_NBQA_ISORT** | `false` | Option to enable fix mode for `NBQA_ISORT`. |
| **FIX_JUPYTER_NBQA_RUFF** | `false` | Option to enable fix mode for `NBQA_RUFF`. |
| **FIX_MARKDOWN_PRETTIER** | `true` | Flag to enable or disable the formatting of Markdown files with Prettier. |
| **FIX_MARKDOWN** | `false` | Option to enable fix mode for `MARKDOWN`. |
| **FIX_POWERSHELL** | `false` | Option to enable fix mode for `POWERSHELL`. |
Expand Down Expand Up @@ -285,6 +289,7 @@ You can configure Super-linter using the following environment variables:
| **JAVA_FILE_NAME** | `sun_checks.xml` | Filename for [Checkstyle configuration](https://checkstyle.sourceforge.io/config.html). Checkstyle embeds several configuration files, such as `sun_checks.xml`, `google_checks.xml` that you can use without providing your own configuration file. |
| **JAVASCRIPT_ES_CONFIG_FILE** | `.eslintrc.yml` | Filename for [ESLint configuration](https://eslint.org/docs/user-guide/configuring#configuration-file-formats) (ex: `.eslintrc.yml`, `.eslintrc.json`) |
| **JSCPD_CONFIG_FILE** | `.jscpd.json` | Filename for JSCPD configuration |
| **JUPYTER_NBQA_CONFIG_FILE** | `.jupyter-nbqa.toml` | Filename for [nbqa configuration](https://nbqa.readthedocs.io/en/latest/configuration.html) |
| **KUBERNETES_KUBECONFORM_OPTIONS** | `null` | Additional arguments to pass to the command-line when running **Kubernetes Kubeconform** (Example: --ignore-missing-schemas) |
| **LINTER_RULES_PATH** | `.github/linters` | Directory for all linter configuration rules. |
| **LOG_FILE** | `super-linter.log` | The filename for outputting logs. Super-linter saves the log file to `${DEFAULT_WORKSPACE}/${LOG_FILE}`. |
Expand Down Expand Up @@ -373,6 +378,12 @@ You can configure Super-linter using the following environment variables:
| **VALIDATE_JSONC_PRETTIER** | `true` | Flag to enable or disable checking the formatting of JSONC and JSON5 files with Prettier. |
| **VALIDATE_JSX** | `true` | Flag to enable or disable the linting process for jsx files (Utilizing: ESLint) |
| **VALIDATE_JSX_PRETTIER** | `true` | Flag to enable or disable checking the formatting of JSX files with Prettier. |
| **VALIDATE_JUPYTER_NBQA_BLACK** | `true` | Flag to enable or disable the linting process of Jupyter Notebooks. (Utilizing: nbqa black) |
| **VALIDATE_JUPYTER_NBQA_FLAKE8** | `true` | Flag to enable or disable the linting process of Jupyter Notebooks. (Utilizing: nbqa flake8) |
| **VALIDATE_JUPYTER_NBQA_ISORT** | `true` | Flag to enable or disable the linting process of Jupyter Notebooks. (Utilizing: nbqa isort) |
| **VALIDATE_JUPYTER_NBQA_MYPY** | `true` | Flag to enable or disable the linting process of Jupyter Notebooks. (Utilizing: nbqa mypy) |
| **VALIDATE_JUPYTER_NBQA_PYLINT** | `true` | Flag to enable or disable the linting process of Jupyter Notebooks. (Utilizing: nbqa pylint) |
| **VALIDATE_JUPYTER_NBQA_RUFF** | `true` | Flag to enable or disable the linting process of Jupyter Notebooks. (Utilizing: nbqa ruff) |
| **VALIDATE_KOTLIN** | `true` | Flag to enable or disable the linting process of the Kotlin language. |
| **VALIDATE_KUBERNETES_KUBECONFORM** | `true` | Flag to enable or disable the linting process of Kubernetes descriptors with Kubeconform |
| **VALIDATE_LATEX** | `true` | Flag to enable or disable the linting process of the LaTeX language. |
Expand Down
Empty file added TEMPLATES/.jupyter-nbqa.toml
View file
Empty file.
1 change: 1 addition & 0 deletions dependencies/python/nbqa.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
nbqa==1.9.0
4 changes: 2 additions & 2 deletions docs/add-new-linter.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ tool, do the following.
violations that the fix mode can automatically fix. Avoid test cases that
fail only because of syntax errors, when possible.
4. Update expected summary reports: `test/data/super-linter-summary`.
5. If the tool supports check-only mode or fix mode, add the `<LANGUGAGE>` to
the `LANGUAGES_WITH_FIX_MODE` array in `test/testUtils.sh`
5. If the tool supports check-only mode or fix mode, add the `<LANGUAGE>` to the
`LANGUAGES_WITH_FIX_MODE` array in `test/testUtils.sh`

## Update the test suite

Expand Down
7 changes: 7 additions & 0 deletions lib/functions/buildFileList.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -457,6 +457,13 @@ BuildFileArrays() {
elif [ "${FILE_TYPE}" == "jsx" ]; then
echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-JSX"
echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-JSX_PRETTIER"
elif [ "${FILE_TYPE}" == "ipynb" ]; then
echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-JUPYTER_NBQA_BLACK"
echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-JUPYTER_NBQA_FLAKE8"
echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-JUPYTER_NBQA_ISORT"
echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-JUPYTER_NBQA_MYPY"
echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-JUPYTER_NBQA_PYLINT"
echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-JUPYTER_NBQA_RUFF"
elif [ "${FILE_TYPE}" == "kt" ] || [ "${FILE_TYPE}" == "kts" ]; then
echo "${FILE}" >>"${FILE_ARRAYS_DIRECTORY_PATH}/file-array-KOTLIN"
elif [ "$FILE_TYPE" == "lua" ]; then
Expand Down
6 changes: 6 additions & 0 deletions lib/functions/linterCommands.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,12 @@ LINTER_COMMANDS_ARRAY_JSONC=(eslint -c "${JAVASCRIPT_ES_LINTER_RULES}" --ext '.j
LINTER_COMMANDS_ARRAY_JSONC_PRETTIER=("${PRETTIER_COMMAND[@]}")
LINTER_COMMANDS_ARRAY_JSX=(eslint -c "${JSX_LINTER_RULES}")
LINTER_COMMANDS_ARRAY_JSX_PRETTIER=("${PRETTIER_COMMAND[@]}")
LINTER_COMMANDS_ARRAY_JUPYTER_NBQA_BLACK=(nbqa black --nbqa-shell)
LINTER_COMMANDS_ARRAY_JUPYTER_NBQA_FLAKE8=(nbqa flake8 --nbqa-shell)
LINTER_COMMANDS_ARRAY_JUPYTER_NBQA_ISORT=(nbqa isort --nbqa-shell)
LINTER_COMMANDS_ARRAY_JUPYTER_NBQA_MYPY=(nbqa mypy --nbqa-shell)
LINTER_COMMANDS_ARRAY_JUPYTER_NBQA_PYLINT=(nbqa pylint --nbqa-shell)
LINTER_COMMANDS_ARRAY_JUPYTER_NBQA_RUFF=(nbqa ruff --nbqa-shell)
LINTER_COMMANDS_ARRAY_KOTLIN=(ktlint "{/}")
LINTER_COMMANDS_ARRAY_KUBERNETES_KUBECONFORM=(kubeconform -strict)
if [ "${KUBERNETES_KUBECONFORM_OPTIONS}" != "null" ] && [ -n "${KUBERNETES_KUBECONFORM_OPTIONS}" ]; then
Expand Down
6 changes: 6 additions & 0 deletions lib/globals/languages.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ LANGUAGE_ARRAY=('ANSIBLE' 'ARM' 'BASH' 'BASH_EXEC' 'CHECKOV' 'CLANG_FORMAT'
"JSONC_PRETTIER"
'JSX'
"JSX_PRETTIER"
"JUPYTER_NBQA_BLACK"
"JUPYTER_NBQA_FLAKE8"
"JUPYTER_NBQA_ISORT"
"JUPYTER_NBQA_MYPY"
"JUPYTER_NBQA_PYLINT"
"JUPYTER_NBQA_RUFF"
'KUBERNETES_KUBECONFORM' 'KOTLIN' 'LATEX' 'LUA' 'MARKDOWN'
"MARKDOWN_PRETTIER"
'NATURAL_LANGUAGE' 'OPENAPI' 'PERL' 'PHP_BUILTIN' 'PHP_PHPCS' 'PHP_PHPSTAN'
Expand Down
3 changes: 3 additions & 0 deletions lib/globals/linterCommandsOptions.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ JAVASCRIPT_PRETTIER_CHECK_ONLY_MODE_OPTIONS=("${PRETTIER_CHECK_ONLY_MODE_OPTIONS
JSON_PRETTIER_CHECK_ONLY_MODE_OPTIONS=("${PRETTIER_CHECK_ONLY_MODE_OPTIONS[@]}")
JSONC_PRETTIER_CHECK_ONLY_MODE_OPTIONS=("${PRETTIER_CHECK_ONLY_MODE_OPTIONS[@]}")
JSX_PRETTIER_CHECK_ONLY_MODE_OPTIONS=("${PRETTIER_CHECK_ONLY_MODE_OPTIONS[@]}")
JUPYTER_NBQA_BLACK_CHECK_ONLY_MODE_OPTIONS=(--diff --check)
JUPYTER_NBQA_ISORT_CHECK_ONLY_MODE_OPTIONS=(--diff --check)
MARKDOWN_PRETTIER_CHECK_ONLY_MODE_OPTIONS=("${PRETTIER_CHECK_ONLY_MODE_OPTIONS[@]}")
PYTHON_BLACK_CHECK_ONLY_MODE_OPTIONS=(--diff --check)
PYTHON_ISORT_CHECK_ONLY_MODE_OPTIONS=(--diff --check)
Expand Down Expand Up @@ -68,6 +70,7 @@ JSONC_FIX_MODE_OPTIONS=("${ESLINT_FIX_MODE_OPTIONS[@]}")
JSONC_PRETTIER_FIX_MODE_OPTIONS=("${PRETTIER_FIX_MODE_OPTIONS[@]}")
JSX_FIX_MODE_OPTIONS=("${ESLINT_FIX_MODE_OPTIONS[@]}")
JSX_PRETTIER_FIX_MODE_OPTIONS=("${PRETTIER_FIX_MODE_OPTIONS[@]}")
JUPYTER_NBQA_RUFF_FIX_MODE_OPTIONS=(--fix)
MARKDOWN_FIX_MODE_OPTIONS=(--fix)
MARKDOWN_PRETTIER_FIX_MODE_OPTIONS=("${PRETTIER_FIX_MODE_OPTIONS[@]}")
NATURAL_LANGUAGE_FIX_MODE_OPTIONS=(--fix)
Expand Down
Loading

0 comments on commit 469f30e

Please sign in to comment.