Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: Rubocop rules don't always capture linter errors #1432

Open
brand-it opened this issue Jan 16, 2025 · 0 comments
Open

Bug: Rubocop rules don't always capture linter errors #1432

brand-it opened this issue Jan 16, 2025 · 0 comments

Comments

@brand-it
Copy link

brand-it commented Jan 16, 2025 •
edited
Loading

What happened?

There is a linter error with rubocop in a Pull Request

> bundle exec rubocop
Offenses:

spec/lib/config/sitemap_spec.rb:381:9: C: RSpec/NestedGroups: Maximum example group nesting exceeded [5/4].
        context 'when batteries category is a child of gears' do                                                                                                       ^^

These errors are generated by the rspec rubocop. However when running QLTY check it never captures them. Above is the rubocop using bundle exec rubocop. Running qlty check

qlty check
✔ No issues

Image

The best guess as to what is happening is that it is not seeing the rubocop-rspec gem.

Generated Gemfile from qlty

source 'https://rubygems.org'
gem 'rubocop', '1.60.1', require: false
gem 'rubocop-factory_bot', '2.25.1', require: false
gem 'rubocop-graphql', '1.5.0', require: false
gem 'rubocop-performance', '1.20.2', require: false
gem 'rubocop-rails', '2.23.1', require: false
gem 'rubocop-rake', '0.6.0', require: false
gem 'rubocop-rspec', '2.26.1', require: false

Source Gem file from the repo. Removed all the extra gems and only copy pasted the code that was relevent.

gem 'rubocop', '1.60.1', require: false
  gem 'rubocop-factory_bot', '2.24.0', require: false
  gem 'rubocop-graphql', '1.4.0', require: false
  gem 'rubocop-performance', '1.19.1', require: false
  gem 'rubocop-rails', '2.22.1', require: false
  gem 'rubocop-rake', '0.6.0', require: false
  gem 'rubocop-rspec', '2.25.0', require: false

qlty.toml

# This file was automatically generated by `qlty init`.
# You can modify it to suit your needs.
# We recommend you to commit this file to your repository.
#
# This configuration is used by both Qlty CLI and Qlty Cloud.
#
#     Qlty CLI -- Code quality toolkit for developers
#     Qlty Cloud -- Fully automated Code Health Platform
#
# Try Qlty Cloud: https://qlty.ai
#
# For a guide to configuration, visit https://qlty.ai/docs/config
# Or for a full reference, visit https://qlty.ai/docs/qlty-toml
config_version = "0"

exclude_patterns = [
  "*_min.*",
  "*-min.*",
  "*.min.*",
  "**/*.d.ts",
  "**/.yarn/**",
  "**/bower_components/**",
  "**/build/**",
  "**/cache/**",
  "**/config/**",
  "**/db/**",
  "**/deps/**",
  "**/dist/**",
  "**/extern/**",
  "**/external/**",
  "**/generated/**",
  "**/Godeps/**",
  "**/gradlew/**",
  "**/mvnw/**",
  "**/node_modules/**",
  "**/protos/**",
  "**/seed/**",
  "**/target/**",
  "**/testdata/**",
  "**/vendor/**",
  "**/assets/**",
]

test_patterns = [
  "**/test/**",
  "**/spec/**",
  "**/*.test.*",
  "**/*.spec.*",
  "**/*_test.*",
  "**/*_spec.*",
  "**/test_*.*",
  "**/spec_*.*",
]

[[source]]
name = "default"
repository = "https://github.com/qltyai/plugins.git"
tag = "v0.125.0"


[[plugin]]
name = "actionlint"

[[plugin]]
name = "brakeman"

[[plugin]]
name = "coffeelint"

[[plugin]]
name = "dotenv-linter"

[[plugin]]
name = "eslint"
version = "8.57.0"
package_file = "package.json"
package_filters = ["eslint", "jest", "prettier"]

[[plugin]]
name = "hadolint"

[[plugin]]
name = "markdownlint"

[[plugin]]
name = "osv-scanner"

[[plugin]]
name = "prettier"
version = "3.2.5"
package_file = "package.json"
package_filters = ["prettier"]

[[plugin]]
name = "ripgrep"

[[plugin]]
name = "rubocop"
version = "1.60.1"
package_file = "Gemfile"
package_filters = ["rubocop"]
config_files = ["it_behaves_like.rb"]

[[plugin]]
name = "shellcheck"

[[plugin]]
name = "shfmt"

[[plugin]]
name = "trufflehog"

[[plugin]]
name = "yamllint"

What did you expect to happen?

Failed to capture linter error with rubocop-rspec

Can you reproduce the problem?

Yes, consistently

CLI Version

qlty 0.472.0 macos-arm64 (e6ba9f1 2025-01-14)

Relevant log output

Relevant log output 
`qty-cli.log`
-01-16T16:59:17Z INFO [T1] qlty (4.1 MB): Executing command: qlty check
2025-01-16T16:59:17Z INFO [T1] qlty_analysis::git::upstream (4.3 MB): Inferred upstream: Some("origin/develop")
2025-01-16T16:59:17Z INFO [T5] qlty_check::planner::plugin (5.2 MB): Found 2 workspace_entries for plugin brakeman
2025-01-16T16:59:17Z INFO [T10] qlty_check::planner::plugin (5.3 MB): Found 2 workspace_entries for plugin trufflehog
2025-01-16T16:59:17Z INFO [T5] qlty_check::planner::plugin (5.3 MB): Found 2 workspace_entries for plugin ripgrep
2025-01-16T16:59:17Z INFO [T13] qlty_check::planner::plugin (5.4 MB): Found 2 workspace_entries for plugin rubocop
2025-01-16T16:59:17Z INFO [T1] qlty_check::planner (5.0 MB): Found 12 CPUs, setting max threads to 12
2025-01-16T16:59:17Z INFO [T1] qlty_check::planner (5.1 MB): Planned 5 invocations (0 cache hits) in 0.36s
2025-01-16T16:59:17Z INFO [T24] qlty_check::executor (5.1 MB): Installing tool ruby
2025-01-16T16:59:17Z INFO [T16] qlty_check::executor (5.1 MB): Installing tool rubocop
2025-01-16T16:59:17Z INFO [T27] qlty_check::executor (5.1 MB): Installing tool ripgrep
2025-01-16T16:59:17Z INFO [T18] qlty_check::executor (5.1 MB): Installing tool trufflehog
2025-01-16T16:59:17Z INFO [T20] qlty_check::executor (5.1 MB): Installing tool brakeman
2025-01-16T16:59:17Z INFO [T1] qlty_check::executor (5.1 MB): All 5 tasks complete in 0.00s
2025-01-16T16:59:17Z INFO [T1] qlty_check::executor (5.1 MB): Staged 2 workspace entries and 4 config files in 0.05s
2025-01-16T16:59:17Z INFO [T1] qlty_check::executor (5.0 MB): Running 4 invocations
2025-01-16T16:59:17Z INFO [T46] qlty_check::executor::driver (5.2 MB): TrkQ4B: Completed ripgrep/lint-fixme in 0.039s (exit 1)
2025-01-16T16:59:18Z INFO [T45] qlty_check::executor::driver (5.2 MB): q44wda: Completed trufflehog/lint in 1.286s (exit 0)
2025-01-16T16:59:19Z INFO [T44] qlty_check::executor::driver (5.2 MB): x5zf2Q: Completed rubocop/lint in 2.335s (exit 0)
2025-01-16T17:01:55Z INFO [T40] qlty_check::executor::driver (5.4 MB): a0wKU3: Completed brakeman/lint in 158.294s (exit 3)
2025-01-16T17:01:57Z INFO [T65] qlty_check::executor::driver (5.5 MB): 33h8OR: Completed rubocop/format in 2.135s (exit 0)
2025-01-16T17:01:57Z INFO [T1] qlty_check::executor (5.4 MB): All 5 invocation tasks complete in 160.44s
2025-01-16T17:01:57Z INFO [T1] qlty_check::executor (5.4 MB): Executed 5 invocations in 160.49s
2025-01-16T17:01:57Z INFO [T1] qlty_check::processor (6.1 MB): Processed results from 5 invocations in 0.00s
2025-01-16T17:01:57Z INFO [T1] qlty (4.5 MB): Command executed successfully in 160.91s: qlty check
2025-01-16T17:03:00Z INFO [T1] qlty (4.1 MB): Executing command: qlty upgrade
2025-01-16T17:03:05Z INFO [T1] qlty (4.1 MB): Executing command: qlty --version
2025-01-16T17:03:14Z INFO [T1] qlty (4.1 MB): Executing command: qlty check
2025-01-16T17:03:14Z INFO [T1] qlty_analysis::git::upstream (4.3 MB): Inferred upstream: Some("origin/develop")
2025-01-16T17:03:14Z INFO [T5] qlty_check::planner::plugin (5.2 MB): Found 2 workspace_entries for plugin brakeman
2025-01-16T17:03:14Z INFO [T8] qlty_check::planner::plugin (5.3 MB): Found 2 workspace_entries for plugin trufflehog
2025-01-16T17:03:14Z INFO [T6] qlty_check::planner::plugin (5.3 MB): Found 2 workspace_entries for plugin ripgrep
2025-01-16T17:03:14Z INFO [T4] qlty_check::planner::plugin (5.3 MB): Found 2 workspace_entries for plugin rubocop
2025-01-16T17:03:14Z INFO [T1] qlty_check::planner (5.0 MB): Found 12 CPUs, setting max threads to 12
2025-01-16T17:03:14Z INFO [T1] qlty_check::planner (5.1 MB): Planned 5 invocations (0 cache hits) in 0.43s
2025-01-16T17:03:14Z INFO [T17] qlty_check::executor (5.1 MB): Installing tool brakeman
2025-01-16T17:03:14Z INFO [T22] qlty_check::executor (5.1 MB): Installing tool ripgrep
2025-01-16T17:03:14Z INFO [T18] qlty_check::executor (5.1 MB): Installing tool trufflehog
2025-01-16T17:03:14Z INFO [T16] qlty_check::executor (5.1 MB): Installing tool rubocop
2025-01-16T17:03:14Z INFO [T25] qlty_check::executor (5.1 MB): Installing tool ruby
2025-01-16T17:03:14Z INFO [T1] qlty_check::executor (5.1 MB): All 5 tasks complete in 0.00s
2025-01-16T17:03:14Z INFO [T1] qlty_check::executor (5.1 MB): Staged 2 workspace entries and 4 config files in 0.05s
2025-01-16T17:03:14Z INFO [T1] qlty_check::executor (5.0 MB): Running 4 invocations
2025-01-16T17:03:14Z INFO [T38] qlty_check::executor::driver (5.2 MB): tjxFBy: Completed ripgrep/lint-fixme in 0.034s (exit 1)
2025-01-16T17:03:16Z INFO [T46] qlty_check::executor::driver (5.2 MB): x3QRZX: Completed trufflehog/lint in 1.261s (exit 0)
2025-01-16T17:03:16Z INFO [T42] qlty_check::executor::driver (5.2 MB): YVcdoq: Completed rubocop/lint in 1.580s (exit 0)
2025-01-16T17:04:49Z INFO [T1] qlty (4.1 MB): Executing command: qlty init --no-upgrade-check --no
2025-01-16T17:04:49Z INFO [T1] qlty (4.1 MB): Command executed successfully in 0.00s: qlty init --no-upgrade-check --no
2025-01-16T17:04:54Z INFO [T1] qlty (4.1 MB): Executing command: qlty init
2025-01-16T17:04:54Z INFO [T1] qlty (4.2 MB): Command executed successfully in 0.38s: qlty init
2025-01-16T17:05:54Z INFO [T40] qlty_check::executor::driver (5.4 MB): GDsM1g: Completed brakeman/lint in 159.938s (exit 3)
2025-01-16T17:05:56Z INFO [T72] qlty_check::executor::driver (5.5 MB): pqTMb6: Completed rubocop/format in 2.175s (exit 0)
2025-01-16T17:05:56Z INFO [T1] qlty_check::executor (5.5 MB): All 5 invocation tasks complete in 162.12s
2025-01-16T17:05:56Z INFO [T1] qlty_check::executor (5.5 MB): Executed 5 invocations in 162.17s
2025-01-16T17:05:56Z INFO [T1] qlty_check::processor (6.1 MB): Processed results from 5 invocations in 0.00s
2025-01-16T17:05:56Z INFO [T1] qlty (4.5 MB): Command executed successfully in 162.64s: qlty check
2025-01-16T17:06:24Z INFO [T1] qlty (4.1 MB): Executing command: qlty plugins plugins upgrade
2025-01-16T17:06:30Z INFO [T1] qlty (4.1 MB): Executing command: qlty plugins upgrade
2025-01-16T17:06:34Z INFO [T1] qlty (4.1 MB): Executing command: qlty plugins upgrade rubocop
2025-01-16T17:06:34Z INFO [T1] qlty (4.1 MB): Command executed successfully in 0.03s: qlty plugins upgrade rubocop
2025-01-16T17:11:02Z INFO [T1] qlty (4.1 MB): Executing command: qlty check
2025-01-16T17:11:03Z INFO [T1] qlty_analysis::git::upstream (4.3 MB): Inferred upstream: Some("origin/develop")
2025-01-16T17:11:03Z INFO [T5] qlty_check::planner::plugin (5.2 MB): Found 2 workspace_entries for plugin brakeman
2025-01-16T17:11:03Z INFO [T9] qlty_check::planner::plugin (5.2 MB): Found 2 workspace_entries for plugin ripgrep
2025-01-16T17:11:03Z INFO [T12] qlty_check::planner::plugin (5.2 MB): Found 2 workspace_entries for plugin rubocop
2025-01-16T17:11:03Z INFO [T11] qlty_check::planner::plugin (5.5 MB): Found 2 workspace_entries for plugin trufflehog
2025-01-16T17:11:03Z INFO [T1] qlty_check::planner (5.0 MB): Found 12 CPUs, setting max threads to 12
2025-01-16T17:11:03Z INFO [T1] qlty_check::planner (5.1 MB): Planned 5 invocations (0 cache hits) in 0.33s
2025-01-16T17:11:03Z INFO [T20] qlty_check::executor (5.1 MB): Installing tool brakeman
2025-01-16T17:11:03Z INFO [T22] qlty_check::executor (5.1 MB): Installing tool trufflehog
2025-01-16T17:11:03Z INFO [T24] qlty_check::executor (5.1 MB): Installing tool ripgrep
2025-01-16T17:11:03Z INFO [T23] qlty_check::executor (5.1 MB): Installing tool ruby
2025-01-16T17:11:03Z INFO [T16] qlty_check::executor (5.1 MB): Installing tool rubocop
2025-01-16T17:11:03Z INFO [T16] qlty_check::tool (5.1 MB): Setting up tool rubocop@Some("bundled"). Logging to /Users/brandit/.qlty/cache/tools/rubocop/bundled-0efae866a994-install.log
2025-01-16T17:11:08Z INFO [T16] qlty_check::tool::ruby::gemfile (5.1 MB): ruby -S bundle list list stdout: Gems included by the bundle:
  * activesupport (8.0.1)
  * ast (2.4.2)
  * base64 (0.2.0)
  * benchmark (0.4.0)
  * bigdecimal (3.1.9)
  * concurrent-ruby (1.3.5)
  * connection_pool (2.5.0)
  * drb (2.2.1)
  * i18n (1.14.6)
  * json (2.9.1)
  * language_server-protocol (3.17.0.3)
  * logger (1.6.5)
  * minitest (5.25.4)
  * parallel (1.26.3)
  * parser (3.3.7.0)
  * racc (1.8.1)
  * rack (3.1.8)
  * rainbow (3.1.1)
  * regexp_parser (2.10.0)
  * rexml (3.4.0)
  * rubocop (1.60.1)
  * rubocop-ast (1.37.0)
  * rubocop-capybara (2.21.0)
  * rubocop-factory_bot (2.24.0)
  * rubocop-graphql (1.4.0)
  * rubocop-performance (1.19.1)
  * rubocop-rails (2.22.1)
  * rubocop-rake (0.6.0)
  * rubocop-rspec (2.25.0)
  * ruby-progressbar (1.13.0)
  * securerandom (0.4.1)
  * tzinfo (2.0.6)
  * unicode-display_width (2.6.0)
  * uri (1.0.2)
Use `bundle info` to print more detailed information about a gem

2025-01-16T17:11:08Z INFO [T16] qlty_check::tool::ruby::gemfile (5.1 MB): ruby -S bundle list list stderr: 
2025-01-16T17:11:08Z INFO [T16] qlty_check::tool (5.1 MB): Set up rubocop@Some("bundled") in 5.30s
2025-01-16T17:11:08Z INFO [T1] qlty_check::executor (5.1 MB): All 5 tasks complete in 5.31s
2025-01-16T17:11:08Z INFO [T1] qlty_check::executor (5.1 MB): Staged 2 workspace entries and 4 config files in 0.06s
2025-01-16T17:11:08Z INFO [T1] qlty_check::executor (5.0 MB): Running 4 invocations
2025-01-16T17:11:08Z INFO [T37] qlty_check::executor::driver (5.2 MB): mGyvhV: Completed ripgrep/lint-fixme in 0.031s (exit 1)
2025-01-16T17:11:09Z INFO [T46] qlty_check::executor::driver (5.2 MB): 3DinDP: Completed trufflehog/lint in 1.184s (exit 0)
2025-01-16T17:11:10Z INFO [T40] qlty_check::executor::driver (5.2 MB): tSEPsa: Completed rubocop/lint in 2.088s (exit 0)
2025-01-16T17:13:34Z INFO [T44] qlty_check::executor::driver (5.4 MB): 1iZDmC: Completed brakeman/lint in 145.609s (exit 3)
2025-01-16T17:13:36Z INFO [T72] qlty_check::executor::driver (5.4 MB): AaX1Fq: Completed rubocop/format in 2.053s (exit 0)
2025-01-16T17:13:36Z INFO [T1] qlty_check::executor (5.4 MB): All 5 invocation tasks complete in 147.67s
2025-01-16T17:13:36Z INFO [T1] qlty_check::executor (5.5 MB): Executed 5 invocations in 147.73s
2025-01-16T17:13:36Z INFO [T1] qlty_check::processor (6.1 MB): Processed results from 5 invocations in 0.00s
2025-01-16T17:13:36Z INFO [T1] qlty (4.4 MB): Command executed successfully in 153.43s: qlty check
2025-01-16T17:14:57Z INFO [T1] qlty (4.1 MB): Executing command: qlty check
2025-01-16T17:14:57Z INFO [T1] qlty_analysis::git::upstream (4.3 MB): Inferred upstream: Some("origin/develop")
2025-01-16T17:14:57Z INFO [T6] qlty_check::planner::plugin (5.2 MB): Found 2 workspace_entries for plugin brakeman
2025-01-16T17:14:57Z INFO [T8] qlty_check::planner::plugin (5.2 MB): Found 2 workspace_entries for plugin trufflehog
2025-01-16T17:14:57Z INFO [T11] qlty_check::planner::plugin (5.3 MB): Found 2 workspace_entries for plugin ripgrep
2025-01-16T17:14:57Z INFO [T4] qlty_check::planner::plugin (5.4 MB): Found 2 workspace_entries for plugin rubocop
2025-01-16T17:14:57Z INFO [T1] qlty_check::planner (5.0 MB): Found 12 CPUs, setting max threads to 12
2025-01-16T17:14:57Z INFO [T1] qlty_check::planner (5.1 MB): Planned 5 invocations (0 cache hits) in 0.28s
2025-01-16T17:14:57Z INFO [T25] qlty_check::executor (5.1 MB): Installing tool rubocop
2025-01-16T17:14:57Z INFO [T26] qlty_check::executor (5.1 MB): Installing tool ruby
2025-01-16T17:14:57Z INFO [T17] qlty_check::executor (5.1 MB): Installing tool trufflehog
2025-01-16T17:14:57Z INFO [T18] qlty_check::executor (5.1 MB): Installing tool brakeman
2025-01-16T17:14:57Z INFO [T20] qlty_check::executor (5.1 MB): Installing tool ripgrep
2025-01-16T17:14:57Z INFO [T1] qlty_check::executor (5.1 MB): All 5 tasks complete in 0.00s
2025-01-16T17:14:57Z INFO [T1] qlty_check::executor (5.1 MB): Staged 2 workspace entries and 4 config files in 0.05s
2025-01-16T17:14:57Z INFO [T1] qlty_check::executor (5.0 MB): Running 4 invocations
2025-01-16T17:14:57Z INFO [T46] qlty_check::executor::driver (5.2 MB): TMXgYO: Completed ripgrep/lint-fixme in 0.032s (exit 1)
2025-01-16T17:14:59Z INFO [T43] qlty_check::executor::driver (5.2 MB): Td0NIA: Completed trufflehog/lint in 1.153s (exit 0)
2025-01-16T17:14:59Z INFO [T37] qlty_check::executor::driver (5.2 MB): UzhH2s: Completed rubocop/lint in 1.543s (exit 0)
2025-01-16T17:17:31Z INFO [T40] qlty_check::executor::driver (5.4 MB): o1hfE1: Completed brakeman/lint in 153.192s (exit 3)
2025-01-16T17:17:33Z INFO [T70] qlty_check::executor::driver (5.5 MB): A8l3HF: Completed rubocop/format in 2.169s (exit 0)
2025-01-16T17:17:33Z INFO [T1] qlty_check::executor (5.5 MB): All 5 invocation tasks complete in 155.37s
2025-01-16T17:17:33Z INFO [T1] qlty_check::executor (5.4 MB): Executed 5 invocations in 155.42s
2025-01-16T17:17:33Z INFO [T1] qlty_check::processor (6.1 MB): Processed results from 5 invocations in 0.00s
2025-01-16T17:17:33Z INFO [T1] qlty (4.5 MB): Command executed successfully in 155.74s: qlty check
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@brand-it