chore: supporting parallelism with rayon

[gvozdvmozgu]

No description provided.

[gvozdvmozgu]

PS > Measure-Command { sqruff.exe lint . *> $nul }        


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 1
Milliseconds      : 194
Ticks             : 11949992
TotalDays         : 1.38310092592593E-05
TotalHours        : 0.000331944222222222
TotalMinutes      : 0.0199166533333333
TotalSeconds      : 1.1949992
TotalMilliseconds : 1194.9992

test files were created using

@echo off
SETLOCAL EnableDelayedExpansion

REM Create a new directory called SqlFolder
mkdir SqlFolder

REM Create the test.sql file with SQL command
echo SELECT * FROM test; > test.sql

REM Duplicate the file 315 times in the SqlFolder
FOR /L %%G IN (1,1,315) DO (
    copy test.sql SqlFolder\test_%%G.sql > NUL
)

REM Clean up the original test.sql
del test.sql

echo Files created successfully.
ENDLOCAL

[benfdking]

@gvozdvmozgu did we prove this had the desired effect on performance? I can see your measurement but not really "impact" per se because I have no control?

[gvozdvmozgu]

We have managed to improve performance in single-threaded mode, so I think we can delay introducing multithreading. Since we are currently adding rules that share data through memory in RuleContext, it seems that adding multithreading could potentially cause problems.

[gvozdvmozgu]

ready for review

[benfdking]

The benchmark goes from 1.19s on my end to 338ms. That is nice!