Adds support for running EVAL with different scripting engines

[rjd15372]

In this PR we re-implement the EVAL commands (EVAL, EVALSHA, SCRIPT LOAD, etc...) to use the scripting engine infrastructure introduced in 6adef8e. This allows EVAL to run scripts using different scripting engines.

The Lua scripting engine implementation code was moved into its own subdirectory src/lua.

This new implementation generalizes the module API for implementing scripting engines to work with both FUNCTION and EVAL commands.

Module API changes include:

• Rename of callback ValkeyModuleScriptingEngineCreateFunctionsLibraryFunc to ValkeyModuleScriptingEngineCompileCodeFunc.
• Addition of a new enum enum ValkeyModuleScriptingEngineSubsystemType to specify the scripting engine subsystem (EVAL, or FUNCTION, or both).
• In most callbacks was added a new parameter with the type ValkeyModuleScriptingEngineSubsystemType.
• New callback specific for EVAL ValkeyModuleScriptingEngineResetEvalEnvFunc.
• New API function ValkeyModuleScriptingEngineExecutionState (*ValkeyModule_GetFunctionExecutionState)(ValkeyModuleScriptingEngineServerRuntimeCtx *server_ctx) that is used by scripting engines to query the server about the execution state of the script that is running.

Fixes #1261
Follow-up of #1277

[codecov]

Codecov Report

Attention: Patch coverage is 62.62443% with 413 lines in your changes missing coverage. Please review.

Project coverage is 71.06%. Comparing base (d72a97e) to head (806893f).

Files with missing lines	Patch %	Lines
src/lua/debug_lua.c	39.27%	354 Missing ⚠️
src/lua/engine_lua.c	88.27%	19 Missing ⚠️
src/eval.c	92.18%	15 Missing ⚠️
src/lua/script_lua.c	70.00%	9 Missing ⚠️
src/server.c	25.00%	9 Missing ⚠️
src/module.c	20.00%	4 Missing ⚠️
src/scripting_engine.c	92.59%	2 Missing ⚠️
src/lua/function_lua.c	97.67%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #1497      +/-   ##
============================================
+ Coverage     70.99%   71.06%   +0.07%     
============================================
  Files           121      123       +2     
  Lines         65175    65334     +159     
============================================
+ Hits          46269    46432     +163     
+ Misses        18906    18902       -4     

Files with missing lines	Coverage Δ
src/defrag.c	91.67% <ø> (+1.74%)	⬆️
src/functions.c	94.31% <100.00%> (-0.19%)	⬇️
src/lazyfree.c	86.11% <100.00%> (ø)
src/server.h	100.00% <ø> (ø)
src/lua/function_lua.c	98.69% <97.67%> (ø)
src/scripting_engine.c	76.85% <92.59%> (+2.08%)	⬆️
src/module.c	9.60% <20.00%> (+<0.01%)	⬆️
src/lua/script_lua.c	89.74% <70.00%> (ø)
src/server.c	87.44% <25.00%> (-0.18%)	⬇️
src/eval.c	88.95% <92.18%> (+31.88%)	⬆️
... and 2 more

... and 10 files with indirect coverage changes

[rjd15372]

@zuiderkwast this is ready for review.

[zuiderkwast]

It's a huge change.

The module API changes look good to me.

It's impossible to proof-read the logic but I guess we have pretty good test coverage. Breaking up eval.c into smaller units is good idea, just hard to review. Can you summarize which code was just moved unchanged and which code was written, so it's possible to tell what to review carefully and what not?

[rjd15372]

[...]. Can you summarize which code was just moved unchanged and which code was written, so it's possible to tell what to review carefully and what not?

@zuiderkwast sure.

• Important changes in eval.c:
  • All ldb* functions (lua debug support) were just moved to lua/debug_lua.[hc] so no need to do a thorough review.
  • Old scriptingRelease is now called evalRelease and has significantly changed to support the async release of resources for any kind of scripting engine.
  • evalRegisterNewScript function.
  • evalGenericCommand function
  • eval.c is the solely responsible for managing the cache for scripts indexed by the SHA1 hash of the script body. The scripting engine is not aware of this cache. So eval.c implements the SCRIPT * and EVAL* commands, by offloading the compilation and execution of scripts to the scripting engine.
  • Lua engine specific code, like initialization, compilation of scripts, and execution of scripts, was moved into lua/script_lua.c
• lua/engine_lua.c is the entry point of the lua scripting engine for supporting both EVAL and FCALL commands. It has the scripting engine callbacks implementation.
• scripting_engine.c is the scripting engines core that implements the management of the scripting engines that are registered in the server. It was improved with the support for EVAL.
• Module API (valkeymodule.h) changes:
  • New enum ValkeyModuleScriptingEngineSubsystemType to specify the script execution subsystem (either EVAL, or FUNCTION, or both). Almost all scripting engine API callbacks have a parameter with this enum type.
  • The callback ValkeyModuleScriptingEngineCreateFunctionsLibraryFunc was renamed to ValkeyModuleScriptingEngineCompileCodeFunc.
  • New callback ValkeyModuleScriptingEngineResetEvalEnvFunc to support the reset of the EVAL runtime context.
  • New Module API function ValkeyModule_GetFunctionExecutionState that returns the execution state of a script. This is used by scripting engines to know whether the script was killed. Scripting engines will have to poll this function periodically.