[ci] Updating lmi-dist ci tests for rubikon-engine

[maaquib]

Description

Following changes have been made

• Removes bitsandbytes test case, since we no longer support it
• Adds speculative decoding test case
• Adds tests for Gemma-7B and Santacoder

• If this change is a backward incompatible change, why must this change be made? - We're dropping support for bitsandbytes quantization
• Interesting edge cases to note here - N/A

[lanking520]

could you also remove the speculative decoding test in vLLM?

[maaquib]

Test run: https://github.com/deepjavalibrary/djl-serving/actions/runs/8382943920

[davidthomas426]

Is it ok for our CI tests to use HF model ids directly rather than s3 links?

Other than that, LGTM.

[lanking520]

Is it ok for our CI tests to use HF model ids directly rather than s3 links?

Other than that, LGTM.

There is a problem with HF id. Firstly it is not very stable. Secondly, sometimes the download speed is very slow and cause timeout (flaky tests)

[lanking520]

@maaquib could you also just replace the lmi-dist handler with the new one in this PR, so we can just cover in CI testing?

[lanking520]

could you also add tests for vLLM as well, just to serve as baseline?

[maaquib]

could you also add tests for vLLM as well, just to serve as baseline?

Added gemma and starcoder2 to vllm

[maaquib]

Build: https://github.com/deepjavalibrary/djl-serving/actions/runs/8399526079
Rolling Batch: https://github.com/deepjavalibrary/djl-serving/actions/runs/8399595919